Resilient IRP Function Developer Guide
Resilient IRP Function Developer Guide
Chapter 1. Objective.............................................................................................. 1
Chapter 2. Overview.............................................................................................. 3
Resilient SOAR Platform.............................................................................................................................. 3
Integration architecture...............................................................................................................................4
Function processor...................................................................................................................................... 4
Resilient Circuits.......................................................................................................................................... 5
Development Overview................................................................................................................................5
Developer website....................................................................................................................................... 6
Chapter 4. Functions..............................................................................................9
Creating the Resilient platform components.............................................................................................. 9
Naming conventions.............................................................................................................................10
Input field considerations.................................................................................................................... 10
Post-process script considerations..................................................................................................... 11
Writing the function processor.................................................................................................................. 12
Modifying the selftest module................................................................................................................... 13
Running the function..................................................................................................................................13
Testing the function................................................................................................................................... 14
Packaging your function............................................................................................................................ 15
Creating your package..........................................................................................................................16
Documenting your package................................................................................................................. 16
Compressing and sharing your package..............................................................................................17
Publishing your function...................................................................................................................... 17
Packaging Resilient components only...................................................................................................... 18
iii
iv
Chapter 1. Objective
This guide provides the information to integrate the Resilient Security Orchestration, Automation and
Response (SOAR) Platform with your organization’s existing security and IT investments using the
functions feature. Integrations makes security alerts instantly actionable, provides valuable intelligence
and incident context, and enables adaptive response to complex cyber threats.
This guide is intended for programmers, testers, architects and technical managers interested in
developing and testing functions with the Resilient platform. It assumes a general understanding of the
Resilient platform, message-oriented middleware (MOM) systems, and a knowledge of writing scripts in
Python.
You should familiarize yourself with the Resilient architecture and the relevant Resilient features, as
described in the following sections, before designing and writing functions.
The Resilient platform contains an interactive Rest API browser that allows you to access the Resilient
REST API and try out any endpoint on the system. When logged into the Resilient platform, click on your
account name at the top right and select Help/Contact. Here you can access the complete API Reference
guide, including schemas for all of the JSON sent and received by the API, and the interactive Rest API.
Function processor
Resilient functions send data to external programs called function processors. The function processor can
then perform integration work, for example:
• Performing a lookup, such as for information about a user or machine in an asset database, and
returning the data values found.
• Searching SIEM logs, such as for an IP address, a URL or a server name, and returning a list of events.
• Sending a file attachment to a sandbox for analysis, producing a report and a collection of observables.
• Opening a ticket in an ITSM system with a type, name and description, and returning the ticket-ID.
• Triggering an external action and then returning results for use in workflows, tasks and other decisions.
In the Resilient platform, a rule is configured on an incident, an artifact or other object. When the rule
fires, it runs a workflow that could have multiple steps. Those steps can include functions that send input
parameters to the function processor using a message destination, receive the results, and use the
results to update the Resilient incident, to decide the direction of subsequent workflow steps or in a
variety of other ways.
Resilient Circuits
Resilient Circuits is a Python circuits framework that automatically manages authenticating and
connecting to the STOMP connection and REST API in the Resilient platform. It simplifies creating
integrations by allowing you to focus on writing the behavior logic. It is the preferred method for writing
integrations.
You can use Resilient Circuits to manage your functions as well as other types of integrations. Each
integration has its own section in the app configuration file. This file stores information about the Resilient
platform, such as user credentials, as well as variables for your functions.
Resilient Circuits use the Resilient helper module, which is a Python library to facilitate easy use of the
Rest API.
Development Overview
Before you write a function, you must understand the purpose of the function, the inputs the function
needs to perform its activity, the expected results, and the actions or decisions to be made based on the
results.
The following procedure provides a high-level overview of the development process. The subsequent
sections in this guide provide the details.
1. Make sure you have Resilient Circuits framework installed and configured on a Resilient Integration
Server, as described in the Integration Server Guide.
2. Log into the Resilient platform and create one or more functions and associated components, such as
message destination and workflow.
3. Using Resilient Circuits and codegen, write the function processor, which is the code for the
integration.
TIP: If you have access to functions that are similar to the one you wish to create, use that function as
a template to save time.
Chapter 2. Overview 5
4. Use the pip install command to deploy the function to the Resilient platform and test the function by
triggering the workflow and checking the results. If you make any changes to any of the function’s
components, repackage your function processor using codegen.
NOTE: The Resilient Circuits codegen command automatically accesses the latest changes to the
Resilient functions and associated components.
5. Write a document that provides information on deploying and using the function.
6. Package your function, all its components and the document then distribute it to other Resilient
administrators. They can deploy the package to any Resilient platform at the same or later version as
your test platform.
7. Optionally, share your package with other developers in the IBM Resilient developer community.
Developer website
The Resilient developer web site contains the core Resilient helper module and Resilient Circuit
packages, additional integration packages, documentation and examples. The links are provided below.
• IBM Resilient Developer website. Provides overview information and access to various areas of
development, such as developing playbooks and publishing integrations.
• IBM Resilient Github. Provides access to library modules, community-provided extensions, example
scripts, and developer documentation. It also contains the Resilient Circuits and helper module
packages. This is also accessible from the developer website reference page.
• IBM X-Force App Exchange. Provides access to the Resilient community apps on IBM X-Force.
• Releases. Lists the apps by Resilient platform release. You can also download from this page.
In addition, you can view the Resilient product guides, such as the Playbook Designer Guide, and
additional information in the IBM Knowledge Center. (This link takes you to a page where you can choose
the version of the Resilient platform.)
The procedures to install and configure the Resilient Circuits framework on a Resilient integration server
are included in the Integration Server Guide. The guide also provides the values for the Resilient platform
within the configuration file, named app.config by default.
In a development environment, you may find it necessary to override one or more values from your
app.config file when running Resilient Circuits. For example, you may want to temporarily run with the log
level set to DEBUG. To accomplish this, run Resilient Circuits with:
You can also use optional parameters to run the application when the values being overridden are
required and missing from the config file.
For a complete list of optional arguments for overrides, run:
NOTE: If on a Windows system and you edit the file with Notepad, ensure that you save it as type All Files
to avoid a new extension being added to the filename, and use UTF-8 encoding.
Before you write a function, you must understand the purpose of the function, the inputs the function
needs to perform its activity, the expected results, and the actions or decisions to be made based on the
results.
If you wish, you can create multiple functions related to the same integration and include all of them in
one package.
TIP: If you have access to functions that are similar to the one you wish to create, use that function as a
template to save time.
Naming conventions
The name of your integration package should reflect the type of integration as well as be unique. Here are
a few examples:
• LDAP Query
• McAfee ePO Integration
• Elasticsearch Query
• Cisco Get Domains
The names of the components in your package should reflect your package name, as follows:
• Use a unique prefix for your function and components, such as the product or provider name. Using
LDAP Query as example, the components would be as follows. When you enter the name of a
component, the API name replaces spaces with underscores.
– Function name: ldap_query
– Message destination name: ldap_query
– Input field: ldap_query_inputname
– Data table: ldap_query_datatable
– Custom field: ldap_query_fieldname
You do not need to add a prefix to commonly used fields, such as incident_id, task_id, attachment_id,
and artifact_id.
• If you package rules that are meant as examples, you should use the prefix, Example, such as:
“Example: Query LDAP”
• If you package workflows that are meant as examples, you should also use the prefix, Example. In your
function, you use the API name of the workflow, so an example of a workflow name would be:
example_ldap_query_workflow_name
• Decoding this input field may require the removal of control characters:
mpa = {}.fromkeys(range(32))
dict = json.loads(test_details.translate(mpa))
log.info("incidentId: "+dict['incidentId'])
• Binary format, such as an attachment, is not supported. If a function needs the content of an
attachment, do not send it through input fields. Instead, the integration needs to call the resilient_client
of its super class to get the file content. For example:
resilientClient = self.rest_client();
"""
Example of call:
/incidents/2095/artifacts/13/contents
"""
api = "/incidents/{}/artifacts/{}/contents".format(incidentID, artifactID)
response = resilientClient.get_content(api)
inputs.jira_description =
"Incident types: {}\nNIST Attack Vectors: {}\n\nAdditional Information: {}"
.format(incident.incident_type_ids, incident.nist_attack_vectors,
incident.description)
You should test all required input parameters for a valid entry. You can also enforce this when defining
the input field (Requirement: Always). Pre-process scripts are needed for field assignment.
incident_id = kwargs.get('incident_id')
if not incident_id:
raise FunctionError('incident_id is required')
Chapter 4. Functions 11
– results.matched_list['file'] should be used for the next level item
• Failed post-process scripts may cause a workflow to remain in the running state. For a given incident,
click the Actions button then select Action Status to verify the successful completion of a function.
Click the Actions button then select Workflow status to terminate any workflow with failed actions.
• Use dict.get("key") when testing whether data exists for that key. The dict["key"] may cause the script
to fail if "key" does not exist.
NOTE: To see additional options, such as packaging multiple functions, see Packaging your function.
The following example packages a function called lookup_mode_by_id and a workflow called
lookup_model into a package called fn_model.
The result is a directory containing the essential files for an installable Python package that implements
the function or functions specified. Within this package, the function code itself is a simple script, such as
the one below, that you can edit to add your integration logic. This script is a component with a method,
decorated with @function()that tells the Resilient Circuits framework how to call it.
"""Function implementation"""
import logging
from resilient_circuits import ResilientComponent, function, StatusMessage, FunctionResult,
FunctionError
class FunctionComponent(ResilientComponent):
"""Component that implements Resilient function 'lookup_model_by_id"""
@function("lookup_model_by_id")
def _lookup_model_by_id_function(self, event, *args, **kwargs):
"""Function: Lookup more information about the specified ID"""
try:
# Get the function parameters:
model_id = kwargs.get("model_id") # text
log = logging.getLogger(__name__)
results = {
"value": "xyz"
}
First, the function gets its parameters (model_id, in this case). The boilerplate implementation logs the
values for ease of debugging, although you may want to change that if it’s too noisy.
At any stage during the function’s processing, you can enter yield StatusMessage ("..."), which
provides a status message that will display to the Resilient user in the Action Status dialog. If your
function might run for several seconds or minutes, this can be a useful way to show progress.
The results are a Python dictionary, containing named values that will be available in the workflow
output and post-process script. If possible, your functions should return a small ‘results’ dictionary with
one or a few named values. In this case, you should include enough documentation to help your users
understand how to find and use these results in a custom workflow.
resilient-circuits selftest
If you have multiple packages, you can run selftest on one or more specific integrations:
Run the integration code from the command-line, as follows. The framework reads your configuration file,
connects to the Resilient platform, finds and loads all the installed components, then subscribes to the
message destination for each function processor component. Leave it running; when a function is
invoked, the code handles it.
resilient-circuits run
Chapter 4. Functions 13
2018-04-12 16:10:31,814 INFO [app] Configuration file: /home/integration/.resilient/app.config
2018-04-12 16:10:31,816 INFO [app] Resilient server: myserver.resilientsystems.com
2018-04-12 16:10:31,817 INFO [app] Resilient user: [email protected]
2018-04-12 16:10:31,818 INFO [app] Resilient org: PartnerLab
2018-04-12 16:10:31,818 INFO [app] Logging Level: INFO
2018-04-12 16:10:38,075 INFO [component_loader] Loading 1 components
2018-04-12 16:10:38,076 INFO [component_loader]
'fn_model.components.lookup_model_by_id.FunctionComponent' loading
2018-04-12 16:10:38,089 INFO [stomp_component] Connect to myserver.resilientsystems.com:65001
2018-04-12 16:10:38,090 INFO [actions_component]
'fn_model.components.lookup_model_by_id.FunctionComponent' function 'lookup_model_by_id'
registered to 'function_example'
2018-04-12 16:10:38,091 INFO [app] Components loaded
2018-04-12 16:10:38,094 INFO [app] App Started
2018-04-12 16:10:38,196 INFO [actions_component] STOMP attempting to connect
2018-04-12 16:10:38,198 INFO [stomp_component] Connect to Stomp...
2018-04-12 16:10:38,199 INFO [client] Connecting to myserver.resilientsystems.com:65001 ...
2018-04-12 16:10:38,825 INFO [client] Connection established
2018-04-12 16:10:39,090 INFO [client] Connected to stomp broker
[session=ID:ip-1-2-3-252.srv.resilientsystems.com-35733-1523282148180-5:243, version=1.2]
2018-04-12 16:10:39,092 INFO [stomp_component] Connected to failover:(ssl://
myserver.resilientsystems.com:65001)?maxReconnectAttempts=1,startupMaxReconnectAttempts=1
2018-04-12 16:10:39,093 INFO [stomp_component] Client HB: 0 Server HB: 15000
2018-04-12 16:10:39,094 INFO [stomp_component] No Client heartbeats will be sent
2018-04-12 16:10:39,095 INFO [stomp_component] Requested heartbeats from server.
2018-04-12 16:10:39,098 INFO [actions_component] STOMP connected.
2018-04-12 16:10:39,205 INFO [actions_component] Subscribe to message destination
'function_example'
2018-04-12 16:10:39,206 INFO [stomp_component] Subscribe to message destination
actions.201.function_example
For this example, the workflow runs on an artifact, so the rule is a menu-item rule that appears as an
action in each incident’s artifact action menu, accessible from the […] button.
The incident’s Action Status menu shows the status of each rule, which can be pending (queued for
delivery to the function processor), processed successfully, or with an error. In the following example, the
action completed with success and included a status message.
If you need to make any changes to any of the function’s elements, repackage your function processor
using codegen.
Consider implementing the actual integration code to the 3rd party system in a separate module from
that which manages the Resilient framework. This separation of logic allows you to test the 3rd party
integration separate from the code as part of Resilient Circuits.
components/
<function_code>.py
<3rd_party_integration_code>.py
tests/
test_<function_code>.py
test_<3rd_party_integration_code>.py
You can also use the Resilient Circuits sefltest command as a troubleshooting aid.
Chapter 4. Functions 15
Creating your package
The following is the basic command you use to create a Python package containing your function:
When you are ready to share your package, you need to include all the related components, such as the
rules, workflows, message destinations and any custom fields and data tables. To do this, use arguments
such as --workflow, --rule, --m (message destination) and --datatable. Use --h to list all the arguments.
The following command includes a workflow and rule, and specifies the export file:
NOTE: The --exportfile is needed only if you do not wish to use the most recent changes.
If using a rule or script name that contains spaces, use quotation marks around the name and be aware
that the name is case sensitive. For example:
Optionally, you can specify multiple functions to codegen into the package by specifying each function
name with the --function parameter. To see a list of functions on your integration system, enter the
following command. Codegen returns the file path to the function, which you can open using a text editor
such as vi or nano.
resilient-circuits codegen
Available functions:
ldap_search
ldap_disable_user
NOTE: The codegen -o argument, shown as follows, should only be used if you need to create a single
Python file with a specific name.
If you have previously created a package and you need to recreate it, you can use the reload option. This
is useful when you wish to add more components without specifying the components you added the last
time you created the package. To use the reload option, you must specify the name of the package to be
reloaded. The following example reloads your package with all previous components then adds a new
rule.
This command also saves the existing customize.py as customize-yyyymmdd-hhmmss.bak in the util
directory of the package.
NOTE: You cannot reload a package generated with a version of Resilient Circuits prior to V31. If you wish
to use reload, you need to rename your customize.py then generate a new package using Resilient
Circuits V31 or later. You can then use reload for the new package.
name='fn_example',
version='1.0.0',
license='<MIT, APACHE, etc.>',
author='<developer or company name>',
author_email='<developer or company email>',
url='<company url>',
description="<Simple description>",
long_description="<Longer description as necessary>",
install_requires=[
<list of required libraries to include when installing this package>
]
The resulting file is added to the dist/ folder using the package name and version, such as dist/
fn_example-1.0.0.tar.gz.
This file can be further wrapped in a zip file for submission to the IBM App Exchange. See Publishing your
function.
If you wish to distribute the package to another Resilient platform, note the following considerations:
• Copy the file to the Resilient integration server that is used with the Resilient platform.
• Make sure that the Resilient platform is at the same or greater version as the Resilient platform used to
create the package.
As described in the Integration Server Guide, a Resilient administrator uses the Resilient Circuits
customize utility to update the Resilient platform with any missing message destinations, function
definitions, and other design elements. The command is:
resilient-circuits customize
Each installed package that includes a “customize” entry-point is called and returns a collection of
customizations described in Resilient JSON format. These are used to update the Resilient platform.
The customize command only rebuilds those components that were referenced from the codegen
command line. For example, if you indicated --workflow, then workflows are rebuilt during the customize
process.
Chapter 4. Functions 17
The IBM Security X-Force App Exchange allows you to make your function available to others in the
Resilient community. You have the option to update the function as needed. For more information on
submission requirements, see the Publishing Integrations page.
The Resilient Community Apps on Github also allows you to share your function source code with others.
Members can copy, modify and enhance your function using the pull request mechanism. See the
Resilient Community Apps page for a list of apps, with developer information at the bottom of the web
page.
You can choose to submit to one or both locations.
The following sections provide advice and recommendations to consider when creating your function.
Data flow
Functions are blocking until results are returned. Returning FunctionError()aborts the result of a
workflow.
All functions are stateless. No persistence of data is retained between function calls.
Error handling
The high level code should be covered with try / except / finally blocks. Any exception should describe the
issue for Resilient Action Status log. The finally block should be used to perform any connection closing,
temporary file cleanup, and so on.
Do not use yield FunctionError() within try/except. It causes the message destination message to
remain stuck. Each time Resilient Circuits is restarted, the message is attempted again. The better
solution is call raise FunctionError() and then the except block can perform the yield. The yield
FunctionError() also interferes with finally in try/except/finally. The fix is to change yield to
raise.
Logging
Log information for debugging purposes. Sensitive information should never be logged, but can be
obscured. For example, [email protected] can become us***@example.com.
Data results
Use the function, FunctionResult(), to return the JSON for post-process script processing. A sample result
should be included in the function's description field.
As much as possible, return top-level items or lists of items (or combinations):
{
'item1': 'result1',
'item2': 'result2'
}
or
{ 'entries':
[
{'item1': 'result1', 'item2': 'result2'},
{'item1': 'result3', 'item2': 'result4'}
]
}
Links back to the integration application are supported by creating a Resilient custom field, such as a
TextArea with rich text enabled. Add the custom field to an incident's Summary section for easy access.
In the workflow's post-processing script, add code similar to this:
Data tables are a convenient way to return and display row results. There are a number of considerations
to bear in mind with their use:
• Data tables are global to an incident (as are all custom fields). This means that the table rows are not
unique to a specific object, such as an artifact.
• In order to deal with the shared use of tables, add two columns to a table to identify the search
argument (such as artifact.value) and a timestamp when the results were returned.
Rich text
Rich text fields may contain HTML markup. This format may be undesirable for the target integration
system. A method such as the one below can strip off the HTML elements, preserving some of the new
line format:
def _cleanHtml(htmlFragment):
'''
Resilient textarea fields return html fragments. This routine will remove
the html and insert any code within <div></div> with a linefeed
:param htmlFragment:
:return: cleaned up code
'''
Temporary files
For integrations that read from or write to files, one method to create temporary files is to use Python's
tempfile:
import tempfile
...
with tempfile.NamedTemporaryFile() as temp_file:
In situations when filehandles are used, consider using StringIO instead. This avoids any OS interaction
with files:
import StringIO
...
with StringIO(filedata) as temp_file:
If using Python to access the Resilient directly, the following script shows how to authenticate using an
API key account and make an API call to get a list of constants from the Resilient REST API.
"""
This script uses API Key Authentication with Resilient platform to
get list of constants from a server. Modify as needed.
Prerequsites:
* Python 2.7 or higher
def main():
"""
* Resilient version must be 33.0 or higher
* API Key ID and Secret are known. Sample values shown in script.
* API Key has the permissions needed for the operation
"""
key_id = "c5b0d84c-73d6-4dca-a9fb-e98870d62826"
key_secret = "e2-5k4IT3WlVH9mfSl_g5ALBdtmVEaRsh4EEoFUs2HI"
server = "9.70.195.31"
resource = "rest/const"
url = "https://{0}/{1}".format(server, resource)
headers = {"Content-Type": "application/json; charset=UTF-8"}
auth = HTTPBasicAuth(key_id, key_secret)
if __name__ == "__main__":
main()