Introduction To Vault Core Architecture
Introduction To Vault Core Architecture
The copyright in the contents of this ebook belong to Thought Machine Group Limited (unless
expressly stated to belong to another person).
The information contained in this ebook is confidential and may not be copied, distributed, published,
or disclosed without the express prior written permission of Thought Machine Group Limited.
This ebook is intended for informational purposes only and should not replace independent
professional advice.
No representation (whether express or implied) is made as to, and no reliance should be placed on, any
information, including estimates, projections, targets and opinions, set out in this ebook, and no liability
whatsoever is accepted as to any errors, omissions or misstatements contained in this ebook, and,
accordingly, neither Thought Machine Group Limited nor any of its officers or employees accepts any
liability whatsoever arising directly or indirectly from the use of this ebook for any purpose.
The provision by Thought Machine Group Limited of any service or product described in this ebook is
subject to contract.
In transcribing and translating this course, effort has been made to ensure that it is an accurate
reflection of the course content, however there may be some differences in the texts and in the
case of any such difference, the English language version shall prevail.
Contents 2
Course Description 3
Course Objectives 3
Course Agenda 3
Banking on Vault 17
Vault API’s 23
Product Engine 28
Operations Dashboard 29
Workflow Engine 31
Summary 37
Course Objectives
● Explain how financial products and business processes are implemented in Vault.
● Understand and model data flows between Vault and third-party systems for common retail
banking use cases, including reporting and analytics.
Course Agenda
● Introduction to components and functions of Vault.
3. You will have improved your understanding of the Role of Vault Core.
1. Review the Documentation Hub for additional resources and API information or reference
material.
It also maintains customer accounts. Fund movements that occur on an account are known as
postings. Postings move in and out of the accounts on Vault and are executed and recorded in the
Vault Ledger. Vault core is a scalable, open architecture platform. It’s built on a distributed
microservices architecture, utilising containers and Kubernetes.
Vault Core is also a system of record and set of tools for the management of products, accounts, and
postings. Products in Vault are implemented using a design pattern called Smart Contracts. These
enable the development of a client’s custom product portfolio to be completed much faster compared
to typical legacy platforms. Smart Contracts are written in a subset of Python that executes within a
sandboxed execution environment.
Smart Contracts can be written by the client, their delivery partner or Thought Machine. A payment
processing capability can be connected directly to the Kernel in order to process payments via the
Postings API.
Operational processes related to the core such as manual customer onboarding, manual account
management etc. can be optionally implemented in Vault using the component which is called the
Workflow Engine. Workflows are written in YAML.
The simplification and automation of business processes using YAML allows clients to speed up
development while remaining flexible. Note that Vault Workflows are designed to replace back office
user orchestration, as opposed to channel integration with Vault.
The optional component, Operations Dashboard is a Vault App and provides a user interface to interact
with the core banking platform. It can be used by the bank’s employees to interact with Vault core and
workflows.
The first is Products. Vault is responsible for the manufacturing and operation of banking products, and
these are expressed as Smart Contracts. What do we mean by this?
● Smart Contracts: We have introduced these earlier in this session. Vault Smart Contracts are
written in Python Code and include the financial logic for your products.
● Manufacturing: You create the Smart Contract by writing Python Code, in this code you will
include all of the financial logic, such as the minimum balance or maximum spending rules.
● Operation: Once you have written and tested your code, you will upload it to Vault. Vault will
then run the code you’ve written, and the relevant microservice will carry out the relevant tasks
as dictated by your code.
The second entity is Accounts. For now, let’s focus on Customer Accounts:
● An account in Vault is an instance of a Product. This means simply that an account in Vault is
backed by a specific version of your Smart Contract and the code in this Smart Contract is run
when you open an account in Vault.
● Accounts within Vault have balances, and these balances are used in order to allocate the
positions a customer holds. Does the customer owe the bank money or does the bank owe the
customer money? You will be able to determine this by looking at the difference balance
addresses on your accounts in Vault.
Third, we have postings. Vault is responsible for generating and recording movements of funds
between accounts on Vault:
● Regarding generating postings: You are able to generate postings within Vault in 2 ways: first is
using our Postings API the Postings API will be used in payments lifecycles to action the
transfer of funds. You will make postings across various accounts and balances using our
postings API. The second method is using the Contracts API.
● The Contracts API is able to generate postings from the Smart Contract when needed, for
example on an interest or fee accrual event the Smart Contract can autonomously make
postings to facilitate the accrual of interest.
● Vault Core’s Ledger will record the postings that have been successfully made against an
account. The ledger itself provides data in real time and can capture multiple types of assets
and currencies with balances that can be configured into any structure.
● Promote self-service, product personalisation and the ability to easily make changes, test and
learn from it.
● And Avoid heavily customised platforms that are unable to upgrade without Vendor
involvement.
Vault addresses this by separating the platform into two layers: The configuration layer and the
platform layer.
The configuration layer consists of the Chart of Accounts and Smart Contracts. The optional
component workflows can also make up part of this configuration layer.
This layer is specific to each client and is intended to be customisable by the client, using the training,
documentation and tooling provided by Thought Machine. What this all means is the client can build
financial products according to their own requirements, and build their internal accounting structure
according to their requirements as well.
The second layer is the platform layer. The platform layer is common to all of our clients, with the
services available to all clients. We provide changes and new features regularly through minor and
major releases. The platform layer is built by Thought Machine only and clients use APIs to upload their
configuration layer content to the platform layer and interact with the Vault system.
Legacy architectures will generally have a large amount of data and configuration that needs to be
kept in sync, if it falls out of sync then incorrect data could be used in reporting or calculations. Let’s
explore an example:
1. In this legacy core example we see that there are two core banking systems, each storing
Customer Data, Customer Balance Data and data relevant for group accounting purposes.
2. These cores will provide data to the Operational Data Store through batch file extracts
performed on a regular (usually daily) basis.
3. Each time an extract of data is performed, the bank will need to complete reconciliation
between the source system and the destination system.
4. This becomes quite a complex set of tasks when you factor in the amount of data that is
repeated across systems - as shown in this diagram we have customer data in 3 systems,
balance data in 4 systems and accounting data in 5 different systems.
To keep all of this repeated data in sync we need to perform regular batches. In a legacy core, we will
need to trade-off between how fresh data is and the amount of work required to both complete the
batch process itself and any reconciliation required. We cannot constantly perform batch extracts of
data because this will take up valuable resources that are required to perform BaU processing activity
during the day.
However, not refreshing data enough will significantly increase the risk of data inconsistency across
systems. The tradeoff between the two options will typically lead to batches being done from a core’s
reporting database once or twice a day, with a team doing a mix of manual and automated checks on
the data to ensure it is successfully extracted.
Each extraction is generally driven on a batch schedule that does not know about previous or
subsequent steps. For example we might extract incomplete data from our Core to our middle tier, and
subsequently to reporting databases in the Group Services area of the bank if the core banking system
extract failed to complete without errors.
This example illustrates the problem - customer data has been incorrectly extracted from the core and
is now wrong in the CMS, and also wrong in the banking group services area too.
Reconciliation will involve validating a range of different characteristics of the data we have extracted
from the core. This will include carrying out both automated and manual checks such as:
This reconciliation will be done through the use of scripts, reconciliation applications and some manual
checking of generated reports.
Data is not always available in real time to downstream systems from a legacy core, and in a legacy
architecture. This is for two reasons:
● And ideally we should complete the reconciliation on data before it is used so that we know
data is consistent across systems.
As a consequence, the bank cannot do real time analytics or have a full up to date understanding of
what is going on within the bank’s systems
Legacy migrations are often the single most challenging and daunting element of a core banking
transformation programme, with the effort underpinned by a number of challenges.
● First, Batch-Based Migration File Transfer: Each batch load demands a significant overhead
that limits how frequently and long data can practically be maintained.
● Second, Multiple Load Touchpoints: This is where data is required to be loaded in a specific
order this is achieved through slow waterfall sequencing of data.
● Third, Complex Load Orchestration: Load process involves multiple steps/ touchpoints,
increasing overall effort and risk of error at each stage.
● Fourth, Batch-Based Migration File Outputs: Event outputs generated after the entire load
completes, making it challenging to know when an individual record has been loaded.
● Fifth, there is Limited Testing Of Scheduled Behaviour. Challenges in testing scheduled account
behaviour without creating complex test frameworks or letting real time pass.
● Finally, Core capabilities limit potential migration strategies, pushing banks towards
unfavourable Big-bang approaches.
Legacy Core Banking systems typically provide no code solutions for building financial products, with a
fixed set of schedules for end of day processing activities and other scheduled events.
This leads to products which are largely the same with little differentiation between different banks,
and difficulties in personalising products for the bank’s requirements or the customer’s requirements.
● We can “solve” the problem of having many sources of truth of our data, by using batches and
reconciliation.
● We can solve the freshness of data (i.e. the risk that someone will take out a loan in one
system, and then in another one before we propagate the change) by having more frequent
batches.
● As the payment systems are plumbed into the core banking system, we can solve the lack of
real-time data (notification, reporting etc) by wrapping them in middleware and notifying
customers from there (another stateful source).
● We can solve the poor performance of the CBS by introducing data caches and good data
practices in our mid-tier.
● All of these work-arounds add cost, and manual complexity. Many of them introduce / require
changes to the core banking systems which are “high energy changes” and happen at a low
frequency because it’s a complex task to change the core .
● We can work around differences in operational support using manual effort and we must
document different processes for modifying different records on different systems.
To resolve some of the duplication of data and services we see in legacy core banking system
architectures. Vault is a single core that can run any financial product with only configuration layer
(smart contract) changes which solves many of the data reconciliation issues.
Allowing each product to control their own schedules and parameters (no global batches and no core
banking deployments required) eases the running of many various products.
In addition, because Vault has a low code solution to developing financial products banks are able to
differentiate themselves and their offerings from each other in order to service their own specific
requirements (such as products that replicate back book products exactly) or their customers
requirements (such as products that meet customer’s specific needs through personalisation).
In order to avoid specific components for specific payments integrations, Vault uses a single postings /
payments API for all integrations (card, payments etc). Any card or payment processor can hook into
Vault in the same manner regardless of jurisdiction or scheme.
Within Vault, payment devices are independent from the product implementations. Your physical debit
card is not dependent on your account logic. One payment device can therefore be associated with
multiple different accounts and multiple different types of products.
For example, you could have one payment card associated with both a debit account and a credit
card account and the customer can switch between the two products to use at will.
Regarding backup and recovery, with its Microservices based Cloud Native solution, Vault is designed
to run active-active with automatic DR across multiple data centres. In future, this will include
active-active with automatic DR across multiple regions with CockroachDB.
You will no longer have to make passive replicas of the database and have to have a regular manual
backup procedure.
Regarding Migration, Vault Core’s migration product offering has been designed to de-risk and simplify
legacy core migrations. With six key properties:
● One, Data can be easily migrated and maintained at an account level using API entry points
that support real-time sync.
● Two, we have native dependency management. You can set configurable dependencies in the
API request, enabling data to be sent in any order and loaded automatically when
dependencies are met.
● Third, the Load process can be fully automated from the point of sending API requests to
receiving streamed events.
● Fourth, Real-time events streaming from Vault enables faster reconciliations and provide
immediate data to support downstream business processes.
● Fifth, you can do time series testing - that is you can Fast-forward account activities as part of
an out the box test to quickly prove back book product behaviour against a known baseline.
● Finally, innovative migration strategies are enabled by Vault. Production parallel run and other
migration designs are enabled by Vault’s event-based streaming architecture.
First, we have the Core API. The Core API is used for integration of external applications like Channels,
CRM, Bank Operator UIs and other systems. It allows you to carry out calls to resources within Vault
such as:
● Account Management - you can manage customer accounts that sit on the Vault system. This
includes the ability to open new accounts, change the status of existing accounts, update the
instance parameters for accounts and close accounts on Vault.
● Product Management - you can upload and change smart contract versions from the Core API.
● Restrictions - you can restrict a customer account in the event of a card being lost, an account
being suspected of malicious activity or any other reason to restrict an account's postings. You
can also restrict an account from being closed or updated, effectively freezing it.
● Smart Contract Simulation: You can use the simulation part of the core api to simulate smart
contract code as part of your testing.
● Internal Accounts - here you can manage your Vault aspect of your chart of accounts.
● Roles management - here you can manage employee roles and service tokens.
● Flags - you can add flags to accounts and customer entities, flags allow you to impact smart
contract behaviour with custom reasons. Have a look at our Smart Contracts Fundamentals
session to learn more.
Now we have the Postings API. The Postings API is responsible for managing all financial movements
on accounts held within Vault, journeys you’ll use it for include:
● Release of funds.
Now we introduce the Streaming API. The streaming API provides native support for real time events
streamed out of Vault according to when something is written to the Vault Database.
Events include: customer Account change events, Accounting events, Posting Balance updates,
Customer entity status changes, Payment Device events, External Workflow request/response, smart
contract activation, configuration changes, and versioning changes.
Let’s talk through this set of Vault Core components highlighting some key aspects, including additional
APIs.
First, let’s highlight the Vault Core part of this diagram. Within Vault Core there are a significant number
of microservices carrying tasks out such as:
● Managing the processing of a posting that has been made and much more.
These microservices work together to make up Vault Core. Within Vault Core we also have the Smart
Contracts which are the banking products you will be using on the system. This works as part of the
product engine. The Vault Postings Ledger will be used in order to capture any postings that are made
to Vault.
You will interact with the Vault Core component using our APIs. We’ve so far introduced the Core API,
Postings API and Streaming API previously. Let’s highlight the next API the Data Loader API.
The Data Loader API is used in migrations to Vault of data from a legacy core banking system. It allows
you to asynchronously load data into Vault via Kafka.
Vault Core can also integrate into our optional workflow engine. You are able to start a workflow from a
smart contract if need be, and this is represented in the diagram as “Workflow Initiation”.
Let’s talk a little more on what will integrate into the Postings API. The postings API will be integrated
into by the Payments Processor or Card Processor, depending on the payments journey you are
planning to undertake. Bear in mind that all payments and card processing systems will hook into Vault
in the same manner.
Both of these integrations will need to come from outside of Vault Core, ie. meaning Vault core is not
responsible for the activities these systems will carry out. You may be interested to explore the use of
Vault Payments in the future. Outside of the payments processing aspect, you will have the actual
payments or card scheme and the fraud engine(s) within your bank.
Highlighting the streaming API aspect - you will be able to integrate systems such as your Data
Warehouse system(s) into the streaming API to consume events from the vault as they are streamed
out. We recommend you consume all events that are streamed out of Vault to build a complete picture
in your data warehouse.
Let’s move on to highlight the workflows engine - workflows can be used for automated or manual
operational back office processes within your bank. You can integrate into the workflows engine using
the Workflow API which is a RESTful synchronous API which is used to start workflows, progress
workflows, cancel workflows and upload new workflows. Workflows can also be used with the
Operations Dashboard for manual approvals, data entry and more. The Workflows Engine can integrate
with an external workflow event router to communicate with external systems.
Moving on to the Operations Dashboard, this is a web based application that allows back office users
to carry out various operational tasks using a GUI, integrated into our Core API and our Workflows
Engine. The Operations Dashboard has full role based access control for back office users.
In this diagram we have provided a box ‘middle tier’, this is where you will integrate Vault into your
wider ecosystem - you may decide to use a domain connector to align data to your enterprise
architecture standards. Connected to the middle tier systems will be your customer channels, including
any customer notifications services, the authentication for customers, the systems of engagement for
your customers and more.
Provide the efficient management of a large complex product set at scale. So if you have products with
multiple price points or products in different business lines, we aim to intuitively manage these within
Vault.
● We want to reduce friction across the product life cycle, crafting a solution that enables low
overhead when it comes to day-to-day management of product sets in the core system.
● We are also aiming to scale the configurability of smart contracts, with the ability to have
multiple products stemming from a small set of smart contracts, meaning that only a small
number of smart contracts are required to run a large product book efficiently.
We’ll be providing more detail on the Vault Product Engine in a further course.
The Operations Dashboard is a browser-based application that enables interaction with the Vault Core
Platform Layer and Configuration Layer components.
The Operations Dashboard provides tools for servicing the accounts and posting records held in the
Platform Layer, allowing for journeys such as investigating account changes, postings, schedules,
customer entity changes and more.
It also provides configuration management tools to upload and manage new smart contracts, and to
upload and manage workflow definitions. Using Workflows you can enhance the capabilities of the
Operations Dashboard, with the ability for operations users to use workflow GUIs on the operations
dashboard for manual interaction.
Let’s highlight a little more on Customer Entity and Configuration Management interaction.
Let’s start by asking what a Workflow actually is within Vault.The workflows engine is used to execute
operational processes that can orchestrate Vault APIs and external integrated systems. We can
highlight five key characteristics of a Workflow in Vault:
● Transparency: Workflows can formally define and visualise back office operational processes
including decisioning logic, integrations and data transformation. You will create your
operational process as a workflow, writing the steps required in that process in order to go from
start to finish. By viewing your workflow in Vault, you effectively are viewing the steps of your
operational process.
● Simple to create: Workflows are written in YAML, a simple markup language. However, you also
have the ability to define advanced data transformation with Starlark, a dialect of Python which
is designed to be more secure than native Python.
● Workflows can also be Autonomous, that is they can be triggered and advanced automatically.
They also have the ability to have manual interaction from back office users where needed.
● And finally workflows are an orchestration tool Workflows have access to all available public
Vault APIs, enabling processes to be built around any Vault component.
Vault Workflows are designed to replace back office operations orchestration, as opposed to channel
integration with Vault. Consequently they have not, and will not, be tested for concurrent or high
throughput use cases.
We can segment the processes workflows enable into three broad categories:
Integrated Processes.You are able to orchestrate processes interacting with both Vault APIs and
integrated external systems to enable cross-system orchestration in back office operations.
Orchestrating events as part of an operational process in the back office, e.g. account opening and
closure processes.
Manual Operations:
Workflows can have both automated and manual steps. Workflows are natively integrated into the
Operations Dashboard with Role Based Access Control.
The Operations Dashboard and the Core API are typically only used during proof-of-concepts or
ad-hoc testing, Most banks will want to integrate deployment of Smart Contracts and Workflows into
their CI/CD pipelines. For that, they should use the Vault Configuration Layer Utilities.
This diagram shows a simple push bank transfer from a customer to a corresponding account in
another bank. Here we see that a customer has initiated a payment from an interaction with a channel
by making a payment out request.
The payments processor will consume this request from the channel and determine that it needs to
make a call to the Postings API. It will make a posting request via the kafka based posting api which will
then be consumed and processed. The posting instruction batch will consist of a posting instruction to
debit money from a customer account and credit money to the internal scheme settlement account.
Once the posting has been successfully added to the ledger, an event is streamed out on the Postings
API as a response and it is consumed by the payments processor. The payments processor will then
communicate with the scheme to complete the payment.
Outbound Push Payments can also use an authorisation model, where amount to be debited from a
customer is first ring fenced, and then based on scheme response, accepted - settle OR rejected -
reverse ringfence
This diagram shows a simple card payment example (that is a pull payment). The card scheme will
send an authorisation message into the card processor which will consume the request. This will
determine that it needs to send a posting to Vault and will construct the posting instruction batch.
The posting instruction batch will then be sent to Vault using the Postings API, and funds will be
authorised from the customer’s available funds and moved into a pending outgoing phase.
Once the auth has been successfully made, a response will be sent back to the card processor via the
postings API and this will be able to communicate with the scheme.
Later on when it’s time to settle the payment, the scheme will send a settlement message to the card
processor which then sends a posting instruction batch to the vault with a settlement posting
instruction type. This then zeros out the pending outgoing phase and will move this money into the
internal scheme settlement account for that particular scheme. The response is then sent back to the
card processor who will communicate with the scheme to finish the transaction.
Here we have a credit risk management process. Vault Core will have accounts making transactions
and stream out events on kafka which are consumed into a credit risk database.
A credit risk engine will use data in the credit risk database to determine risk scores for customers.
When it’s determined that it is time to update the customer’s account parameters (perhaps changing
the credit limit on a credit card) you will use the core API to update the smart contract parameters. This
is an iterative and in theory never ending process.
In this diagram we show what Vault is responsible for when it comes to data analytics. The key
message here is that Vault streams out data on a real time basis but Vault does not actually do the
analytics. We stream our posting instructions, balances, account events and more.
You will then consume these events and capture them into a general use database. You will then
transform data for your use case - in this example for accounting use cases. In this example we then
move data into a general ledger database for use by our general ledger application. Vault does not do
data analytics innately - you will use our data for these purposes.
We have then walked through the key types of data you would store within Vault, and highlighted some
of the key resources involved within Vault - it is important to highlight that we cover this in more detail
in a later session: Vault Key Resources and Data Model as part of the Fundamentals pathway.
Thanks again for your time and interest in Vault Core. Your next step should be to complete the rest of
your fundamentals training to gain a rounded view of the Vault Core Banking system, and if you have
questions please contact us at Thought Machine.