HOW TO BUILD AN

END TO END BI SOLUTION

TAKE THE FAST TRACK - BUILD IT IN 1 WEEK!

If you’re in charge of planning and/or implementing

an analytics solution for your company there is a

ton of things you must consider. We’ve mapped it all

out to help you make sure that you haven’t missed

anything.

FAST TRACK

If you’re in a great hurry to start pulling initial insights

right away, consider taking the “Fast Track” - here

we share with you some useful tips on how to

launch your BI project adopting the “Lean approach”,

evolving it into a more sophisticated one as you go.

 BI STRATEGY

You need to start with a plan or a “BI Strategy”. Consider your BI strategy as a roadmap; It should chart out the
optimal route you need to take to get your data management solution from its current state to the vision you
have for it. Here’s what you need to consider:

DAY 1

1 Report and analytics needs 4 Historical data saved as aggregations

Data can be consumed in many ways; governed Consider the historical depth required for analytics.

dashboards, ad-hoc analysis, AB testing, real time Define what can be summed up to reflect several

analytics, etc. dimensions and what needs to be analyzed on a

Define 1-2 major reporting needs. If you already raw data level.

have an existing solution simply replicate it in your In order to choose the right technological solution it

new BI. is crucial to evaluate the above, But implementation

can be done on a later phase.

FAST TRACK

2 Industry KPI's
5 BI Clients
There are standard KPI’s and metrics for most

industries today, for example sales, return on Consider who will be using your BI solution-

investment, profit margin etc. Use industry partners,customers,high- level executives, analysts,

benchmarks to validate and expand your own KPIs. sales representatives, account and marketing

Start with the 3-4 most important KPI’s managers etc..

Select 1 or 2 clients to begin with , usually starting

3 Custom KPIs with the department that is expected to adopt the BI

first.
Unique business derivatives and professional

systems require tailor made KPI’s to reflect business

specific measurements

If the data needed to calculate the KPIs is

not complete at the source system, consider

implementing these at a later phase

 MAP DATA SOURCES

Mapping data sources is the process of selecting the sources of your data and filtering the relevant data from

each source. In order to map the data required, consider the following inputs - Existing reports, management

requests, industry standard KPI’s and of course your vision!

at this point it is advantageous to start from the end-goal and ask yourself: Ultimately, what reports should my BI

be able to supply?

DAY 2

3 External data
1 Core data
Gather the data that is “out there”; if you do website
The data generated by your business, whether it is a
crawling, sentiment analysis, gather intelligence on
mobile app, website, online shop, etc...
your competitors or create dictionaries, collect this
This data is typically generated as log files and
external data to feed your BI solution
tends to be too messy to be used directly for
Bring only the data that will add significant insight
analytics.
to the BI solution creating actionable decisions not

available today.
FAST TRACK

2 peripheral data

Data generated from purchased products or

services. Your CRM software holds additional

valuable data which you will likely want to analyze in

synergy with your core data. For example: customer

geography, purchase history, demographics etc. .

Same goes for your ERP system, Google analytics

data, ad campaigns etc.

Initially consider only the critical data that answers

essential business questions that need to be

answered from day one.

Starting your project with mapping all of the data

sources may take a long time. It is often better to

start with few specific sources and fields, and add

the rest gradually as your BI solution evolves.

 ETL

Layer that manages the ‘Extract Transform Load’ Processes.

Can be implemented based either on Scripts (Custom Code) or based on an ETL Tool. Some of the common
functions that this layer performs are:

DAY 3

1 Parsing/cleansing 4 Velocity

Data generated by applications is saved in various The frequency of loading data – is it Daily/Hourly.

formats such as JSON or CSV. The purpose of the Should existing data be updated or only new data

parsing stage is to map the data into a table format inserted?

with headers, columns and rows while extracting

values only from the specified fields.

FAST TRACK

2 Enrich
Implementing the ETL layer through a SaaS tool is
To prepare the data for analytics, usually enrichment
highly recommended, as it facilitates highly effective
steps need to be deployed. For example: tweaking,
database management and seamless collaboration
injecting expert knowledge, implementing geo
within your team.
modifications, matching between sources and

correcting bugs.

3 History depth

During the initial run, history should be loaded either

in bulk or in a loop in order to set the existing data.

On the later runs, only incremental ETL is required.

 DWH

There are several crucial criteria of selecting the right database to serve as the DWH. It should load data from
the source files in the required velocity, store it in an optimized manner and serve various queries from the BI
tool and from any other query you plan to use..
The work should be done in cycles and phases. Load the first batch of the source data as soon as it is ready.
Don't wait until all the ETL work has been fully completed.

DAY 4 4 DB size

In order to choose the right technological solution

1 Schema design
one of the most important considerations is the data

To make the data accessible for analytics and size

reporting, a data schema should be deployed to

map the types of data to the facts, dimensions, 5 Concurrency

measures and attributes. At this stage, The indexes
The number of users and queries that will run
should be defined and the relationship between the
simultaneously on the database should be
tables needs to be set.
estimated and carefully planned for.
Start with the main entities and expand the schema
FAST TRACK

Most probably concurrency will not be a big issue

as you progress.
on day one, as It takes time until all departments and

applications adopt a new DWH.

2 Cloud Vs. On-Premise

Deploying the DWH in the cloud is a highly 6 Scaling

recommended and popular option as it offers
The rate at which the volume of data is expected to
multiple advantages , such as scalability, availability,
grow. If the growth is linear scaling out is usually the
ease of use and flexible pricing.
prefered option (adding hardware to the existing db),
Nevertheless, if you have the required hardware
if the growth will be exponential scaling up might be
and infrastructure already in place, or if most of
a better solution (replacing the db)..
your source data is stored on-premise it makes
Scaling is very simple to execute on cloud platforms.
sense to maintain an on-premise data warehouse. In
With BigQuery for example, scaling requires no
some cases your data may be subject to regulatory
action at all.
constraints which may limit your choice to an on-

premise solution.

Consider the location of your core data source first

 Choosing a BI Tool

The BI reporting tool will be the face of the entire BI solution, Serving dashboards, data visualizations and client
facing reports. Here are the issues to consider when choosing a reporting tool.

DAY 6

The internet is full of benchmarks, POC’s and

1 Clients (internal & external) comparisons between different tools, narrow down

to 2-3 tools as finalists and run a short POC on 1 or

Who are the departments/business functions in the
more of them.
organization and what type of use will they require

from the BI tool? Are there also partners/clients that

will require access to reports?

2 Self-service

An environment for users to do their own analytics

and create custom reports

FAST TRACK

3 Type of analytics

There are different methods of creating reports and

there are BI tools excelling at each one. Some of

the types can be: Lists and tables, Visualizations,

Governed dashboards and ad hoc analysis?

4 Integration with DWH

is the BI tool optimized to query the selected DB?

 Designing Dashboards and Reports

Creating dashboards, reports and analytics might quickly become more complex than initially anticipated. Below
are the main points to consider

DAY 5 DWH, schema, query and dashboard development

will be required and will take several days to

1 Dashboards design implement.

Use in memory capabilities or aggregate the data

When designing a dashboard, ask the following
prior to loading it into the BI tool.
questions: What are the elements in the dashboard?
Don't use too many elements in your initial
How do they interact with each other?
dashboard design.
Is all the data available from the same datasource?

Which filters should be used?

4 adoption plan
What kind of visualization is most suited for each

element in the dashboard? We believe that this is THE most crucial point to
FAST TRACK

Utilize existing dashboards and simplify them as consider. Define who are the end users for each

needed. Do not try to stretch the limits of your report. Prepare a training program to familiarize the

solution at the initial stage of development.. Keep it users with the reports and with the interface and

simple so your users can easily understand what has functionality of your BI tool. Measure ROI figures for

been done. specific reports whenever possible. Your main task

is to make the users realize the benefits of the data

2 self-service environment - mostly its reliability. Teach them the main functions

of the DWH schema and how they can access it

Self service means that a user of the BI is able to
through the BI tool. Your final goal is to educate
login to the BI tool and perform an ad-hoc analysis
and train your users to become “self served”. It
or generate custom reports.
is recommended to involve the end users in the
Having the ablility to share the reports and add
development of the BI tool already at the planning
comments is considered as a nice to have.
stage They should stay involved also during the
At the first stage choose an initial model out of the
development phase as active QA and acceptance
DWH schema to be public for self service
testers.

Choose your first users carefully. As your BI solution

3 performance tuning
evolves, the early adopters will become the

In many cases long loading times of the carefully advocates and active promoters of your analytics

designed dashboards will occur.. Take into account solution within your company.

that performance tuning and optimization for the

 TEAM

BI teams differ between organizations in both size and internal composition

Plan your recruiting in stages and consider hiring an external contractor to temporarily fill in the skill gap.

DAY 7

1 Head of BI/ Director/ VP of analytics/ analytical tools, data analysts construct insights from

Team leader data. They supply their organizations with reports,

summaries, visualizations, thereby transforming

The person responsible for making a firm truly
the data into digestible insights to be used by non-
data-driven. Equipped with both business and
technical personnel. Their work helps management
technological skills, they establish and execute
make data-driven decisions and set goals based on
strategies that generate insights, creating self-
evidence. Furthermore, data analysts run unique
service analytic platforms and other business
queries for stakeholders..
intelligence solutions.
FAST TRACK

Make sure the candidate you consider for this position

Often the right candidate is someone who already
has the right mix of business and technical skills and is
works in your company.Current employees could
not proficient in only one of them.
have the edge because they know the business

inside out and frequently have the right mix of

4 DBA
technical and managerial skills..

Orchestrates the capacity planning, design,

2 BI developer installation, configuration, performance monitoring,

migration, and troubleshooting of all things database

Data professionals who design and build data
related. DBAs maintain database systems, create
pipelines to integrate data from various sources,
new database applications, support existing database
assuring the data is extracted, transformed, and
applications, and manage an organization’s data and
loaded into databases or data warehouses.
metadata..
Consider using an ETL tool to be able to generalise
Knowing the inner workings of the specific database
and scale this position easily.
your company utilises is crucial for this role Therefore

it is advised to select the DB first and then to start the

3 Data/business analyst
recruitment process.

Data analyst acquires, processes, and summarizes

data. By using specific queries and existing

5 Data scientist

data scientists have the ability to invent new

algorithms to solve complex analytical problems.

They utilize computer programming, statistics,

analytical tools, and sophisticated machine learning

to pull out actionable insights from big data.

The range of qualifications required to fill this role is

often very wide. Prioritise the skills most important

for your company.

Creating a full stack business analytics solution

for your company is a process that requires
careful planning and meticulous execution.
This infographic walks you through the tasks
you will need to tackle on the way, from
formulating a strategy to hiring the right team.
Keep this cheat-sheet handy to refer to as
needed.

To learn how easy it is to integrate your data

with Xplenty click here