14.balanced Scorecard Case Study
14.balanced Scorecard Case Study
RESEARCH INSTITUTE
ABSTRACT
The balanced scorecard (BSC) initially developed by Kaplan and Norton, is a
performance management system that enables businesses to drive strategies based
on measurement and follow-up. In recent years, the BSC has been applied to
information technology (IT). The IT BSC is becoming a popular tool with its concepts
widely supported and dispersed by international consultant groups such as Gartner
Group, Renaissance Systems, Nolan Norton Institute, and others. As a result of this
interest, the first real-life applications are starting to emerge. In this paper, the
development and implementation of a departmental BSC within an Information
Services Division (ISD) serving a Canadian financial group will be described and
discussed. We use an IT BSC maturity model to determine the maturity level of the IT
BSC under review.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-2/27
INTRODUCTION
Kaplan and Norton (1992, 1993, 1996a, 1996b) introduced the balanced scorecard (BSC) at an
enterprise level. Their fundamental premise is that the evaluation of a firm should not be
restricted to a traditional financial evaluation but should be supplemented with measures
concerning customer satisfaction, internal processes and the ability to innovate. Results
achieved within these additional perspective areas should assure future financial results and
drive the organization towards its strategic goals while keeping all four perspectives in
balance. For each of the four perspectives they propose a three layered structure: 1. mission
(e.g. to become the customers most preferred supplier), 2. objectives (e.g. to provide the
customers with new products), and 3. measures (e.g. percentage of turnover generated by new
products). The balanced scorecard can be applied to the IT function and its processes as Gold
(1992, 1994) and Willcocks (1995) have conceptually described and has been further
developed by Van Grembergen and Van Bruggen (1997), Van Grembergen and Timmerman
(1998) and Van Grembergen (2000).
In this paper, the development and implementation of an IT BSC within the Information
Services Division (ISD) of a Canadian tri-company financial group consisting of Great-West
Life, London Life and Investors Group (hereafter named The Group) is described and
discussed. We use an IT BSC maturity model (adapted from the capability maturity model
developed by the Software Engineering Institute) to determine the maturity level of the IT BSC
under review. An important conclusion of the paper is that an IT BSC must go beyond the
operational level and must be integrated across the enterprise in order to generate business
value. This can be realized through establishing a linkage between the business balanced
scorecard and different levels of IT balanced scorecards and through the definition of clear
cause-and-effect relationships between outcome measures and performance drivers throughout
the whole scorecard.
IT BALANCED SCORECARD CONCEPTS
In Figure 1, a generic IT balanced scorecard is shown (Van Grembergen and Van Bruggen,
1998). The User Orientation perspective represents the user evaluation of IT. The Operational
Excellence perspective represents the IT processes employed to develop and deliver the
applications. The Future Orientation perspective represents the human and technology
resources needed by IT to deliver its services over time. The Business Contribution perspective
captures the business value created from the IT investments.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-3/27
Figure 1
Generic IT balanced scorecard
USER ORIENTATION
BUSINESS CONTRIBUTION
How do users view the IT department?
How does management view the IT
department?
Mission
To be the preferred supplier of information
Mission
systems.
To obtain a reasonable business contribution
from IT investments.
Objectives
Objectives
Preferred supplier of applications
Control of IT expenses
Preferred supplier of operations vs.
proposer of best solution, from whatever
Business value of IT projects
source
Provision of new business capabilities
Partnership with users
User satisfaction
OPERATIONAL EXCELLENCE
FUTURE ORIENTATION
How effective and efficient are the IT
How well is IT positioned to meet future
processes?
needs?
Mission
Mission
To deliver effective and efficient IT
To develop opportunities to answer future
applications and services.
challenges.
Objectives
Objectives
Efficient and effective developments
Training and education of IT staff
Efficient and effective operations
Expertise of IT staff
Research into emerging technologies
Age of application portfolio
Each of these perspectives has to be translated into corresponding metrics and measures that
assess the current situation. These assessments need to be repeated periodically and aligned
with pre-established goals and benchmarks. Essential components of the IT BSC are the causeand-effect relationships between measures. These relationships are articulated by two key
types of measures: outcome measures and performance drivers. A well developed IT scorecard
contains a good mix of these two types of measures. Outcome measures such as programmers
productivity (e.g. number of function points per person per month) without performance
drivers such as IT staff education (e.g. number of educational days per person per year) do not
communicate how the outcomes are to be achieved. And performance drivers without outcome
measures may lead to significant investment without a measurement indicating whether the
chosen strategy is effective. These cause-and-effect relationships have to be defined
throughout the whole scorecard (Figure 2): more and better education of IT staff (future
orientation) is an enabler (performance driver) for a better quality of developed systems
(operational excellence perspective) that in turn is an enabler for increased user satisfaction
(user perspective) that eventually will lead to higher business value of IT (business
contribution).
Figure 2
Cause-and-effect relationships
IF
IT employees expertise is improved
(future orientation)
THEN
this may result in a better quality of developed systems
(operational excellence)
THEN
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-4/27
(user orientation)
THEN
this may enhance the support of business processes
(business contribution)
The proposed standard IT BSC links with business through the business contribution. The
relationship between IT and business can be more explicitly expressed through a cascade of
balanced scorecards (Van der Zee, 1999; Van Grembergen, 2000). In Figure 3, the relationship
between IT scorecards and the business scorecard is illustrated. The IT Development BSC and
the IT Operational BSC both are enablers of the IT Strategic BSC that in turn is the enabler of
the Business BSC. This cascade of scorecards becomes a linked set of measures that will be
instrumental in aligning IT and business strategy and will help to determine how business
value is created through information technology.
Figure 3
IT
Development
BSC
Business
BSC
IT
Strategic
BSC
IT
Operational
BSC
RESEARCH METHODOLOGY
Case research is particularly appropriate for research within the IT area because researchers in
this field often lag behind practitioners in discovering and explaining new methods and
techniques (Benbasat et al., 1987). This is certainly true for the balanced scorecard and its
application to IT. The Balanced Scorecard is becoming a popular technique with its concepts
supported and dispersed by consultants. A single case design is appropriate when the
investigator has access to a situation previously inaccessible to scientific observation (Yin,
1994). Like Benbasat et al. (1987) we believe that the case research strategy is well-suited to
capturing the knowledge of practitioners and developing theories from it.
A case study research approach is used to study the phenomenon of the IT BSC and its
development and implementation in a single organization. In case study research, the
researcher is an observer/investigator rather than a participant (Benbasat et al., 1987). The
Chief Information Officer (CIO) of the case company (also the second author of the article)
applied the balanced scorecard technique to his IT organization. The other co-authors
conducted all interviews (including interviews with the CIO) to gather data for this study.
Their role was purely the role of observers who were interested in investigating how the IT
BSC concepts, they and other researchers developed in earlier publications, were applied by
practitioners and how the experience and knowledge of practitioners could help to improve the
earlier proposed IT BSC frameworks. Although, the CIO/author used one of the leading
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-5/27
authors publications (Van Grembergen and Van Bruggen, 1997) to build his first scorecard,
the leading author/researcher was never involved as an advisor in the further developments and
implementations.
The initial research took place from the end of 1999 until mid 2000 ( Period-1). The IT BSC
project within the case company is still an ongoing project. During the revision period of
August-November 2002 (Period-2) the article has been updated.
In both research periods, the data was collected through in-depth interviews with the CIO by
means of multiple e-mail conversations and also through some casual face to face
conversations when the authors met during international conferences on IT performance
measurement. During the second research period in 2002, additional in-depth interviews were
conducted with the project manager of the IT balanced scorecard project. Also six individuals
who have key roles and accountabilities for scorecard deliverables at the Group were
interviewed (including the Vice President Information Services, the Financial Control Director,
the Operations & Technical Support Technology Services Director, the Mainframe Technical
Support Manager, the Career Centers Director and the Project Management Career Center
Leader). These interviews were done by means of e-mail and telephone conversations and an
intensive workshop at the headquarters of the company. Data from other sources such as
internal reports and slides from the CIOs presentations for his management were used to
develop and complete an understanding of the case company, its processes, its technology, its
IT organization and its development and implementation of the IT BSC.
CASE COMPANY INTRODUCTION: A TRI-COMPANY
The Great-West Life Assurance Company, London Life and Investors Group are members of
the Power Financial Corporation group of companies, with London Life as a wholly owned
subsidiary of The Great-West Life Assurance Company. In 2001, MacKenzie financial was
also acquired by the Power Financial Corporation Group, but as the IT balanced scorecard
project does not cover this company, MacKenzies organization and IT division will not be
taken into account in this article.
The Great-West Life Assurance Company is an international corporation offering life
insurance, health insurance, retirement savings, specialty reinsurance and general insurance,
primarily in Canada and the United States. Great-West serves the financial security needs of
more than 13 million people in Canada and the United States. Great-West has more than $86.9
billion (all figures in this article are in Canadian dollars) in assets under administration and
$477 billion of life insurance in force. Founded in Winnipeg in 1891, Great-West is now a
leading life and health insurer in the Canadian market in terms of market share.
London Life was founded in Ontario in 1874 and has the leading market share of individual life
insurance in Canada. London Life markets life insurance, disability insurance and retirement
savings and investment products through its exclusive sales force. The company is a supplier
of reinsurance primarily in the US and Europe, and is a 39% participant in a joint venture life
insurance company Shin Fu in Taiwan. London Life has more than $30 billion assets under
administration and $142.6 billion of life insurance in force.
Investors Group, with its corporate headquarters in Winnipeg, was founded more than 70 years
ago. Investors Group is Canadas leading provider of mutual funds, offering a wide spectrum
of funds, including those created through strategic partnerships with some of the best known
Canadian and international investment management firms. It also offers a wide range of
insurance and mortgage options, and currently has $17.1 billion of life insurance coverage in
force through three different carriers, and administers with more than $7.6 billion of primarily
residential mortgages. Investors Group manages assets of $40.5 billion.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-6/27
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-7/27
Figure 4
Merged
IT division
Account
Management
Career Centers
Application
Delivery
Technology
Services
Management Services
Corporate
Technology
eBusiness
Solutions
Centre
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-8/27
strategy. Within The Group the specific concerns for the different stakeholders were (Figure
5):
Figure 5
Stakeholders
Board of Directors
Executive Management Committee
Key questions
Does IT support the achievement of business
objectives?
What value does the expenditure on IT
deliver?
Are IT costs being managed effectively?
Are IT risks being identified and managed?
Are targeted inter-company IT synergies
being achieved?
IT Organization
The concepts of the balanced scorecard and its application to information technology were
discovered through an internet search primarily through the web site of the IT Governance
Institute (www.itgi.org). Departing from this web site, relevant publications on the IT
Balanced Scorecard from academics and practitioners were identified and consulted. It was
believed that the scorecard could provide an answer to the key questions of the different
stakeholders.
The formal development of the IT balanced scorecard began in 1998 and from the start the
objectives were clearly stated:
align I.T. plans and activities with business goals and needs
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-9/27
Service provider
IT is for efficiency
Budgets are driven by external
benchmarks
IT is separable from the business
IT is seen as an expense to control
IT managers are technical experts
Strategic partner
IT for business growth
Budgets are driven by business strategy
IT is inseparable from the business
IT is seen as an investment to manage
IT managers are business problem solvers
The newly constructed ISD is viewed as a strategic partner. During several meetings between
IT and executive management, the vision, strategy, measures of success and value of IT were
jointly created. Typically, pure business objectives were used as the standard to assess IT. The
vision and strategy of ISD were defined as:
ISD is a single IT organization focused on developing world-class capabilities to serve the
distinct customer needs of its three sponsoring companies,
ISD operates as a separate professional services business on a full recovery, non profit
basis,
ISD supports the achievement of company strategies and goals through the industry
consolidation period,
ISD becomes the supplier of choice of information services,
ISD establishes a forward looking enterprise architecture strategy which enables the use of
technology as a competitive edge in the financial service market place,
ISD becomes the employer of choice for career-oriented IT professionals in the markets
in which ISD and The Group operate.
These issues go to the heart of the relationship between IT and the business and will be
reflected in the IT strategic balanced scorecard as is illustrated in Figures 7 and 8. Figure 7
shows the perspective questions and mission statements for the four quadrants: corporate
contribution, customer orientation, operational excellence and future orientation. Figure 8
displays the measures for each perspective. The details regarding the individual perspectives
and their measures are in annex.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-10/27
Figure 7
CUSTOMER ORIENTATION
Perspective question
How should IT appear to business unit
executives to be considered effective in
delivering its services?
CORPORATE CONTRIBUTION
Perspective question
How should IT appear to the company
executive and its corporate functions to be
considered a significant contributor to
company success?
Mission
To be the supplier of choice for all information
services, either directly or indirectly through
supplier relationships.
Mission
To enable and contribute to the achievement
of business objectives through effective
delivery of value added information
services.
OPERATIONAL EXCELLENCE
FUTURE ORIENTATION
Perspective question
At which services and processes must IT excel
to satisfy the stakeholders and customers?
Perspective question
How will IT develop the ability to deliver
effectively and to continuously learn and
improve its performance?
Mission
To deliver timely and effective IT services at
targeted service levels and costs.
Mission
To develop the internal capabilities to
continuously improve performance through
innovation,
learning
and
personal
organizational growth.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-11/27
Figure 8
Development services
performance
alignment
Competitive costs
Business/IT
Value delivery
Operational service
performance
Cost management
Customer
orientation
Corporate
contribution
Risk
management
Customer satisfaction
Inter-company
synergy
achievement
Vision
and
strategy
Development process
performance
Operational
excellence
Future
orientation
Human
resource
management
Operational process
performance
Employee satisfaction
Process maturity
Knowledge management
Enterprise architecture
management
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-12/27
A cascade of balanced scorecards has been established to create a link between the scorecards
at the unit level and the overall business objectives (see Figure 9). A link between the IT BSC
and the Business BSC is not yet implemented as there is currently no formal Business BSC for
the Group. The scorecards at the unit level are classified into three groups: operational services
scorecards (e.g. IT service desk scorecard), governance services scorecards (e.g. career center
scorecard), and development services scorecards (e.g. application development scorecard).
The measures of these unit scorecards are rolled-up or aggregated in the IT strategic balanced
scorecard. This, in turn is fed into and evaluated against the business objectives. In this way,
the service (and value) delivered by IT is directly measured against the objectives of the
overall business. Further, on an annual basis, the IT strategic BSC is reviewed by business and
IT management and the result is fed back into the next annual planning cycle. This planning
cycle defines what the business needs are and what IT must do to accomplish those needs.
For example, from the IT service desk scorecard (i.e. a unit scorecard, which is situated in the
operational services scorecard group), metrics such as average speed of answer, overall
resolution rate at initial call and call abandonment rate (all three customer orientation metrics)
are rolled-up to service level performance metrics in the IT strategic balanced scorecard. Other
metrics of this unit scorecard, such as expense management (corporate contribution
perspective), client satisfaction (customer orientation perspective), process maturity of incident
management (operational excellence perspective) and staff turnover (future orientation
perspective), will aggregate as part of the IT strategic scorecard. The overall view of the IT
strategic balanced scorecard is then fed into and evaluated against the defined business
objectives.
Figure 9
Business Objectives
Operational Services
Scorecards
Governance Services
Scorecards
Development Services
Scorecards
The second way to demonstrate business value is situated within the IT strategic balanced
scorecard. The cause-and-effect relationships between performance drivers and outcome
measures of the four quadrants are established as indicated in Figure 10. These connections
help to understand how the contribution of IT towards the business will be realized: building
the foundation for delivery and continuous learning & growth (future orientation perspective)
is an enabler for carrying out the roles of the IT divisions mission (operational excellence
perspective) that is in turn an enabler for measuring up to business expectations (customer
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-13/27
Customer
orientation
Measuring up to
business expectations
Corporate
contribution
Ensuring effective
IT governance
Vision and
strategy
Carrying
out the roles of the IT
divisions mission
Operational
excellence
Building
the foundation for
delivery and
continuous
learning & growth
Future orientation
Establishing the link with the business objectives through a cascade of scorecards and defining
the cause-and-effect relationships within the scorecards are important steps in determining the
maturity of the IT balanced scorecard. This maturity can be assessed through a maturity model.
We therefore used an IT Maturity Model (MM) to match the case companys scorecard level
against the levels of the IT MM. The IT MM we used is based on the Software Engineering
Institutes Capability Maturity Model CMM (Paulk et al, 1993). Our IT BSC Maturity Model
highlights five maturity levels with the following characteristics (Figure 11):
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-14/27
Figure 11
Level 1 Initial
There is evidence that the organization has recognized there is a need for a measurement
system for its information technology division. There are ad hoc approaches to measure IT
with respect to the two main IT processes, i.e. operations and systems development. This
measurement process is often an individual effort in response to specific issues.
Level 2 Repeatable
Management is aware of the concept of the IT balanced scorecard and has communicated its
intent to define appropriate measures. Measures are collected and presented to management in
a scorecard. Linkages between outcome measures and performance drivers are generally
defined but are not yet precise, documented or integrated into strategic and operational
planning processes. Processes for scorecard training and review are informal and there is no
compliance process in place.
Level 3 Defined
Management has standardized, documented and communicated the IT BSC through formal
training. The scorecard process has been structured and linked to business planning cycle.
The need for compliance has been communicated but compliance is inconsistent. Management
understands and accepts the need to integrate the IT BSC within the alignment process of
business and IT. Efforts are underway to change the alignment process accordingly.
Level 4 Managed
The IT BSC is fully integrated into the strategic and operational planning and review systems
of the business and IT. Linkages between outcome measures and performance drivers are
systematically reviewed and revised based upon the analysis of results. There is a full
understanding of the issues at all levels of the organization that is supported by formal training.
Long term stretch targets and priorities for IT investment projects are set and linked to the IT
scorecard. A business scorecard and a cascade of IT scorecards are in place and are
communicated to all employees. Individual objectives of IT employees are connected with the
scorecards and incentive systems are linked to the IT BSC measures. The compliance process
is well established and levels of compliance are high.
Level 5 Optimized
The IT BSC is fully aligned with the business strategic management framework and vision is
frequently reviewed, updated and improved. Internal and external experts are engaged to
ensure industry best practices are developed and adopted. The measurements and results are
part of management reporting and are systematically acted upon by senior and IT
management. Monitoring, self-assessment and communication are pervasive within the
organization and there is optimal use of technology to support measurement, analysis,
communication and training.
According to this IT BSC maturity model the case company is at the Repeatable stage (Level
2). Parts of the level 3 maturity are achieved, but the basic principle of maturity assessment
states that all conditions have to be fulfilled before moving to a higher maturity level. The
challenge is to reach stage 4, the Managed level within two to three years. It is understood
that major milestones in this further development will be:
the detailed cause-and-effect relationships between the output measures and performance
drivers have to be further elaborated,
short and long term targets have to be further defined,
individual and group objectives of IT employees have to be further linked to the IT BSC,
the scorecards have to be further integrated in the strategic and operational management
processes.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-15/27
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-16/27
This desired timeline is probably quite optimistic and it may well take twice as
long to accomplish these changes.
However, the most important aspect is that all stakeholders in the process were engaged by the
end of 2001 and that progress is made each subsequent year. At this moment, most metrics are
manually captured, which is a labor-intensive task. However, according to the Mainframe
Technical Support Manager:
It is important to first identity the correct metrics that need to be captured
before implementing tools that automate the data collection.
LESSONS LEARNED
The following lessons can be attributed to this IT BSC case:
1. Start simultaneously constructing a business and IT scorecard
The IT BSC within the case company was started within the IT organization primarily with the
objective to ensure that IT is fairly evaluated by the business. This is a rather defensive
approach and focuses merely on the internal IT processes. Although it is clearly recognized
within the case company, that a more explicit linkage with the business (with a business
balanced scorecard) has to be developed and supported, the question still remains whether it is
more appropriate (a) to start with a business balanced scorecard followed by the subsequent
creation of the corresponding IT scorecards or (b) to develop both scorecards simultaneously?
It is now our conclusion that it is probably more ideal to start simultaneously with both
scorecards which requires both IT and senior management to discuss the opportunities of
information technologies which supports the IT/business alignment and IT governance
process.
2.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-17/27
manager is assigned to the project. A group of 15 individuals have key roles and
accountabilities for scorecard deliverables.
5. Provide best IT practices
Introducing an IT balanced scorecard in an IT environment with poor management and IT
practices is too large a challenge. The implementation of the IT BSC within the case company
was certainly supported by practices already in place such as ROI-evaluation of IT projects,
the existence of IT steering committees, Service Level Agreement practices, etc. If it is decided
to implement e.g. the Information Economics approach to score and evaluate projects and to
integrate this method within the IT BSC, this will take considerable time and is to be seen as a
separate project.
6. Revisit the dynamic measures
The implementation of the IT balanced scorecard requires the establishment and definition of a
large number of metrics. The appropriateness of these metrics should be regularly evaluated.
In the words of the Mainframe Technical Support Manager:
As business requirements change, the metrics are dynamic and should be reevaluated on a regular basis. Most important in this regard is that it should
always remain very clear why a certain issue is measured, i.e. what the value is of
measuring it. When this value can not be demonstrated any more, a measure
should be challenged and changed or replaced by another one.
CONCLUSION
In this paper, the development and implementation of an IT balanced scorecard within a large
Canadian insurance group is described and discussed. It was shown that building and
implementing such a scorecard is a project that needs substantial human and financial
resources. Furthermore, setting up an IT BSC is a project that is characterized by different
phases in time. The current status of the case scorecard is Level 2 of the IT BSC Maturity
Model that is introduced in this paper. This implies that the case IT scorecard to- date has to be
linked with the business scorecard or at least the business objectives to support the IT/business
alignment process and the IT governance process. Currently, a plan for the next two years has
been developed with the objective to build a mature IT BSC explicitly linked to the business. It
is recognized within the case company that this will be a great challenge for both IT and
business people.
The case under review illustrated one of the most crucial issues in building and implementing
an IT strategic Balanced Scorecard: its required linkage with the business objectives. To
create this link a cascade of balanced scorecards has been established with at the lower level
unit scorecards for the operational and development services. The measures of these unit
scorecards are rolled-up or aggregated in the IT strategic scorecard that ultimately realizes the
link with the business objectives through its corporate contribution perspective. The precise
articulation of the cause-and-effect relationships through the identification of outcome
measures and their corresponding performance drivers, seemed to be a critical success factor.
These relationships are implicit in the current IT strategic balanced scorecard but are to be
defined more explicitly.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-18/27
REFERENCES
1. Bensabat, I., Goldstein, D. and Mead, M. The case research strategy in studies of
information systems, MIS Quarterly, September 1987, pp. 369-386.
2. IT Governance Institute, CobiT (Control Objectives for IT and related Technologies) 3e
edition, www.isaca.org/cobit.htm, 2000.
3. Gold, C. Total quality management in information services IS measures: a balancing
act, Research Note Ernst & Young Center for Information Technology and Strategy,
Boston, 1992.
4. Gold, C. US measures a balancing act, Ernst & Young Center for Business Innovation,
Boston, 1994.
5. Kaplan, R. and Norton, D. The balanced scorecard measures that drive performance,
Harvard Business Review, January-February 1992, pp. 71-79.
6. Kaplan, R. and Norton, D. Putting the balanced scorecard to work, Harvard Business
Review, September-October 1993, pp. 134-142.
7. Kaplan, R. and Norton, D. Using the balanced scorecard as a strategic management
system, Harvard Business Review, January-February 1996a, pp. 75-85.
8. Kaplan, R. and Norton, D. The balanced scorecard: translating vision into action,
Harvard Business School Press, Boston, 1996b.
9. Paulk, M., Curtis, B., Chrissis, M.-B., and Weber, C. Capability maturity model for
software, version 1.1, Technical Report Software Engineering Institute, CMU/SEI-93TR-024, ESC-TR-93-177, February 1993.
10. Saull, R. The IT Balanced Scorecard A roadmap to effective governance of a shared
services IT organization, Information Systems Control Journal (previously IS Audit and
Control Journal), Volume 2, 2000, pp. 31-38.
11. Van der Zee, J. Alignment is not enough: integrating business and IT management with
the balanced scorecard, Proceedings of the 1st Conference on the IT Balanced Scorecard,
Antwerp, March 1999, pp. 1-21.
12. Van Grembergen, W. and Van Bruggen, R. Measuring and improving corporate
information technology through the balanced scorecard technique, Proceedings of the
Fourth European Conference on the Evaluation of Information technology, Deflt,
October 1997, pp. 163-171.
13. Van Grembergen, W. and Timmerman, D. Monitoring the IT process through the
balanced score card, Proceedings of the 9th Information Resources Management (IRMA)
International Conference, Boston, May 1998, pp. 105-116.
14. Van Grembergen, W. The balanced scorecard and IT governance, Information Systems
Control Journal (previously IS Audit & Control Journal), Volume 2, 2000, pp.40-43.
15. Venkatraman, N. in Valuing the IS contribution to the business, Computer Sciences
Corporation, 1999.
16. Willcocks, L. Information management. The evaluation of information systems
investments, Chapman & Hall, London, 1995.
17. Yin, K. Case study research. Design and methods. Sage Publications, Thousand Oaks,
1994.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-19/27
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-20/27
Synergy achievement is measured through the achievement of single system solutions, targeted
cost reductions and the integration of the IT organizations. This measure is very crucial in the
context of the merger of the three IT organizations in the sense that it enables a post evaluation
of this merger and demonstrates to management whether the new IT organization is effective
and efficient. The selection of single system solutions was a cooperative effort between
business leaders and IT staff, resulting in a Target State Architecture depicting the target
applications architecture. The synergy targets were heavily influenced by the consulting firm
(Bain & Co.) that was used to assist in evaluating the London Life acquisition and the tricompany IT merger potential. The consultants suggested specific dollar reduction targets for
technology services (IT operations) and application delivery services (IT development) largely
based on norms they had developed from their previous merger and acquisition work. The
approval of the Target State Architecture plan and the attainment of the targeted integration
cost reductions will be measured. The IT organization integration metric refers to the
synergies within the IT organization, e.g. is there one single service desk for the three
companies or are there three different ones?
Figure 13
Objective
Business/IT alignment
Value delivery
Cost management
Risk management
Measures
Operational plan/budget
approval
Measured in business
unit performance
Attainment of expense
and recovery targets
Attainment of unit cost
targets
Results of internal audits
Benchmarks
Not applicable
Execution of Security
Initiative
Delivery of disaster
recovery assessment
Single system solutions
Target state architecture
approval
Attainment of targeted
integration cost
reductions
IT organization
integration
Inter-company synergy
achievement
Not applicable
Industry expenditure
comparisons
Compass operational
top performance levels
OSFI sound business
practices
Not applicable
Not applicable
Not applicable
Not applicable
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-21/27
In the Customer satisfaction area, the IT BSC of the merged IT organization is relying on
annual interviews with key business managers. It is the intent to set up one generic survey,
which can be re-used, with relevant questions that cover the topics mentioned in Figure 14.
Insight into the competitive costs area can demonstrate to the business how cost competitive
the IT organization is compared to other (e.g. external) parties. This insight is realized by
measuring the attainment of IT unit cost targets and the blended labor rate. This rate model
provides an overall single rate for any IT professional who is appointed to the business. The
competitive costs measures are benchmarked against Compasss operational Top Performing
level and against the offerings of commercial IT service vendors (market comparisons).
Development services performance measures are project oriented using attributes such as goal
attainment, sponsor satisfaction and project governance (i.e. the way the project is managed).
These data are mostly captured by interviews with key managers. The most effective time to
establish the basis for these (project) development measures is at the point where business
cases are being prepared and projects are evaluated. Each IT project initiative will be evaluated
by the IS Executive Committee in which IT and business managers determine - based on the
business drivers, budget and state architecture compliance - which projects need to be
executed. When a project is approved, the project manager defines clear targets for cost,
schedule, quality, scope and governance. The quantitative data (e.g. budget) are reported
throughout the lifecycle of the project. After completion of the project, the quantitative and
qualitative data are evaluated during the major project review and the main success drivers,
delivery issues and lessons learned are documented.
In terms of Operational service performance, IT management measures achievement against
targeted service levels. For each operational unit (e.g. data center), average response time,
service availability and resolution time for incidents are rolled-up to these service performance
metrics in the strategic balanced scorecard. The results are benchmarked against the
performance of competitors.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-22/27
Figure 14
Objective
Customer satisfaction
Measures
Business unit survey
ratings:
Cost transparency and
levels
Service quality and
responsiveness
Value of IT advise
and support
Contribution to
business objectives
Attainment of unit cost
targets
Benchmarks
Not applicable
Compass operational
Top Level
Performing levels
Market comparisons
Development services
performance
Not applicable
Operational services
performance
Competitor
comparisons
Competitive costs
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-23/27
these processes. The Group has identified 15 out the 34 priority processes that should have a
maturity assessment in 2003 and the other processes will be measured later.
Enterprise architecture management deals with the IT responsibility to define an enterprise
architecture which supports long term business strategy and objectives and to act as a steward
on behalf of business executives to protect the integrity of that architecture. Major project
architecture approval measures the compliance of net new systems as they are proposed,
developed and implemented. Product acquisition compliance technology standards measures
our adherence to detailed technology standards which are at the heart of minimizing
technology diversity and maximizing inter-company technology synergies. The State of the
Infrastructure assessment measures the degree to which IT has been able to maintain a robust
and reliable infrastructure as required to deliver effectively to business needs. It does so by
comparing each platform area against risk based criteria for potential impact to business
continuity, security and/or compliance.
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-24/27
Figure 15
Objective
Development process
performance
Operational process
performance
Process maturity
Enterprise architecture
management
Measures
Function point measures
of:
Productivity
Quality
Delivery rate
Benchmark based
measures of:
Productivity
Responsiveness
Change
management
effectiveness
Incident occurrence
levels
Assessed level of
maturity and compliance
in priority processes
within:
Planning and
organization
Acquisition and
implementation
Delivery and
support
Monitoring
Major project
architecture approval
Product acquisition
compliance to
technology standards
State of the
infrastructure
assessment
Benchmarks
to be determined
Selected Compass
benchmark studies
To be defined
Not applicable
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-25/27
charge lower rates to the business for the IT assigned people), and professional development
days per staff member.
Employee satisfaction is measured by using surveys with questions relating to compensation,
work climate, feedback, personal growth, and vision and purpose. Benchmark data of North
American technology dependent companies are provided by a third party.
In the knowledge management area, the delivery of internal process improvements to the
Cybrary is very important. The Cybrary refers to the intranet that all employees can assess
for seeking and sharing knowledge. To measure improvements, metrics (e.g. number of hits
per day on the Cybrary) still need to be developed. Closely linked to this, knowledge
management is also measured by the implementation of the lessons learned sharing process.
Here too, specific metrics still need to be developed.
Figure 16
Objective
Human resource
management
Employee satisfaction
Knowledge management
www.uams.be/itag
Measures
Results against targets:
Staff complement
by skill type
Staff turnover
Staff billable ratio
Professional
development days
per staff member
Employee satisfaction
survey scores in:
Compensation
Work climate
Feedback
Personal growth
Vision and purpose
Delivery of internal
process improvements
to Cybrary
Implementation of
lessons learned sharing
process
Benchmarks
Not applicable
Market comparison
Industry standard
Industry standard
North American
technology dependent
companies
Not applicable
Not applicable
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-26/27
About UAMS
UAMS (University Antwerp Management School) has the ambition to be a learning partner in
management, by offering a broad range of training programmes for future and current
managers in the business world, in public services and social-profit organizations. The
priorities cover optimal quality control, interactive teaching methods, an emphasis on researchbased knowledge and best practice, an international orientation and a continuous adaptation of
our programmes to the needs of the market.
About ITAG
The Information Technology Alignment and Governance (ITAG) Research Institute, was
established in within UAMS to host applied research in the domains of IT Governance and
business/IT alignment. The research centre is an initiative of Prof. dr. Wim Van Grembergen
and dr. Steven De Haes. Both have research and practical experience in the IT Governance
and Strategic Alignment domains. Recently, this team was reinforced by senior researcher
Hilde Van Brempt.
Contact
UAMS - ITAG Research Institute
Sint-Jacobsmarkt 9-13
B-2000 Antwerpen
Belgium
www.uams.be/itag
Wim Van Grembergen is professor and Chair of the MIS Department at the Business Faculty
of UFSIA (University of Antwerp). He has teaching assignments at the University of
Stellenbosch in South Africa and the Institute of Business Studies in Moscow. He teaches
information systems at undergraduate and executive level, and researches in business
transformations through information technology, audit of information systems, IT Balanced
Scorecard and IT Governance. He served as Academic Director of the MBA Program of
UFSIA (1989-1995) and presently he is academic co-ordinator of an IT-audit master program
and an E-business master program. Dr. Van Grembergen presented at the European
Conference on Information Systems (ECIS) in 1997 and 1998, at the Information Resources
Management Association (IRMA) Conferences in 1998, 1999 and 2000, and at the Hawaii
International Conference on Systems Sciences (HICSS) in 2001 and 2002. He was Track
Chair on IT Evaluation for the 2000 and 2001 IRMA-conference and Minitrack Chair IT
Governance and its Mechanisms for the 2002 and 2003 HICSS-conferences. He published
articles in journals such as Journal of Strategic Information Systems, Journal of Corporate
Transformation, Journal of Information on Technology Cases and Applications, IS Audit &
Control Journal and EDP Auditing (Auerbach). He also has several publications in leading
Belgian and Dutch journals and published in 1997 a book on business process reengineering in
Belgian organizations and in 1998 a book on the IT Balanced Scorecard. He edited a book
entitled IT Evaluation Methods and Management which was published in 2001 by Idea Group
Publishing and in 2002 Information Systems Evaluation Management also published by IGP.
Currently, he is editing a book titled Strategies for Information Technology Governance.
Professor Van Grembergen serves on the editorial boards of Journal of Global Information
Technology Management (JGITM), Journal of Information Technology Cases and
Applications (JITCA) and Annals of Cases on Information Technology and Management in
Organizations. Professor Van Grembergen was engaged in the development of CobiT 3rd
Edition and has presented papers and conducted workshops at ISACA (Information Systems
Audit and Control Association) international conferences. He is also member of the Academic
Relations Task Force of ISACA and is currently conducting a research project for ISACA on
IT Governance. Dr. Van Grembergen is a frequent speaker at academic and professional
www.uams.be/itag
Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial group-27/27
meetings and conferences and has served in a consulting capacity to a number of firms. He is a
member of the Board of Directors of an IT company servicing a Belgian financial group. His
e-mail address is: [email protected]
Ronald Saull, MBA, CSP is the Senior Vice-President and Chief Information Officer of the
Information Services Division of Great-West Life, London Life and Investors Group
headquartered in Winnipeg, Canada. He has over 25 years experience as an information
systems professional and manager in both the public and private sectors. Mr. Saull is a VicePresident of the International Board of Trustees of ISACA (Information Systems Audit and
Control Association) consisting of over 20,000 information systems professionals worldwide.
He is the past chairman of ISACAs Research Board and is currently active as a member of the
Board of Directors, the CobiT Steering Committee and the IT Governance Board. He has
published an article on the IT Balanced Scorecard in the Information Systems Control Journal
and presented on this topic at ISACA conferences. His e-mail address is:
[email protected]
Steven De Haes is responsible for the Information Systems Management executive programs
at the University of Antwerp Management School (UAMS). He is engaged in research in the
domain of IT Governance and in this capacity performing research for ISACA (Information
Systems Audit and Control Association). Currently, he is preparing a Ph.D. on the practices
and mechanisms of IT Governance. He has several publications on IT Governance primarily in
the Information Systems Control Journal. His e-mail address is: [email protected].
www.uams.be/itag