A BALANCED SCORECARD APPROACH TO PROJECT

MANAGEMENT LEADERSHIP
JAMES NORRIE, director, School of Information Technology Management, Ryerson University,
Toronto, Canada; and Doctor of Project Management candidate, RMIT University, Melbourne
Australia.

DEREK H. T. WALKER, professor of Project Management and program director for Doctor of Project
Management, RMIT University, Melbourne, Australia.

Introduction
ABSTRACT There are numerous reports that document cases of projects, particularly infor-
mation technology (IT) projects, delivered substantially beyond the due date and
In this paper, we discuss ways that project
well above the outlined budget. One such project is the United Kingdom’s (UK)
managers can use measurement (using a
tool such as the balanced scorecard) to notorious IT project Taurus. It was abandoned after it amassed UK£500 million
improve the operational performance of their in costs and produced few results. Project reviewers found a lack of project lead-
project teams. Project managers will see that ership and project definition as factors in causing Taurus’s failure (Drummond,
attaching measures to outcomes clarifies
1998). Such failures are often publicly touted by the popular business press in
project objectives and supports well-defined
and well-communicated links between the articles that frequently seem intent on vilifying the project management field.
project vision and business strategy. These Such attention gives the public a lingering negative impression of our field’s
also enable project managers to more effec- strategic value.
tively monitor and control project activities Project management researchers, however, as the current literature shows,
for the purpose of improving project results.
This paper reinforces the importance of strat-
widely recognize the important role organizational leaders play in envisioning a
egy as an added dimension to the traditional preferred future that encompasses both general strategy and change management.
triple constraint. For example, Briner, Hastings, and Geddes (1996) state, "The most significant suc-
We present this information through cess factors for project teams is that they have a common and shared idea of what
our comparison and survey of two projects
difference they are trying to make as a result of the project" (p. 89). To develop a
undertaken by project teams at a large North
American global telecommunications organi- preferred project outcome through exploratory dialogue with various project stake-
zation. The results of our study provide early holders, organizational leaders must have a clear picture of the strategy the com-
evidence of the usefulness of the balanced pany will implement to achieve the preferred outcome. The leadership’s purpose is
scorecard (BSC) as a tool for improving proj- to define and scope a project so that its reason-for-being is well understood by
ect management effectiveness. Our study
also shows that balanced performance
those who can influence the project’s successful execution. A leader’s vision helps
measurement is an important technique for the project team articulate the project’s objectives, goals, and products.
establishing on-strategy project delivery. We As a solution to the dilemma of lacking a clear project vision, Baccarini
propose using this technique primarily as an (1999) and Davis (1995) offer the Logical Framework Method (LFM) as a tool
extension of current practices by adding a
for defining project success. We agree with the solution offered by these
strategic measurement dimension.
researchers; but we argue that this method can be strengthened if organizations
Keywords: balanced scorecard; leadership; implement it within a strategic measurement framework. Doing so enhances the
project management practice; measurement clarity of the objectives the team seeks to accomplish. And in doing so, organi-
theory; business performance management. zations could help project teams connect specific project objectives to current
©2004 by the Project Management Institute
strategic gaps. By linking the outcomes of a project with a measurable vision,
Vol. 35, No. 4, 47-56, ISSN 8756-9728/03 organizations can enhance the commitment of the individuals on its project
teams to their projects.

December 2004 Project Management Journal • 47

 As far back as the mid-1980s, Best practice implicitly assumes responsibility of the project sponsor
Tuman (1986) and Cleland (1986) that project teams have a clear vision and the project managers. This shared
concurrently recognized — and simul- of the project that evolves from a responsibility demands that these indi-
taneously presented findings — that process led by the executive sponsor viduals wrestle with this issue via an
contradicted the then-common notion and a project leader. This is the process ongoing dialogue in order to create a
that on-time, on-budget, and on-qual- used in project management’s tradi- process that resolves any ambiguities.
ity were the most strategically impor- tional triple-constrained model, which The traditional triple constraints, how-
tant and valid measures of project — as mentioned — focuses on time, ever, are more clearly the direct respon-
success. Yet now, almost two decades budget, and quality outcomes and pre- sibility of the project manager and
later, professionals in our field remain supposes that all projects that are project team, once the project sponsor
quite focused on this legacy and often approved are therefore strategic. To agrees to the project. Figure 1 notes
seem committed to only using this tra- negate this assumption creates com- that the connection of the newly
ditional triple-constrained model. This munication and decision-making chal- added on-strategy dimension is central
dependency may indicate our field’s lenges that are perhaps new to many to the achievement of the other three
lack of finding a definitive alternative project managers. What if the projects traditional constraints.
to the traditional model. were not strategic? Or what if the strat- The problem of providing project
Our review of the current litera- egy evolves more quickly than the pro- leaders with the tools and process
ture revealed that numerous projects ject’s timelines? If over time a project needed to resolve any potential gaps
are perceived as failing because of poor that was once a highly strategic imper- when emerging strategy does not obvi-
leadership and enfeebled articulation ative devolves into a less strategic ini- ously or consistently intersect with
of the project vision or a lack of mean- tiative, project managers may find existing project goals is the focus of
ingful business impact. These percep- themselves lost: Traditional business our proposed change to the current
tions show the ways that organizations case methods do not provide guidance triple-constraint methodology.
have failed to align their overall strate- on ways to confront this situation. To support our conclusions, we
gic goals with the objectives of their Such strategic ambiguity creates severe are reporting on our previous study of
individual projects. This information leadership challenges. two projects. In this study we tested
may also reflect how quickly business In response, project managers may our above-mentioned proposition.
strategies change and evolve in relation try to create an illusion of tangible Our research goal was to show that
to project timelines. Nonetheless, any progress by relying more heavily upon when an organization uses a BSC
tool or technique that can help organ- traditional on-time, on-budget, and framework, it could improve the ways
izations develop better-articulated on-quality measures—yet this tactic it develops its project goals and objec-
strategic goals and objectives and more fails to address the strategy ambiguity tives because the BSC enables an
concrete project visions is a valuable or establish appropriate project goals. organization to create a better link
leadership tool. We assert that the bal- Organizations accepting this approach between project vision and business
anced scorecard (BSC) could serve as would divorce their projects, in regard strategy, which results in more success-
one such tool. to an on-strategy measure, from critical ful outcomes. We also wanted to learn
In our ensuing argument we iden- executive insight and leadership. The which areas of current project manage-
tify the challenges involved in provid- centrality of this point is best illustrat- ment practice were impacted the most
ing project leaders with the tools, ed with a diagram that creates a by an organization’s use of a BSC. We
methods, and information they need quadruple constraint by inserting into wanted to discover if the BSC actually
to develop a clear intersection the pyramid of the traditional three improved the project team’s ability to
between business strategy and project constraints an on-strategy dimension make positive strategic decisions.
goals. We also recognize that business central to managing project success.
environments may be too dynamic to A major premise of this paper is Project Leadership, Project Vision, and
permit the organization to continual- that the on-strategy aspects of project the BSC
ly articulate and update the intersec- management are clearly the shared To better understand project manage-
tions between its business strategy ment, we believe practitioners must dis-
and a particular project’s goals. On-Time tinguish between the management of a
Hence, we see the potential for a pos- project (the day-to-day operations of a
sible gap in project management prac- project plan in pursuit of an agreed set of
tices to emerge, one that arises at the outcomes—on-time and within budget)
point where these intersections are On- and project leadership (the higher pur-
not well understood and where the Strategy suit of the project team’s creating pur-
On- On-
ill-defined project vision is internal- Budget Quality poseful, strategic action that will
ized by the project team’s members augment the organization’s business
and acted on to the organization’s Figure 1. A quad constrained project strategy and achieve results within the
ultimate detriment. management model norms and values of the organization).

48 • Project Management Journal December 2004

 The current literature shows a gen- future state, of taking a solution-build- begun assessing the impact of this
eral agreement among researchers ing negotiation approach to defining topic in a project context (Yukl, 1998),
about the differences between leader- the scope of the project and then clear- although primarily from a social-psy-
ship and management (Bennis, 1989; ly communicating this to the project chological perspective. We propose
Kotter, 1990; Zaleznik, 1977). There is team and other stakeholders, defines a that researchers look at this problem
also an extensive body of literature strategy for realizing the vision, and from the perspective of enhancing the
that has already explored this domain; translating the strategies into opera- way project leaders use strategic meas-
exploring these differences in this tional plans and results. urement.
paper has limited utility. What is A vexing problem particularly rele- For projects with a long time-hori-
important to note is that, in general, vant to our study is the organization’s zon, it would seem that the most like-
researchers agree that leadership must providing leadership to a project team ly cause of a lack of strategic
exert itself most when the business when the project context or business connections is rapidly changing indus-
context is vague, dynamic, or challeng- strategy is either ill-defined or dynam- try or business circumstances. These
ing. (There is a lack of citations in the ic to the point where there is not a circumstances most likely occur in
literature indicating how to accom- clear and measurable connection mega-project settings; an example is
plish this in a project management set- between the project goals and out- the implementation of enterprise
ting when these same conditions comes and the business strategy. The resource planning (ERP) or customer
appear.) latter problem could also involve team relationship management (CRM) tech-
By definition, project manage- members who do not necessarily agree nology, where the time horizon for a
ment is about implementing a change on the interpretation of the strategy in significant IT project may run into
program (Briner et al., 1996; Cleland, relation to their assigned goals and years and impact upon all core opera-
1999; Turner & Cochrane, 1993) in the objectives. This problem has been tions of a company. It is critical —
form of system changes — as in IT explored in the past, most notably by especially in these types of projects,
projects — or in building projects, new Bennis and Nanus (1997), Bennis, where changed strategies can radically
automotive products, airplanes, or Spevietzer, and Cummings (2001), and influence a project’s goals and objec-
weapons systems. This creates a dilem- Turner and Cochrane (1993). The tives — that such projects are directly
ma for project managers who, when existing literature contains both gener- connected, and remain connected, to
faced with a set of ambiguous circum- al team-based solutions (Katzenbach & the company’s emerging and changing
stances, do not appear to have at their Smith, 1993; Robbins & Finlay, 1997; business strategy. Unless project lead-
disposal very many tools to address Yukl, 1998) and specific project-based ers constantly and purposefully meas-
these situations. solutions (Briner et al., 1996; Thite, ure the on-strategy dimension of an
The project manager acts in many 1999). In exploring this problem, we organization’s projects, they may fail
ways as both a leader and a manager; found that the connection is either to successfully keep projects connected
however, the project sponsor may, ambiguous or understood by only a to the organization’s evolving business
depending on the project and person- few key stakeholders, rather than more strategy. If these leaders rely purely on
alities of the project sponsor and proj- broadly accepted by everyone who can the traditional measures of the triple
ect manager, influence the extent of influence the full range of project out- constraint, they could, quite possibly,
this overlap (Briner et al., 1996; comes. Many researchers, particularly successfully deliver an ultimately non-
Cleland, 1999; Morris, 1994). We do Senge (1990), have stressed in their strategic project on-time, on-budget,
not wish to underestimate either of work that a narrowly held vision is and on-quality!
these issues or the behavior and com- insufficient in most leadership con- Regardless of the originating rea-
petence of project sponsors and project texts and fails to create purposeful sons for the problems identified above,
managers; rather, we want to provide a coordinated action among all follow- any gaps between vision and strategy
more concrete method for addressing ers. This problem has been so thor- create potential challenges for the daily
this gap than simply relying on mana- oughly studied and documented that operational management of projects,
gerial competence. we agree with this conclusion. especially when managed under
Many leadership tasks relate to We noticed another possible prob- ambiguous conditions. While we do
developing a vision of the project out- lem that arises when a corporate cul- not intend to minimize the efforts of
come that is practical and yet capable ture or a particular internal set of previous authors to address this prob-
of mobilizing and motivating team values is incongruous with project suc- lem, in general we found that the solu-
members to accomplish the project’s cess. This topic is also well studied in tions almost always focus on a desire
goals and objectives. This leadership the change management literature ( to present methods directed at the
vision engages stakeholders who are Collins & Porras, 1996; Kotter, 1995): behavior of the team leader or on
not actively involved in the project; it The symptoms and causes of this kind addressing aspects of the problem
also inspires them to maintain their of discord at the corporate level are, resulting from the company’s culture
support over the project’s timeline. according to this literature, well under- or values. There is often a heavy
This process of envisioning a preferred stood. More recently, researchers have emphasis on motivational theory as

December 2004 Project Management Journal • 49

the underpinnings of how leaders the benefits of this approach to the ment focus and an appropriate empha-
influence team members. While use- management of major projects. sis on process as a method of tracking
ful, these studies do not seem (based Current literature in the strategy and reporting tasks, usually in the
on our experience) to address the core domain suggests that organizations form of a project plan based on a work
problem: The insufficiency or instabil- should link planned outcomes to their breakdown structure (Project
ity of strategy to properly develop and corporate strategy using a measure- Management Institute, 1996). An
express a project vision that is connect- ment framework. This process is understanding of the project manage-
ed through measurement to tangible referred to as performance manage- ment process, its phases, and the
business outcomes. In response to this ment and is common within corpora- appropriate methods to manage deliv-
deficit of clarity, we set out a succinct tions. We find it is also a critical erables has clearly been the emphasis
methodology that involves a BSC deliverable for project leaders interest- of the early evolution of our discipline.
framework — modified for the project ed in on-strategy project delivery and However, we must not over-emphasize
management context — to assist lead- that it is essential to link project out- the management aspects without due
ers who are facing this dilemma. comes to corporate strategy using regard for the essentials of leadership
Although our research used the BSC measurement as the enabler. in a project management context.
framework, we believe that any appro- To address this increasing need for We suggest that project manage-
priately defined and balanced per- leaders to operationalize strategy ment professionals acknowledge that
formance management system could through projects, we hypothesized that the management of a project is the
potentially be substituted and would a project-level BSC might enable lead- easier of the two things to accomplish.
produce a similar effect. ers to use appropriate performance Task and process management is rela-
measurement and leadership tech- tively easy to learn and can be applied
A Balanced Scorecard Approach to niques, which already exist in other routinely; leadership of a project,
Project Management parts of most corporations, to help however, is a different matter altogeth-
We now turn our attention to the spe- project teams improve their under- er. In their landmark work Project
cific methods for operationalizing standing of their organization’s busi- Leadership (Briner et al., 1996, p. 67)
business strategy. The BSC is one tool ness strategy. emphasize the role of a “sustainer” as
— extensively developed, tested, and Our purpose was to develop, test, a key aspect of successful project spon-
demonstrated — that has proven its and apply a strategic measurement sys- sorship. They also stress the need for
value within corporate settings. This tem, based on the BSC methodology, project managers to orient themselves
instrument transforms strategy into specifically for projects. We anticipated towards alignment and away from
operational plans and strategic meas- that this approach would itself be enforcement—an elusive concept of
ures that enable the organization to immediately valuable to project man- trying to create congruence among the
decide whether or not a project is oper- agers and senior executives of large team and with the project’s goals by
ating on-strategy. This outcome occurs corporations who are faced with using a variety of activities and sources
regardless of situational ambiguity increasingly complex issues related to of power to influence others to act in
about any one individual’s under- ensuring a timely understanding of accordance with the project leaders’
standing of — or agreement with — business strategy among disparate and desired outcomes, rather than relying
the underlying strategy that led to the dispersed global project teams—teams on a traditional command-and-control
definition of these measures. One that may have varying degrees of abili- management orientation. We submit
major benefit of this approach is its ty and interest in understanding the that the use of a project BSC offers
capability to take a very complex, often company’s overall strategy. Using a organizations a powerful, additional
ill-defined business strategy and project BSC can help address a project tool for accomplishing this critical
reduce it to a level of specific measure- vision gap by making strategy easier to success factor and improves project
ments that shows stakeholders and understand in a practical rather than team alignment.
team members their particular and theoretical form. The idea is that this Our collective experience in work-
expected contribution to its ultimate helps the company improve its com- ing worldwide with corporate clients
achievement. This tool creates tangible petitive position by ensuring that proj- has shown us that many project man-
value from measurement and implies ect managers pursue the on-strategy agers are not fully schooled — either
taking vague notions of strategy and and on-quality aspects of project man- in formal academic settings or
turning these into executable plans. agement with the same level of effort through on-the-job project experience
However, while proven at the cor- and vigor they direct to on-budget and — to necessarily note the subtle but
porate level, to the best of our knowl- on-time concerns. By making this exer- important differences between per-
edge only a few researchers have cise easier to do and more visible at the ceived power and actual power.
published papers (Stewart, 2001; project level, this overall objective is Learning to distinguish between influ-
Stewart & Mohamed, 2001) about more readily achieved. ence and control to achieve results
applying this methodology to IT proj- We found that successfully com- often means the difference between
ects. These papers discussed extending pleted projects required a task manage- temporarily controlling an outcome

50 • Project Management Journal December 2004

by forced compliance versus creating a 1. In a project context, the role of States and Europe; it specializes in
lasting change in people’s behavior the BSC must change from business services and generates annual
(Greiner & Schein, 1988; Kotter, 1999; measuring the overall achieve- revenues in excess of US$300 million.
Loosemore, 1999; Pinto, 1998). Some ment of strategic objectives to The firm was a good research candi-
project managers, however, may not measuring the specific results of date because of its representative
see the two techniques as very differ- the project and comparing these nature and because it allowed us to ini-
ent. As a result, we often observe that to the project’s intended impact tiate and follow two similar-scope and
project managers eventually come to on the organization’s execution similar-budget projects. These projects
the conclusion that they really cannot of its business strategy. gave us the opportunity to test the
be everywhere at once to vet every deci- 2. Instead of only focusing on impact of using BSC in a project man-
sion to ensure that the team appropri- mapping business strategy, we agement setting.
ately conducts itself in performing its must map the intersection of Each of the two candidate projects
roles and realizing its project. As a the project strategy and busi- (code names Blue and Pip) met the cri-
result, most managers revert to some ness strategy and more closely teria specified earlier in terms of each
kind of exception-based or situational align these strategies as a result being vital to the company’s business
leadership method to address ongoing of this review and use the proj- strategy. The project teams were virtu-
challenges, as recommended by estab- ect BSC as a tool for leadership; ally identical in size: Between 40 and
lished theory (Hersey, Blanchard, & doing so tests this alignment 45 full-time personnel teaming with
Johnson, 1996). While somewhat issue more directly. various consultants and contractors
effective, this tactic does not complete- 3. We must modify our approach that were required to provide key proj-
ly address the issue. to measure specific project-relat- ect deliverables. A number of these
Results from an earlier study ed deliverables and objectives, as individuals were Project Management
(Hersey et al., 1996) suggest that a opposed to higher-level business Professional (PMP®) certified. Both
project level BSC can also become a outcomes, and appropriately set- projects had estimated timelines —
tool that provides an indirect form tle on project-based measure- from initiation to completion — of 9
of influence on daily decision-mak- ments that link to strategy. to 12 months; both had multi-million
ing within a project team: This tool dollar budgets.
is perhaps more powerful than other Once these discernible differences During the project period, we
methods of influence. By demand- are understood and applied, the basic closely monitored both project teams
ing that project team members link steps of the original methodology and consulted with each team’s global
their own actions and decisions with remain the same and are applied simi- project managers and executive spon-
the overall intended strategy of the larly. In the interests of brevity, we have sors. The Blue team was managed using
project (which, in turn, is an exten- chosen to avoid a detailed discussion the firm’s existing project management
sion of the corporate strategy, if the of the specifics of the original BSC methodology and according to existing
BSC methodology was consistently approach in this article to instead company practices. These methods and
applied), can assist with on-strategy focus on its benefit in a project setting. practices were generally quite consis-
project execution. It extends a virtu- The hypothesis that we were inter- tent with — and compliant to — stan-
al leadership presence, which injects ested in testing was whether or not the dard professional practices, as specified
itself into every critical project event BSC would have a discernible impact in A Guide to the Project Management
and decision. While this requires on the project team’s understanding of Body of Knowledge (PMBOK® Guide)
additional exploration, this method- business strategy and the specific pro- (Project Management Institute, 1996).
ology could, if proven in future stud- ject’s connection to strategy—a critical Table 1 summarizes this project’s final
ies, represent a fairly significant point in project management leader- on-time and on-budget results.
breakthrough in the decision-mak- ship. Therefore, the null hypothesis we The concept of the BSC was intro-
ing methods used for everyday proj- seek to reject (at a 95% confidence duced to the Pip team during initial
ect operations. level) is that the BSC has no significant training workshops that preceded the
The sequence that is used to impact on project management out- final project planning stages. These ses-
build a BSC, originally designated comes and does not positively impact sions, which lasted two days, were held
by Kaplan and Norton (1998a, the project team’s performance. on the company’s premises. The team
1998b), is well documented. In the also learned, over a period of three
literature, what this sequence pro- The Pilot Study days, the process of extending and
duces is often referred to as a strategy We initiated our research while one of integrating the methodology into the
map, a form of causal model. If we us was providing strategic project man- company’s existing project manage-
intuitively move to apply this exist- agement consulting services to a glob- ment framework. Pip (the group using
ing methodology in a project con- al telecommunications firm. The firm the BSC) showed actual final project
text, we must consider several has an extensive network and a partic- outcomes in the three key result areas
important changes: ularly large presence in the United — on-time, on-budget, and on-quality

December 2004 Project Management Journal • 51

— that were better than those achieved During our post-project interviews compared individual results to
by team Blue. While it is not possible in January 2001, we gathered addition- respondents’ answers, in terms of
to categorically state that this is all due al anecdotal evidence about the quali- their job title and the length of time
to the use of this methodology, we tative impact of using the BSC. These — measured in months — that each
observed strong evidence that shows interviews involved both in-person and respondent served on a project
the BSC had a positive impact on proj- over-the-phone discussions with mem- team. The survey results do not indi-
ect performance. The use of this tool bers of the firm’s executive team who cate any meaningful variations
did create a higher degree of owner- were directly involved in this research. based on demographics or project
ship and involvement from the team tenure, which suggests that the
in the execution of the project than in Results and Discussion value of the methodology was con-
the non-scorecard project, as measured We applied a variety of statistical sistently experienced within our
by an executive assessment during on- techniques to our survey data. We small sample group.
site post-project interviews. The influ-
ence of this methodology on executive Specific key result Blue (non-BSC using Pip (BSC using Actual %
behavior is a significant finding in its group) group) difference
own right.
We surveyed each project team with On-budget delivery 112 % (US$9.6 million 98% (US$12.1 million 14%
a particular interest in measuring any actual vs. US$8.6M actual vs. US$12.3M
apparent differences between the teams budget) budget)
that we could attribute to the use of the
BSC. (Appendix A shows the question- On-time delivery +122% (11 months actual +110% (11 months 12%
naire that we used during this study.) versus 9 months planned) actual versus 10 planned)
This method closely follows the case
Completed 94% (36 of 38 deliverables 95% (18 of 19 deliverables 1%
study research method proposed by Yin
deliverables were 100% complete) were 100% complete)
(1994), which added a substantial
amount of qualitative insight to our
Table 1. Actual project results
study’s less-reliable quantitative results.
The sample of 34 returned surveys from
among the approximately 40 individu-
als who worked on the two projects,
while an excellent return rate, is not sta- ANOVA Analysis* for Sum of squares Sum of DF Mean F Sig.
tistically representative of the total pop- project results squares
ulation of respondents, and so it is a questions
limiting factor that could not be
addressed by an alternate research Questions 1 & 11: Between groups 7.912 1 7.912 13.975 .001
On-time delivery Within groups 18.117 32 .566
design in this instance. As a result, the
Total 26.029 33
reliability of the sample — measured
using Cronbach’s Alpha — is a low .63; Questions 2 & 12: Between groups 8.147 1 8.147 14.391 .001
this result is a function of the limited On-budget delivery Within groups 18.117 32 .566
variability in the responses to individual Total 26.265 33
items from such a small sample. This
meant that using statistical tests relevant Questions 3 & 13: Between groups 24.308 1 24.308 35.810 .000
for a small sample size inhibited mean- On-strategy delivery Within groups 21.722 32 .679
ingful reporting of extensive statistical Total 46.029 33
variations between or among groups.
The primary benefit of the sur- Questions 4 & 14: Between groups 48.074 1 48.074 63.671 .001
veys was to identify specific aspects of Executive/Sponsor Within groups 24.161 32 .755
traditional project management prac- Communication Total 72.235 33
tice that appear to be most influenced
Questions 6 & 16: Between groups 17.152 1 17.152 20.156 .000
by the application of the BSC in a
External communication Within groups 27.231 32 .851
project context (see Table 3). This
Total 44.382 33
knowledge may allow future
researchers to more specifically iden- *The two independent sample t-tests require that the difference between the two samples is distributed. This may well not
tify the best practices that can impact be the case here, so the Mann-Whitney test (which does not require this assumption) was also conducted, with similar
the various elements of traditional predictions for each difference tested.

project management practice that

most companies use today. Table 2. ANOVA Analysis of responses between groups

52 • Project Management Journal December 2004

 Factor Analysis F value P value (Project Management Institute, 1996)
(Our determination of significant factors was based on means- were positively impacted by the team’s
between-groups, using ANOVA analysis at .05 confidence level use of the BSC at the project level. This
is useful not only as a conclusion in
1. Clear project vision 6403.765 .000 this study but also to help other
2. Clear goals & objectives 18158.824 .000 researchers facilitate future research
3. Clearly communicating project status 18158.824 .000 that confirms our initial findings. Our
4. Creating detailed project plans 22162.824 .000 analysis supports the ways that the BSC
7. Assuring adequate project resources 22162.824 .000
measures appear to improve project
9. Assuring an appropriate project team structure 8562.071 .000
performance in areas such as project
10. Status reporting 6403.765 .000
11. Project tracking & control 10094.374 .000 communication, monitoring and con-
12. Executive/sponsor involvement 6403.765 .000 trol, status reporting, and resource
15 Implementing good risk management practices 5798.118 .000 management.
While we, along with the project's
Table 3. Project factors impacted by the balanced scorecard participants, observed meaningful qual-
itative contributions — during project
While both projects ran longer hypothesis and therefore conclude that execution — in each of the 15 areas list-
than initially planned, both groups the BSC did have a positive impact on ed in Table 3, our statistical analysis of
reported high self-report scores for the Pip team’s project management per- this data makes the differences mean-
"On-time delivery" in the question- formance in at least some of the antici- ingful at the prescribed confidence
naire. In our view, this is explained by pated areas we set out to investigate. level, thus allowing us to also reject the
the tendency of this company to set Summarizing the results above null hypothesis that the BSC had no
very aggressive timelines that project into a simple enumerated list indicates impact on project performance.
teams often do not feel are realistic or that the factors most strongly impacted Therefore both quantitatively and qual-
reasonable. Delivery times, as a conse- by the team using the BSC were: itatively we find support for the value of
quence, even close to these very aggres- 1. On-time delivery (question #1) this approach in project management.
sive targets can be perceived as a 2. On-budget delivery (question #2)
reasonable approximation of accept- 3. Executive/Sponsor communica- Conclusions
able on-time performance by the proj- tions (question #14) Our pilot study suggests that moving
ect teams. While statistically this data 4. External communications beyond existing internal project com-
only weakly supports the impact of the (question #16) munication practices, and using a
BSC on actual project performance, we BSC framework to make strategic
believe that it intuitively supports a The Pip team also felt that it had a measures and connections clearer, can
notional conclusion showing the direct impact on on-strategy delivery; possibly improve a project team’s
potential positive impact of the BSC however, this belief is biased on the internal performance in traditional
on project performance. BSC method training they received, deliverables: on-time, on-budget, and
As previously noted, with the which deliberately cultivated this on-quality. We propose that this is
small sample sizes available for this belief in order to gain the team’s com- because using the BSC in project set-
research, we found it difficult to con- mitment to piloting the BSC approach. tings facilitates a wider perspective on
duct reliable parametric analysis. Therefore, we have recognized this bias project management successes and
However, if we assume that the normal and have not drawn this conclusion as facilitates a team’s linking to a wider
mean of responses between the two a result. range of strategic performance indica-
groups is the same (what we believe is We expected this final outcome tors that it can use to appropriately
an intuitively acceptable assumption, because the entire purpose of applying develop a clearer project vision and to
based on our knowledge of the com- the methodology is to enable the proj- more clearly monitor and control
position of the project teams in ques- ect team to directly connect their proj- individual project goals and objec-
tion), then it becomes possible to ect goals and objectives to measurable tives. Results from our research fur-
conduct a basic analysis on each strategic outcomes. The Pip team’s ther suggest that the BSC framework
group’s responses to key questions. ability to do this consistently improves provided project management teams
Even given the limitations of a small its capacity — both internally within with additional benefits; however,
pilot study, Table 2 shows how the the team and externally to the team — some of these benefits require more
application of the BSC had a definite to communicate about project out- attention and further study to draw
impact on some key aspects of project comes more succinctly. firm conclusions about the effective-
management practice. Most interesting was the Pip proj- ness of this framework.
On the basis of this additional ect team’s belief that the specific fac- One of the most clearly cited ben-
analysis, we find sufficient statistical tors (listed in Table 3 above) within efits of the BSC by the project team is
support in Table 2 to reject the null the generally accepted PMBOK® Guide that it is a tool for communicating

December 2004 Project Management Journal • 53

with internal and external project References 68(3),103-111.
stakeholders. The scorecard itself is Baccarini, D. (1999). The logical Kotter, J. P. (1995). Why transforma-
less significant than the value of its framework method for defining project tion efforts fail. Harvard Business Review,
application so long as the process of success. Project Management Journal, 30(4), 73(2), 59-67.
developing and using this tool 25-32. Kotter, J. P. (1999). John P. Kotter on
increases the effectiveness of commu- Bennis, W. (1989). On becoming a what leaders really do. Boston: Harvard
nication between the project sponsor, leader. San Francisco: Peresus Publishing. Business School Press.
the project management team mem- Bennis, W., & Nanus, B. (1997). Loosemore, M. (1999).
bers, and other project stakeholders. Leaders strategies for taking charge. New Responsibility, power and construction
This information was confirmed often York: Harper Business. conflict. Construction Management and
during our post-project interviews Bennis, W., Spevietzer, G., & Economics, 17(6), 699-709.
with all of these groups. As a result of Cummings, T. G. (2001). The future of lead- Morris, P. W. G. (1994). The manage-
having participated in the develop- ership. San Francisco: Jossey-Bass. ment of projects a new model. London:
ment of the project scorecard, we pro- Briner, W., Hastings, C., & Geddes, M. Thomas Telford.
vided each team member with a (1996). Project leadership. Aldershot, UK: Pinto, J. K. (1998). Power & Politics in
common communication template to Gower. project management. Sylva, N.C.: Project
articulate project performance meas- Cleland, D. I. (1986). Measuring suc- Management Institute.
ures and results. We argue that this cess: The owner’s viewpoint. Proceedings of Project Management Institute.
created a more powerfully committed the Project Management Institute’s Annual (1996). A Guide to the Project Management
team that embodied a deep sense of Seminar & Symposium, Montreal, 6-12. Body of Knowledge. Upper Darby, PA:
purpose and vision (the essence of Cleland, D. I. (1999). Project manage- Project Management Institute.
good project leadership). ment: Strategic design and implementation. Robbins, H., & Finlay, M. (1997).
We also argue that using the BSC Singapore: McGraw-Hill Education— Why teams don’t work - What went wrong
in the way we have indicated has the Europe. and how to make it right. London: Orion
benefit of making complex strategy Collins, J., & Porras, J. I. (1996). Publishing Group Ltd.
more understandable at the opera- Building your company's vision. Harvard Senge, P. M. (1990). The fifth discipline
tional level in terms of specific oper- Business Review, 74(5), 65-78. - The art & practice of the learning organiza-
ating targets for the project (project Davis, K. H. (1995). Logical framework tion. Sydney: Random House.
management). The importance of analysis: A methodology to turn vision into Stewart, R. A., & Mohamed, S. (2001).
achieving this objective cannot be reality. Proceedings of the AIPM National Utilizing the balanced scorecard for IT/IS
underestimated since it is the essential Conference, Adelaide, Australia, 393-397. performance evaluation in construction.
value-adding element of improved Drummond, H. (1998). Riding a Journal of Construction Innovation, 1(2),
project management practices within tiger: Some lessons of Taurus. Management 147-163.
an organization. Decision, 36(3), 141-146. Stewart, W. E. (2001). Balanced score-
We therefore suggest that the Greiner, L. E., & Schein, V. (1988). card for projects. Project Management Journal,
process of building a project BSC Power and organization development: 32(1), 38-53.
brings added depth and reliability to Mobilizing Power to implement change. New Thite, M. (1999). Leadership styles in
any project’s business case and to York: Addison-Wesley Management. information technology projects.
management’s ability to provide Hersey, P., Blanchard, K., & Johnson, International Journal of Project Management,
oversight for achieving the totality D. E. (1996). Management of organizational 18(2), 235-241.
of project benefits, the ultimate goal behavior: Utilizing human resources. London: Tuman, J. (1986). Success modeling: A
of any well-formed project manage- Prentice Hall International. technique for building a winning project
ment methodology. As in a corpo- Kaplan, R. S., & Norton, D. P. team. Proceedings of the Project Management
rate setting, the BSC adds value to (1998a). Putting the balanced scorecard to Institute’s Annual Seminar & Symposium,
participants and promotes more reli- work. Harvard Business Review on Measuring Montreal, 94-108.
able communication and more Corporate Performance,147-181. Turner, J. R., & Cochrane, R. A. (1993).
effective decision-making. We pro- Kaplan, R. S., & Norton, D. P. The goals and methods matrix: Coping with
pose that the BSC — with a demon- (1998b). Using the balanced scorecard as projects with ill-defined goals and/or meth-
strated ability to make a strong a strategic management system. Harvard ods of achieving them. International Journal of
connection between business strate- Business Review on Measuring Corporate Project Management, 11(2), 93-102.
gy and project vision through an Performance, 183-211. Yin, R. (1994). Case study research.
understanding of measures and met- Katzenbach, J. R., & Smith, D. K. Thousand Oaks, CA: Sage.
rics that guide coordinated and (1993). The wisdom of teams - Creating the Yukl, G. (1998). Leadership in organiza-
deliberate action at the project level high-performance organization. Boston: tions. Sydney: Prentice-Hall.
— is a valuable tool for project man- Harvard Business School Press. Zaleznik, A. (1977). Managers and lead-
agement professionals to adopt and Kotter, J. P. (1990). What leaders ers: Are they different? Harvard Business
make their own. really do. Harvard Business Review, Review, 55(3), 67-78.

54 • Project Management Journal December 2004

 Appendix A – Project Questionnaire

In our continuing quest to improve our methodologies and results with clients, we would appreciate you taking a few
minutes to complete the following brief questionnaire. It is designed to have you describe your experiences on this
project and your use of the Align360 Project Balanced Scorecard, if applicable. We are interested in understanding its
value as a project management tool. Please return this questionnaire immediately to your Align360 project manager.

Demographics
1. Project you are involved with:
____________________________________________________________________________

2. Have you been with the project since the beginning? YES NO

3. If not, please indicate in number of months you have been on the project: __________________ months.

4. Your title/level within the organization (optional): ______________________________________________

Your specific project results

Please tell us how well you think your project team did on this project. Using a scale of 1 to 5, with 1 being poor and
5 being excellent, please rate the following project outcomes.

QUESTION RATING

1. We delivered our project on-time. 1 2 3 4 5

2. We delivered our project on-budget. 1 2 3 4 5
3. We achieved our project’s strategic objective for the company. 1 2 3 4 5
4. Our project team understood how the project fit into the big picture. 1 2 3 4 5
5. We had clear goals and objectives for the project that all project team members understood. 1 2 3 4 5
6. We communicated well with others in the company about our project. 1 2 3 4 5
7. We understood and mitigated project risk as best as we could. 1 2 3 4 5
8. By our team standards, I would consider this project well managed. 1 2 3 4 5

9. Of the factors listed below, please circle any that you feel are critical to managing a project to its successful
conclusion (measured as on-time, on-budget, and on-strategy). Please circle all factors that apply.

1. Clear project vision 2. Clear goals & objectives 3. Clearly communicating project status

4. Detailed project plan 5. Detailed work breakdown 6. Detailed staffing plan

7. Adequate resources 8. Formal budget 9. Appropriate project team structure

10. Status reporting 11. Project tracking & control 12. Executive/Sponsor involvement

13. Contingency funds 14. Good project methodology 15. Good risk management practices

10. If you could identify only one of the 15 factors listed in question #9 that is frequently, in your experience, not
performed well on projects, which factor would you chose (please identify the number and the description):

# ______________ Factor Description: ________________________________________

December 2004 Project Management Journal • 55

 Pg 2/… (Note: Only respond to this section if your team used a project balanced scorecard.)

On a scale of 1 to 5, please rank your understanding of the impact that using the project balanced scorecard had on
the project you were working on. On this scale, 1 means virtually no impact or difference from other projects you have
worked; 5 means the scorecard had an immediate and measurable impact on the project’s results.
19. Of the factors listed below, please circle any that you feel were strongly impacted by your team’s use of the proj-
ect balanced scorecard (a measurable impact to on-time, on-budget, or on-strategy delivery, as a result of using this
tool for this project). Please circle all that apply.

QUESTION RATING

11. The project balanced scorecard improved our on-time delivery. 1 2 3 4 5

12. The project balanced scorecard improved our on-budget delivery. 1 2 3 4 5
13. The project’s strategic goals were clearer to me using the scorecard. 1 2 3 4 5
14. It improved my ability to communicate with executives about the project’s progress. 1 2 3 4 5
15. It improved my ability to communicate with my team about the project’s progress. 1 2 3 4 5
16. It improved our project team’s ability to communicate with each other. 1 2 3 4 5
17. It made us more aware of project risks. 1 2 3 4 5
18. It helped me manage the project better. 1 2 3 4 5

1. Clear project vision 2. Clear goals & objectives 3. Clearly communicating project status
4. Detailed project plan 5. Detailed work breakdown 6. Detailed staffing plan
7. Adequate resources 8. Formal budget 9. Appropriate project team structure
10. Status reporting 11. Project tracking & control 12. Executive/Sponsor involvement
13. Contingency funds 14. Good project methodology 15. Good risk management practices

20. If you could choose only 1 of the 15 factors for project success listed in question #19 that was most
impacted by your use of the project balanced scorecard, which one would you chose
(please note the number and the description below):

# ________ Factor description: __________________________________

21. Would you use this tool again for future projects? YES NO

22. Please write, in the space below, any additional comments about using the project balanced scorecard
that you wish to pass along to us:

__________________________________________________________________________________________________

JAMES L. NORRIE as an academic and a consultant, Mr. DEREK H. T. WALKER is a professor in the
Norrie has published and presented numerous papers School of Business at RMIT University and
on topics related to information technology strategy, the program director for that university’s
project management, and corporate leadership and Doctor of Project Management program.
governance. As director of the School of Information He is also the director of Research for the
Technology Management (ITM) at Toronto’s Ryerson CRC in Construction Innovation in
University (Ryerson University is one of the biggest of Australia. A graduate of RMIT, Aston
its kind in North America and the largest ITM program University, Swinburne Institute of
in Canada), Mr. Norrie teaches undergraduate ITM Technology, and the University of
courses in systems analysis & design, IT strategy and Glamorgan, Dr. Walker has written more
management, ethics and professional practice, and than 80 peer-reviewed papers and 17
project management at both the intermediate and book chapters on topics relating to
advanced levels. He is currently a Doctorate in Project project management.
Management candidate at RMIT University in
Melbourne, Australia and a current member of the
Project Management Institute.

56 • Project Management Journal December 2004