This may be the author’s version of a work that was submitted/accepted

for publication in the following source:

Sawang, Sukanlaya
(2011)
Key performance indicators for innovation implementation: Perception vs.
actual usage.
Asia Pacific Management Review, 16(1), pp. 23-29.

This file was downloaded from: https://eprints.qut.edu.au/28486/

c Copyright 2011 College of Management, National Cheng Kung Uni-

versity

This work is covered by copyright. Unless the document is being made available under a
Creative Commons Licence, you must assume that re-use is limited to personal use and
that permission from the copyright owner must be obtained for all other uses. If the docu-
ment is available under a Creative Commons License (or other specified license) then refer
to the Licence for details of permitted re-use. It is a condition of access that users recog-
nise and abide by the legal requirements associated with these rights. If you believe that
this work infringes copyright please provide details by email to [email protected]

Notice: Please note that this document may not be the Version of Record
(i.e. published version) of the work. Author manuscript versions (as Sub-
mitted for peer review or as Accepted for publication after peer review) can
be identified by an absence of publisher branding and/or typeset appear-
ance. If there is any doubt, please refer to the published source.

http:// apmr.management.ncku.edu.tw/ comm/ updown/ DW1103213952.

pdf
 Asia Pacific Management Review 16(1) (2011) 23-29

www.apmr.management.ncku.edu.tw

Key Performance Indicators for Innovation Implementation: Perception vs.

Actual Usage

Sukanlaya Sawang*
School of Management, Queensland University of Technology, Australia
Received 18 April 2009; Received in revised form 11 June, 2009; Accepted 7 September 2009

Abstract

This paper aimed to explore the proportion associated with the perceived importance and
the actual use of performance indicators from manufacturing and non manufacturing
industries. The sample was 86 small and medium sized-organizations in Thailand. The
perceived importance and the actual use of financial and non-financial indicators were found
to be significantly related among manufacturing and non-manufacturing industries. Three key
performance indicators (KPIs) (i.e. sales and sales growth, quality of products and/or services,
and process time) were perceived the most importance among manufacturing managers
(85.3%, 79.4% and 76.5% respectively). Three KPIs (i.e. customer satisfaction, quality of
products and/or services; and process time) were perceived the most importance among non-
manufacturing managers (84.8%, 93.5%, and 84.8% respectively). Interestingly, the most
used KPIs for manufacturing were sales and sales growth (64.7%); profit margins (61.8%);
and customer satisfaction (84.8), while non-manufacturing used quality products/services
(60.9%); sales and sales growth (54.3%) and employee development (54.3%) respectively.
Limitation and implication were also discussed.

Keywords: Key performance indicator, innovation, SME, performance

1. Introduction

This study aims to provide a tentative framework for exploring a proposed relationship
between perceived importance and the actual use of financial and non-financial indicators
capturing technical and administrative innovation performances among small and medium-
sized organizations in Thailand. In this study, an innovation is considered to be “A technology
or practice that an organization is using for the first time, regardless of whether other
organisations have previously used the technology or practice” (Klein et al., 2001).
Key performance indicators (KPIs) help an organization define and measure progress
toward organizational goals. KPIs are quantifiable measurements to examine the
improvement in performing an innovation implementing activity that is critical to the success
of a business (Cox et al., 2003). Innovation adopters often assume that investments in
innovation will lead to productivity improvements. However, investment in innovation does
not guarantee the effective implementation. Previous research has often shown that adopted
innovations (e.g. Lean production, Customer Relation Management (CRM), Enterprise
Resource Planning (ERP), and Rapid Prototyping) fail to successfully complete the
implementation phase and perceive the improvements (Dennis, 2003). Shiba et al.(1993)
indicated that the implementation effort must be monitored and diagnosed. This is essential
because it enables individuals and groups to assess where they stand in comparison to their
*
Corresponding author : E-mail: [email protected]

23
 S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

competitors. In addition, assessing innovation performance provides the opportunity of

recognising problems and weaknesses and taking corrective action before these problems
escalate (Kueng, 2000). Moreover, assessing innovation performance enables organisation
claiming their implementation effectiveness accurately. As a result, organisation, perhaps, has
been growing and productive because of the innovation, but traditional measurements have
failed to capture that growth.
Many of the measurements used today are those developed by accountants and are
financially oriented (Barbosa, 2004; Norman and Bahiri, 1972). There are organisations that
attempt to evaluate the worth and effectiveness of innovation implementation by adopting
financial measurement (e.g. return on investment). These traditional evaluations do not
provide feedback on the effectiveness of internal process, and employees perspectives (Abdel-
Maksound, 2004; Kaplan and Norton, 1992).
No single performance indicator can capture the full complexity of an innovation
performance (Amaratunga et al., 2001). In order to be successful in today’s world-wide
competitive environment, organizations must be capable of capturing objective (e.g. unit cost,
profit) and subjective (e.g. quality, satisfaction) performances. Subsequent research examined
the effectiveness of balancing financial and non-financial measurement on performance
measurement system (Bremser and Barsky, 2004; Hudson et al., 2001; Kanji and Sa', 2002;
Kerssens-van Drongelen and Bilderbeek, 1999; Savioz and Blum, 2002). Questions were
asked as to whether organization realized the importance of this issue and whether
organizations implement non-financial indicator as well as financial indicator to measuring
innovation performance. This paper aimed to explore the proportion associated with the
perceived importance and the actual use of performance indicators from manufacturing and
non-manufacturing industries. The previous studies indicated the positive relationship
between managerial perception and the use of key performance indicators within organization
(e.g. Cox, et al., 2003; Kaplan and Norton, 1993). Thus, the present study hypothesized that
there is a positive correlations between perceived importance and the actual usage of
performance indicator such that organizations which perceive the importance of performance
indicators would utilize such indicators to measure innovation performance.

2. Methodology

2.1 Participant
The sample was 86 small and medium sized-organizations in Thailand, based mainly in
Bangkok and suburban areas. These companies have adopted and implemented technical
and/or administrative innovations in the past three years. Of these, 34% were in the
manufacturing industry, 46% were in the non-manufacturing industries (e.g. automotive
supplies, construction, pharmaceuticals, and telecommunication). The sample organizations
(40%) had the average organizational gross value during year 2003-2004 of 50-200 million
baht. Twenty eight percent of sample organizations had the range of employees of 101-200
people. The mail survey sent to the CEO or Managing Director of each organization; the aim
of the questionnaire was to explore the measurements, which are used for capturing technical
and administrative innovation performance, and how managers perceived the importance of
these performance indicators.
2.2 Performance metrics
The performance metrics which are commonly used in business fall into two categories: (a)
finance-based, such as return on investment, and (b) non-finance-based, such as customer
retention, productivity, and employee development. By examining the innovation literature,
the common 14 performance metrics (Table 1) were employed in the past research to measure

24
 S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

the effectiveness of both product and processes innovation (Ahmad and Dhafr, 2002; Aycock
et al., 1999; Bremser and Barsky, 2004; Luria and Wiarda, 1996; Prajogo and Sohal, 2001).
Thus, the current study employed the same set of performance metrics which derived from the
previous innovation studies.

Table 1. List of the studied KPIs.

KPI Definition
KPI 1 Return on investment
KPI 2 Various profit margin measures
KPI 3 Sales and sales growth
KPI 4 Payback and payback period
KPI 5 Cash flow
KPI 6 Customer satisfaction
KPI 7 Customer retention rate
KPI 8 Labour productivity
KPI 9 Quality of products and/or services
KPI 10 Lead time
KPI 11 Delivery reliability and/or speed
KPI 12 Process time
KPI 13 Employee development
KPI 14 Employee knowledge

Respondents were asked whether they think these metric were important for their
organisation and whether their organisation used these metrics to measure innovation
effectiveness. Because some respondents may report using these metrics to measure
innovation effectiveness but may not actually use them, we employed a technique designed to
decrease social desirability effects. To ensure accurate reporting, participants were also asked
to identify the actual measurement systems used to collect information on each metric. For
example, respondents who reported that they measured employee morale may mention a
measurement record system such as an employee survey as the tool used to obtain this
information.
The translation of the questionnaire into Thai language was accomplished through a two-
stage translation-back translation procedure. First, I translated the questionnaire from English
into Thai. The Thai version was then back-translated into English by a bilingual volunteer,
who was not aware of the purpose of the study. Following this, the original questionnaire was
compared with the back-translated English version, and differences resolved through
discussion. This process ensured an accurate, literal translation of the original English
language version of the questionnaire.

3. Results

The two-way cross tabulation chi-square analysis was conducted to evaluate whether
organizations which perceived the importance of performance indicators would utilize such
indicators to measure innovation performance. The perceived importance and the actual use of
financial indicators were found to be significantly related among manufacturing and non-
manufacturing industries, Pearson χ2(25, 34) = 54.27, p < 0.001; and Pearson χ2(25, 46) =
78.58, p < 0 .000 respectively.

25
 S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

Correspondingly, the perceived importance and the actual use of non-financial indicators
were also found to be significantly related among manufacturing and non-manufacturing
industries, Pearson χ2(81, 34) = 133.80, p < 0.001; and Pearson χ2(80, 45) = 173.50, p < 0.001
respectively. Follow up pairwise comparisons were conducted to demonstrate the different
proportion of each indicator. Figures 1 and 2 show the results of these analyses.

Perceived importance and the actual use of indicators to measuring the innovation performance
(Manufacturing)

90

80

70

60
Percentage

50
Actual Use No
Actual Use Yes
40

30

20

10

0
Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No
KPI1 KPI2 KPI3 KPI4 KPI5 KPI6 KPI7 KPI8 KPI9 KPI10 KPI11 KPI12 KPI13 KPI14
Perceieved importance of KPIs

Figure 1. Perceived importance and the actual use of indicators to measuring

the innovation performance (Manufacturing).

Perceived importance and the actual use of indicators to measuring the innovation performance
(Non manufacturing)

100

90

80

70

60
Percentage

Actual Use No
50
Actual Use Yes

40

30

20

10

0
Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No
KPI1 KPI2 KPI3 KPI4 KPI5 KPI6 KPI7 KPI8 KPI9 KPI10 KPI11 KPI12 KPI13 KPI14
Perceieved importance of KPIs

Figure 2. Perceived importance and the actual use of indicators to measuring

the innovation performance (Non-manufacturing).

26
 S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

KPIs 3, 9, and 12 (i.e. sales and sales growth; quality of products and/or services; and
process time) were perceived the most importance among manufacturing managers (85.3%,
79.4% and 76.5% respectively), while KPIs 6, 9, and 12 (i.e. customer satisfaction, quality of
products and/or services; and process time) were perceived the most importance among non-
manufacturing managers (84.8%, 93.5%, and 84.8% respectively). Interestingly, the most
used KPIs for manufacturing were sales and sales growth (64.7%); profit margins (61.8%);
and customer satisfaction (84.8%) while non-manufacturing used quality products/services
(60.9%); sales and sales growth (54.3%) and employee development (54.3%) respectively
(Figures 1 and 2). Although, some organizations may not perceive the importance of some
indicators, they used, in particular, traditional financial indicator (e.g. return on investment,
profit margin, pay back and cash flow) to measure the innovation performance.

4. Conclusion

We anticipated that organizations which perceived the importance of KPIs would employ
those KPIs to measuring the innovation performance. Chi-square analyses of the actual use of
KPIs yielded statistically significant differences among the level of perceived importance.
Mostly, organizations realized the importance of those financial and non-financial KPIs and
used those KPIs for capturing innovation progress. Surprisingly, organizations which did not
perceived the importance of some financial indicators, but they had used those indicators (e.g.
return on investment, profit margin, pay back and cash flow) for their innovation project.
This result can be explained by Jarvis et al.(2000). They found that financial measurement,
such as profit, cash flow, and turnover, were mentioned significantly among managers whom
they interviewed with. However, the results show that 38-40% of manufacturing managers,
who highly recognized the importance of non-financial KPIs (i.e. the quality or
products/services indicators, process time and employees’ development and knowledge), did
not employ those KPIs for the actual usage. Likewise, 41% of non-manufacturing managers,
who highly recognized the importance of customer satisfaction, did not use this KPI. As a
result, we have to acknowledge that there may be other potential moderators existing between
the managerial perception and the actual use of non-financial indicators. Further research
could be conducted to capture the moderating affect. Other limitations in our study are our
sample of organization was small with the low response rate (10%). Therefore, the stability of
our findings is uncertain.
The present study demonstrated the important of managerial perception toward the
performance indicators. Nevertheless, it has been realized that not all performance indicators
are equally reliable. Some indicators such as ROI can be considered as a lag indicator. This
means the actual gain of can be lagged a year later after the initial innovation implementation.
Additionally, some indicators can be difficult to make a good comparison with other
organizations. As a result, it has been recommended for organizations using multiple
indicators in order to track an improvement from the innovation project.

5. Implications

Assessing the innovation outcome is to ensure that the change initiative is successful et
al., 2001; Tushman and O'Reilly, 1997). As organizations which are implementing innovation
in order to survive in a world-wide competitive market, or even to be a world-class level, they
should always be interested in two questions: Does an organisational performance improve
after implementing the innovation? To what degree are the target values fulfilled? (Kueng,
2000). Empirical evidence indicates that using a proper performance measurement system is
critical to answer those two questions (Chiesa et al., 1996; Griffin and Page, 1993; Hudson et

27
 S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

al., 2001; Verhaeghe and Kfir, 2002). Based on our results, increasing awareness of balancing
financial and non-financial indicators among managers would enhance the actual utilization of
those indicators. The more managers perceive the significance benefits of KPIs, the more they
will use them to identify the innovation performance. Providing training, seminar or
discussion network regarding to these issues will develop the higher level of perceived
importance of financial and non-financial KPIs.

References

Abdel-Maksound, A.B. (2004) Manufacturing in the UK: Contemporary characteristics and

performance indicators. Journal of Manufacturing Technology Management, 15(2),
155-171.
Ahmad, M.M., Dhafr, N. (2002) Establishing and improving manufacturing performance
measures. Robotics and Computer Integrated Manufacturing, 18(3-4), 171-176.
Amaratunga, D., Baldry, D., Sarshar, M. (2001) Process improvement through performance
measurement: The balanced scorecard methodology. Work Study, 50(5), 179-188.
Aycock, D.G., Jean-Normand, D., Thomas, F.Y. (1999) TL 9000 performance metrics to
drive improvement. Quality Progress, 32(7), 41.
Barbosa, J.P. (2004) Assessment of PR manufacturing readiness with respect to
communication and information technology and its impact on productivity.
Unpublished PhD Thesis, University of Puerto Rico.
Bremser, W.G., Barsky, N.P. (2004) Utilizing the balanced scorecard for R&D performance
measurement. R&D Management, 34(3), 229-238.
Chiesa, V., Coughlan, P., Voss, C.A. (1996) Development of a technological innovation audit.
Journal of Production Innovation Management, 13(2), 105-136.
Cox, R.F., Issa, R.R.A., Ahrens, D. (2003) Management's perception of key performance
indicators for construction. Journal of Construction Engineering and Management,
129(2), 142-151.
Dennis, P. (2003) What if your feaf plan isn't working? Canadian Machinery and
Metalworking, 98(4), 39.
Ford, M.W., Evans, J.R., Matthews, C.H., Burchill, G. (2001) Theoretical foundation of the
seven infrastructures model. Center of Quality of Management Journal, 10(2), 51-76.
Griffin, A., Page, A.L. (1993) An interim report on measuring product development success
and failure. Journal of Product Innovation Management, 10(4), 291-308.
Hudson, M., Smart, A., Bourne, M. (2001) Theory and practice in SME performance
measurement systems. International Journal of Operations & Production Management,
21(8), 1096-1115.
Jarvis, R., Curran, J., Kitching, J., Lightfoot, G. (2000) The use of quantitative and qualitative
criteria in the measurement of performance in small firms. Journal of Small Business
and Enterprise Development, 7(2), 123-134.
Kanji, G.K., Sa', P.M.E. (2002). Kanji's business scorecard. Total Quality Mangement, 13(1),
13-27.
Kaplan, R.S., Norton, D.P. (1992) The balanced scoredcard-measures that drive performance.
Harvard Business Review, 70(1), 71-79.
Kaplan, R.S., Norton, D.P. (1993) Using the balanced scorecard as a strategic management
system. Harvard Business Review, 71(5), 134-148.
Kerssens-van Drongelen, I.C., Bilderbeek, J. (1999) R&D performance measurement: More
than choosing a set of metrics. R&D Management, 29(1), 35-46.
Klein, K.J., Conn, A.B., Sorra, J.S. (2001) Implementing computerzied technology: An
organization analysis. Journal of Applied Psychology, 86(5), 811-824.

28
 S. Sawang / Asia Pacific Management Review 16(1) (2011) 23-29

Kueng, P. (2000) Process performance measurement system: A tool to support process-based

organizations. Total Quality Management, 11(1), 67-85.
Luria, D., Wiarda, E. (1996) Performance benchmarking and measuring program impacts on
customers: Lessons from the Midwest Manufacturing Technology Center. Research
Policy, 25(2), 233.
Norman, R.G., Bahiri, S. (1972) Productivity Measurement and Incentives. Butterworths,
London.
Prajogo, D.I., Sohal, A.S. (2001) TQM and innovation: A literature review and research
framework. Technovation, 21(9), 539-558.
Savioz, P., Blum, M. (2002) Strategic forecast tool for SMEs: How the opportunity landscape
interacts with business strategy to anticipate technological trends. Technovation, 22(2),
91-100.
Shiba, S., Graham, A., Walden, D. (1993) New American TQM. Oregon Productivity Press,
Portland.
Tushman, M., O'Reilly, C.A. (1997) Winning through innovation: A practical guide to leading
organizational change and renewal Boston, Harvard Business School Press, MA.
Verhaeghe, A., Kfir, R. (2002) Managing innovation in a knowledge intensive technology
organisation (KITO). R&D Management, 32(5), 409-417.

29