The current issue and full text archive of this journal is available at

www.emeraldinsight.com/1741-0401.htm

IJPPM
59,2 A framework to assess
performance measurement
systems in SMEs
186
Paola Cocca and Marco Alberti
Department of Mechanical and Industrial Engineering, Brescia University,
Received March 2009
Revised July 2009 Brescia, Italy
Accepted September 2009

Abstract
Purpose – The purpose of this paper is to develop a framework that small and medium size
enterprises (SMEs) can use to assess their performance measurement system (PMS) in order to identify
the main weaknesses and take corrective measures.
Design/methodology/approach – Starting from a review of the recommendations provided in
literature regarding performance measurement in companies, a list of general characteristics of a
“good” PMS is defined. The identified PMS general features are then tailored to SME needs through an
analysis of the characteristics of SMEs and a survey of their performance measurement current
practice. Finally all the requirements are included in a framework designed to support SMEs in
critically reviewing their PMS and enabling its effective use.
Findings – The main characteristics of SMEs are reviewed and classified; the general characteristics
of a “good” PMS are collected from the literature and then adapted to the SME context, thus obtaining
a list of PMS requirements for SMEs; moreover a tool for PMS self-assessment is developed.
Research limitations/implications – Research is still at an early stage and applications of the tool
in real settings will offer the possibility to refine and validate it.
Practical implications – The tool proposed codifies best practices and makes them accessible by
SMEs in a simple way, thus supporting companies in the process of continuous improvement of their
PMS.
Originality/value – The field of performance measurement in SMEs is an emergent area of study.
The paper highlights the necessity of PMS assessment instruments suitable for SMEs and defines a
new tool starting from the analysis of SME needs and characteristics.
Keywords Performance measures, Small to medium-sized enterprises, Assessment
Paper type Research paper

1. Introduction
Nowadays companies are required to compete in globalized and turbulent markets. In
order to survive in such a dynamic environment they need to be able to satisfy all their
stakeholders and excel at the same time along all performance dimensions (Neely et al.,
2002).
A necessary condition to achieve high performance standards is being able to
effectively measure and monitor company’s performance. In fact, the truthfulness of
International Journal of Productivity some famous sayings like “What gets measure gets attention” or “What you measure is
and Performance Management what you get” is widely recognized both among academics and practitioners (Eccles,
Vol. 59 No. 2, 2010
pp. 186-200 1991; Kaplan and Norton, 1992). Therefore performance measurement systems (PMSs)
q Emerald Group Publishing Limited
1741-0401
are considered as a means to gain competitive advantages and continuously react and
DOI 10.1108/17410401011014258 adapt to external changes. According to Neely et al. (2002), “A Performance
Measurement System is the set of metrics used to quantify the efficiency and Performance
effectiveness of past actions” and “it enables informed decisions to be made and actions measurement
to be taken because it quantifies the efficiency and effectiveness of past actions
through the acquisition, collation, sorting, analysis and interpretation of appropriate systems in SMEs
data”. Furthermore they highlight that a PMS can be examined at three different levels:
the individual measures of performance; the performance measurement system as a
whole; the relationship between the PMS and the environment within which it operates. 187
To be effective a PMS should be dynamic and has to be modified as circumstances
change so that performance measures remain relevant (Lynch and Cross, 1991).
However few organizations appear to have systematic processes in place for assessing
and maintaining their PMS (Kennerley and Neely, 2002).
The ability of keeping the PMS continuously updated is a challenge for every firm,
but particularly for small and medium-sized enterprises (SMEs), which need to be
extremely flexible and reactive to market changes while being characterized by lack of
resources and managerial expertise (Garengo et al., 2007; Hudson et al., 2001). However,
most of the literature on performance measurement treats the subject regardless of the
size of the business (Carpinetti et al., 2008) and even if many performance measurement
approaches have been proposed, few are the publications focusing specifically on
performance measurement in small and medium-sized enterprises (Garengo et al.,
2007). At the same time, models developed for large enterprises seem not to apply well
to SMEs (Cassell et al., 2001), as confirmed also by the gap between theory and practice
observed by numerous authors in SME (Hudson et al., 2001; Sousa et al., 2006; Cocca
and Alberti, 2008).
The objective of this paper is to develop an assessment tool that SMEs can use to
evaluate the effectiveness of their PMS and identify directions for improvement. The
main assumption is that to develop a good tool for SMEs, it is necessary to start from a
detailed analysis of SME needs, taking into consideration SME characteristics. In fact,
in order to be relevant for SMEs, an assessment tool should not be a simple miniature
of the tools developed for large enterprises; it should remain simple, comprehensive,
not too demanding in terms of resources and it must be able to guide owner-managers
towards action and improvement (St-Pierre and Delisle, 2006).
The paper is organized as follows. The second section analyses the tools available in
literature for PMS assessment in general and in the context of SMEs in particular. In
the third section the literature is reviewed to identify the main general features of a
“good” PMS; furthermore, the most important characteristics of SMEs along with the
main weaknesses of their current PMSs are reviewed. Finally, on the basis of all the
elements identified, the requirements that a PMS should fulfil to allow a SME to
effectively manage its performance are obtained and a PMS assessment tool is
proposed in Section 4. The paper ends with some conclusions and directions for future
research.

2. Background
Many authors have underlined the importance for all major businesses of evaluating
and modifying performance measures in order to adapt to the rapidly changing and
highly competitive business environment (Eccles, 1991; Kennerley and Neely, 2002). To
this end Dixon et al. (1990) developed the Performance Measurement Questionnaire,
that is a collection of questions that should help managers identify the improvement
IJPPM needs of their organization, determine to which extent the existing performance
59,2 measures support improvements and establish an agenda for performance measures
improvements. Also Medori and Steeple (2000) proposed a framework consisting of a
performance measurement grid and a checklist for auditing existing performance
measures in order to identify the measures no longer relevant or useful for the
company (“false alarms”) and the measures that are not being measured by the
188 company but are important for the company’s success (“gaps”). Another method for the
evaluation and revision of performance measures has been proposed by Tangen (2004).
The method, called “the Performance measurement progression map”, is formed as a
flowchart and consists of nine steps separated into three phases. Phase A focuses on
finding an appropriate and useful set of measures; Phase B is concerned with how each
individual performance measure is designed, while Phase C includes the actual
implementation of the results from the previous two phases.
However a PMS is much more than just a collection of measures, as it includes five
basic elements: people, procedures, data, software, and hardware (Wettstein and
Kueng, 2002). In order to perform a complete review of a PMS it is therefore necessary
to assess not only the effectiveness of the measures but also the effectiveness of the
system as a whole.
The literature provides some examples of tools for PMS assessment. Bititci et al.
(1997) identified “integrity” and “deployment” as key characteristics of an integrated
PMS and developed an audit method based on a reference model called the
“Integrated Performance Measurement System Reference Model” to enable
organizations to verify whether their existing measurement systems display these
two characteristics. They also describe the application of the technique developed in a
SME; however the company cannot be considered a standard SME because it is not
autonomous but part of a larger group and, for this reason, probably influenced by
the parent company in the performance measurement approach. Caplice and Sheffi
(1995), on the basis of literature review and company interviews, developed a set of
six criteria for the evaluation of a PMS as a whole, but just focusing on logistic PMSs.
According to their approach, a “good” system should be comprehensive, causally
oriented, vertically integrated, horizontally integrated, internally comparable and
useful. They also applied successfully their review framework in two large
enterprises. Wettstein and Kueng (2002) on the basis of empirical data and an
analysis of previous maturity models available in information system literature,
developed a PMS maturity model for assessing PMSs in place in companies. They
define the evolution of PMSs along four stages (Ad-hoc, Adolescent, Grown-up,
Mature), on the basis of six dimensions: scope of measurement, data collection,
storage of data, communication of performance results, use of performance measures,
quality of performance measurement processes. Najmi et al. (2005) describe a
structured framework for reviewing business performance and the PMS
simultaneously, in terms of strategic relevance of the measures as well as
efficiency and effectiveness of the PMS itself. The framework employs a range of
approaches and tools already available in literature (e.g. EFQM self-assessment
process, affinity diagram, prioritization grid) in order to feature a high level review
card which identifies three review stages (ongoing, periodic, overall) of the
organizational performance. Another tool for assessing the maturity and effectiveness
of enterprise PMSs has been developed by Van Aken et al. (2005). The tool, named
Improvement System Assessment Tool (ISAT), is not an assessment for PMSs in Performance
general, but for PMSs that are part of an overall system for organizational measurement
improvement. The assessment elements are based on the literature and are grouped
into two categories: improvement processes/outputs and results on key performance systems in SMEs
measures. The scoring approach is adapted from Malcolm Baldrige National Quality
Award (MBNQA, 2009) and the European Foundation for Quality Management
framework (EFQM, 2002), and for each assessment element an assessment template is 189
provided. Also Tangen (2005) proposed a procedure for evaluating and improving an
existing PMS in a company. The procedure consists of three steps: requirement
evaluation; system class determination; revision of the PMS. In practice, on the basis
of a list of requirements, the PMS is classified into a specific system class and then
reviewed in order to improve and progress towards an upper-level class.
All the tools for PMS assessment provided by the literature and described above do
not make explicit reference to the size of the target companies; furthermore, they
appear too complex and resource intensive to be used effectively in a SME context. The
authors have found only one paper (Garengo, 2009) dealing to some extent with PMS
assessment in SMEs. The paper proposes a framework to classify PMSs in SMEs
taking part in Quality Award Programmes and study their evolution. PMS are
classified according two dimensions: PMS characteristics (i.e. how companies are using
measures to manage performance) and PMS scope (i.e. what companies are measuring).
However, the framework does not represent a tool that SMEs can use to assess the
effectiveness of their PMS, rather it is a model intended for theoretical reasoning and
company classifications by external academics.
Therefore there appears to be a gap in literature regarding practical self-assessment
tools that can support SMEs in the process of identifying the main weaknesses of the
PMS they have in use and take corrective measures.

3. Features of a “good” PMS for SMEs

In order to develop a PMS assessment tool for SMEs, first of all it is necessary to
identify which are the characteristics of a PMS that enable the company to effectively
and efficiently measure and manage its performance.
For this reason a review of the main recommendations provided in literature
regarding the characteristics of a well-designed PMS is performed.
However the general requirements identified need to be adapted to the SME context.
Therefore, an analysis of the characteristics of SMEs is carried out, along with an
evaluation of the main weaknesses of their performance measurement current practice.
On the basis of the elements collected, a list of features of a “good” PMS for a SME is
finally obtained.

3.1 General characteristics of a “good” PMS

In literature there is plenty of performance measurement recommendations, which are
pieces of advice related to the discipline of performance measurement.
Recommendations can be grouped into two categories: recommendations concerning
performance measures and recommendations concerning PMS design (Folan and
Browne, 2005).
Several authors have already provided comprehensive overviews of the most
popular recommendations (Neely et al., 1997; Folan and Browne, 2005). In addition, the
IJPPM main frameworks for business performance measurement proposed in literature have
59,2 been compared in order to identify both the characteristics that are recognized to be
essential for a PMS and the most common shortcomings and pitfalls of PMS
implementations that should be avoided (Cocca et al., 2007).
The most important elements collected, that should be considered as “best practices”
for a PMS, are summarized in the following list:
190 (1) Performance measures characteristics:
.
Derived from strategy.
.
Link operations to strategic goals.
.
Simple to understand and use.
.
Clearly defined/explicit purpose.
. Stimulate continuous improvement/right behaviour.
.
Relevant and easy to maintain.
.
Provide fast, accurate feedback.
.
Balanced/multidimensional picture of business.
.
Monitoring past performance.
.
Planning future performance.
.
All stakeholders considered.
.
Promote integration.
.
Defined formula and source of data.
(2) PMS design requirements:
.
Evaluation/audit existing PMS.
. Strategic objectives identification.
.
Top management support/commitment.
.
Key users/employee involvement/support.
.
Facilitator.
.
Maintenance structure.
.
Targets/benchmarks setting.
.
Timescales setting.
.
A responsible for the measure.
.
Performance monitoring process.
.
Alarm signal/corrective actions.
.
Double-loop learning/challenge strategy.
.
Relationships between measures.
.
Linking performance to compensation process.
.
Procedures defined.
.
IT infrastructure support.
3.2 SME characteristics and performance measurement practice Performance
For the purpose of this study the definition of “SME” provided by the European Union measurement
has been adopted:
systems in SMEs
The category of micro, small and medium-sized enterprises (SMEs) is made up of enterprises
which employ fewer than 250 persons and which have an annual turnover not exceeding 50
million euro, and/or an annual balance sheet total not exceeding 43 million euro (EC, 2005).
191
However, although numbers may broadly be used to identify whether a company is a
SME or not, the authors believe that this should be completed by a set of characteristics
which enable a better definition of the term “SME”. In fact, despite the recognised
heterogeneity of SMEs, there appears to be a consensus from researchers in this field
that many SMEs share a number of general characteristics (Hudson et al., 2001).
In order to identify these characteristics a literature review has been performed and
many papers focusing on SMEs in different field of science have been analyzed (see
Cocca and Alberti, 2009). All the characteristics have been grouped into two main
categories: external environment and internal environment. External environment
represents the context in which the organization operates and the factors essentially
outside the control of organizational members. It is divided into two main
subcategories: market and customers. Internal environment includes the factors
which are inside the company or under the managers’ control, like the resources, both
human and financial, and the way they are managed (managerial practice).
From the review it emerged that, with reference to the external environment, SMEs
operate in highly competitive, turbulent and uncertain markets (Garengo et al., 2005).
Usually they do not have control or influence over the market and thus they need to
adopt a reactive approach and adapt to market changes (Hudson, 2001).
Since SMEs rely on a limited customer base, they are usually closer to the customers
and have the possibility to develop more personal relationships with them (Hong and
Jeong, 2006). However this sometimes forces the development of deferential
relationships with their customers and SMEs are often subservient to their larger
counterparts (Hudson, 2001). In fact, SMEs’ demand is made by stronger customers
throughout the supply chain and this implies difficulties in leveraging payments of
debts and consequently in coping with fluctuations in cash flow, causing a lack of
control over the future (Hudson, 2001).
From an internal point of view, all the authors highlight scarcity of resources as one
of the main problems and typical characteristic of SMEs (Singh et al., 2008). The term
“resources” is considered both in terms of personnel, including also managerial time,
and financial stability and security. In addition also skills are limited, not only among
staff (Singh et al., 2008), but also owner-managers often do not have enough managerial
expertise or organizational capabilities and this implies poor strategic business
planning and human resource management (Pansiri and Temtime, 2008).
Even though size represents a weakness in terms of available resources, on the other
side, it favours a flat organizational structure with lack of bureaucracy and this has a
positive impact on flexibility, adaptability and rapidity in responding to the changing
environment (Garengo et al., 2005). For this reason SMEs have usually a high potential
for innovation and the ability to satisfy customers’ emerging and evolving
requirements. Furthermore, a structure with few management layers favours
face-to-face relations, simplifying communication processes and offering to the
IJPPM manager high visibility on the processes and the opportunity to directly influence
59,2 employees (Singh et al., 2008).
Managers very often are also the owners of the company and the control in SMEs
rests primarily with one or a few people with a high level of autonomy (Pansiri and
Temtime, 2008). Organizational success or failure in SMEs is seriously affected by the
managerial competencies of the owner-manager; in fact, decisions are mainly based on
192 the director’s personal skills and intuition rather than on analysis of information. The
owner-manager usually adopts a highly personalized management style, tending to
follow a “react and adapt” philosophy and fire-fighting strategies, focusing on short
term horizons and not engaging actual strategic planning (Hudson et al., 2001).
Improvements are usually incremental and there is a preference to adjust processes
and systems in response to specific identified needs and to learning-by-doing
approaches (Garengo et al., 2005). With reference to performance measurement, SMEs
are still relying mainly on accountancy information and financial measurement
(Carpinetti et al., 2008); focus on technical aspects and production (Hong and Jeong,
2006) usually leads SMEs to a misconception about performance measurement, often
considered as a wasting-time activity (Garengo et al., 2005).
In addition to the review of SME characteristics, in order to deepen the analysis of
the issue of performance measurement in SME, also an empirical survey, involving a
probability sample of Italian SMEs, has been performed by the authors. For an
in-depth examination of this issue the reader is referred to Cocca and Alberti (2008).
The research allowed to identify the main shortcomings of the PMSs currently used by
SMEs, and provided a first overview of the evolutionary path of PMS in SMEs. The
results of the survey showed that the main weaknesses of PMSs in SMEs concern the
scope of measurement and data collection and storage. In fact SMEs seem to suffer
from financially based systems and from the lack of adequate IT infrastructure. Other
difficulties in managing the PMS are related to the communication and use of
performance measures. Furthermore poor quality of the performance measurement
processes has been highlighted. Therefore all of these critical issues should be included
in a PMS assessment tool. Regarding PMS maturity, the survey highlighted that a
company could be very advanced regarding one dimension, while being rather
antiquated regarding another dimension. For this reason the approach of strictly
classifying a PMS according maturity levels categories appears ineffective. Therefore,
an approach that considers the maturity level of each PMS dimension separately seems
more appropriate.

3.3 PMS requirements for SMEs

In order to develop a tool to assess PMSs, it is necessary to define what constitutes a
best practice for a PMS, identifying a list of distinctive features against which a SME
could evaluate the system being used for the measurement and management of its
performance.
In the previous sections a list of PMS general requirements has been obtained from
the literature. However SMEs present some distinctive characteristics that differentiate
them from large enterprises. This implies that also their needs in terms of performance
measurement processes and tools are different from those of larger companies.
Therefore, the PMS general requirements identified have to be tailored to SME needs
by means of an analysis of SME characteristics.
 For this reason, the main characteristics of SMEs and their current PMSs have Performance
been reviewed by the authors and for each element respectively one or more measurement
consequent requirements for the ideal PMS have been logically deducted. For
example, since SMEs operates in highly dynamic and turbulent environment, they systems in SMEs
need to continuously adapt to market changes and these changes should be quickly
reflected in the PMS and the measures monitored. Therefore PMS has to be very
flexible, rapidly changeable and maintainable. Furthermore, since SMEs suffer from 193
lack of resources, the performance measures should be very simple, synthetic and
easily collectable, otherwise the effort needed for measuring would be higher than
the benefit gained. Similarly also the procedures for measures collection should be
well defined and resource effective. Moreover it would be better to use only a few
vital metrics, better if reported in a graphically and visually effective way, in order
to enable the manager to focus only on key performance factors and quickly take
informed decisions. The PMS requirements identified through the deductive process
have then been collated and integrated with the general attributes collected in the
list “best practices” shown previously. Finally a list of the main features that a PMS
should display to be suitable for a SME has been obtained. All the features are
gathered in the following list and catalogued into three categories: performance
measure requirements; characteristics of the PMS as a whole; requirements of the
process of performance measurement and PMS use:
(1) Performance measures:
.
Derived from strategy.
.
Link operations to strategic goals.
.
Simple to understand and use.
.
Clearly defined/explicit purpose.
.
Stimulate continuous improvement/right behaviour.
.
Relevant and easy to maintain.
.
Easy to collect.
.
Provide fast, accurate feedback.
.
Monitoring past performance.
.
Planning future performance.
.
Promote integration.
.
Defined formula and source of data.
(2) PMS as a whole:
.
All stakeholders considered.
.
Flexible, rapidly changeable and maintainable.
.
Balanced (internal/external, financial/non-financial).
.
Synthetic.
.
Easy to implement, use and run.
.
Causal relationships shown.
.
Strategically aligned.
IJPPM .
Graphically and visually effective.
59,2 .
Incrementally improvable.
.
Linked to rewarding system.
.
Integrated with IS.
(3) Performance measurement process:
194 . Periodic evaluation existing PMS.
.
Strategy development.
.
Long- and short-term planning.
.
Information sharing and communication.
.
Manager’s commitment.
.
Employee involvement/support.
.
Facilitator.
.
Maintenance procedure.
.
Systematic targets setting.
.
Roles assignment and responsibilities sharing.
.
Performance revision procedure.
.
Linking performance to compensation process.
.
Procedures clearly defined.
.
IT infrastructure support.

4. SME PMS self-assessment tool

According to the European Foundation for Quality Management, a self-assessment
audit is defined as “a comprehensive and systematic review of an organization’s
activities and results referenced against a model of business excellence. It allows the
organization to discern clearly its strengths and areas in which improvements can be
made and culminates in planned improvement actions which can be monitored for
progress” (van der Wiele et al., 1995). Self-assessment is being used increasingly by
industry as a tool to help identify “best practice” and areas for improvement. Research
studies show that the process of measurement against a reference, seeking for
continuous improvement, is particularly effective in SMEs and allows them to improve
their performance (Cassell et al., 2001; St-Pierre and Delisle, 2006).
In literature different approaches to assess an attribute or a characteristic of a
process or a system have been proposed (Tables I-III).
The simplest means is through binary yes-no response (Table I, scale 1). However
this approach is highly subjective since responses are open to an extremely wide

Scale 1: Binary yes/no scale

Do you communicate performance results to your

Table I. stakeholders? Yes No
Approaches to
self-assessment processes Source: Adapted from Moultrie et al. (2007)
 Performance
1 2 3 4 5 6 7
measurement
Scale 2: Likert-type scale systems in SMEs
We always Strongly Disagree Agree Strongly agree
communicate disagree
performance
results to our 195
stakeholders
Scale 3: Modified Likert-type scale
How do you Performance Performance
communicate results rarely results
performance communicated systematically
results to your to stakeholders communicated
stakeholders? to all
stakeholders Table II.
Approaches to
Source: Adapted from Moultrie et al. (2007) self-assessment processes

Performance results communication

Level 1 Level 2 Level 3 Level 4

Scale 4: Maturity scale with multiple descriptions

Performance results Performance results Performance results Performance results
rarely communicated to sometimes often communicated to always communicated
stakeholders communicated to some most relevant to all stakeholders
stakeholders stakeholders
Scale 5 Maturity grid with extended descriptions
Performance results Performance results Performance results Performance results
rarely communicated to sometimes often communicated to always communicated
stakeholders communicated to some most relevant to all stakeholders
stakeholders stakeholders
Only upper Only upper and middle Communication Results transmitted
management involved management involved structures are electronically to all
established stakeholders (push)
Dissemination on an Dissemination on a Most dissemination via Results can also be
ad-hoc basis periodical basis push mechanism accessed (pull) Table III.
Approaches to
Source: Adapted from Moultrie et al. (2007) self-assessment processes

degree of interpretation. Furthermore it does not provide advice regarding good

practice and it offers little granularity when scoring.
An alternative consists in the use of Likert-type scale, as shown in Table II scale 2.
The issue is expressed in the form of a positive or negative statement and the assessor
has to score the extent to which he or she agrees or disagrees. In this case granularity is
higher; however it still provides little insight about best practice.
Another type of means is to adapt the Likert-style questionnaire to provide phrases
describing performance at each end of the scale (Table II, scale 3). This approach
provides greater insight into the potential extremes of performance. However, since the
transition among different performance levels is not necessarily linear, little insight is
IJPPM offered about the mean of intermediate points and possible actions to take in order to
59,2 achieve higher performance levels.
This approach can be extended by adding descriptions at a number of points along
the scale, adopting a process maturity approach (Table III, scale 4). These intermediary
descriptions provide insight into how to progress from a level to the next one and thus
help to improve objectivity when scoring.
196 Finally the process maturity approach has been further extended through richer
descriptions at each point along the scale, thus developing the process maturity grid
(Table III, scale 5). Process maturity is defined as the degree to which a process or
activity is institutionalized and effective (Dooley et al., 2001). A maturity grid aims to
codify what might be regarded as good – and bad – practice along with a number of
intermediate or transitional stages. It codifies best practice understanding in an
accessible way to enable opportunities for improvement to be determined (Moultrie
et al., 2007). Furthermore it provides information that can be used in developing action
plans to improve performance (Chiesa et al., 1996).
According to Moultrie et al. (2007), in SMEs, where managers notoriously have little
time to consider process improvement, maturity grids offer a simple and user-friendly
solution and are thus the most appropriate approach to use for an assessment tool.
Also Garengo et al. (2005) identify maturity grids as the most suitable type of
assessment tool for SMEs. In their paper they classify maturity grids in the category of
“synthetic benchmarking on managerial practices”, which includes all the models
based on scorecards such as the one proposed by Voss et al. (1994). They proposed

Level 1 Level 2 Level 3

Measurement procedures are not Measurement procedures are Measurement procedures are
defined. Form and frequency of roughly documented for some documented and standardized
data collection are not clearly main metrics. Frequency of for all metrics. Frequency of
defined. Source of data are not measurement and source of data measurement and source of data
Table IV. clearly defined. Success depends are defined for some main are defined for all metrics. A
Scorecard for the on individual effort and ability metrics. Some templates are standardized template for
performance available for collecting the collecting the measures is
measurement process measures. Successful execution available. Successful execution
requirement “Procedures of some procedures can be of all procedures can be repeated
clearly defined” repeated also by new operators

Level 1 Level 2 Level 3

All performance-relevant data Some performance data are Collection of most performance
are collected manually. Most collected manually and some by data is automated by IT
performance data are stored in operational IT systems. systems. All performance data
Table V. paper format. Software for data Performance relevant data are are stored in a central database
Scorecard for the analysis or performance stored in local PCs. Data are integrated with the IS. Specific
performance reporting is not available/used analyzed and performance is software for data analysis and
measurement process reported by means of performance reporting is
requirement “IT spreadsheets or simple office available and used (e.g. Business
infrastructure support” software Intelligence software)
maturity grids as appropriate for SMEs because they can be easily understood and Performance
used, implemented in a short period of time and structured in such a way that as to not measurement
require the use of external consultants. This type of tool can be used to meet the needs
of SMEs in that it makes it possible to highlight excellent practices in a simple way and systems in SMEs
by using limited resources, thus stimulating a learning process. Scorecards enable the
assessor to see the strengths and weaknesses of the object of study and show the steps
to be taken to carry out improvement to fill the gap between the practice in use and the 197
best one (Garengo et al., 2005).
On the basis of the previous remarks, maturity grids seem to be the most suitable
approach also to develop an effective tool for PMS assessment in SMEs. Therefore, a
PMS self-assessment tool based on maturity grids has been developed. The tool
incorporates all the elements of performance measurement best practice identified in
the previous sections and some elements of other PMS maturity models available in
literature (Wettstein and Kueng, 2002; Cocca and Alberti, 2008; Garengo, 2009). The
structure of the tool proposed is similar to the one presented by Voss et al. (1994) for
assessing innovative processes and then used by many authors (e.g. Moultrie et al.,
2007; Garengo et al., 2005) to develop tools to be used in the context of SMEs. The PMS
assessment tool consists of a series of scorecards, one for each PMS requirement
identified and listed in Table II. Each scorecard contains three areas which describe
three stages of the development of the practice under consideration following an
evolutionary path: level 1 is the elementary practice while level 3 corresponds to a good
practice. For length reasons it has not been possible to include the whole tool in the
paper. Tables IV and V show, as an example, the scorecards developed for the two
performance measurement process requirements concerning the clear definition of
procedures and the support provided by information technologies.

5. Conclusion
This paper describes the development of a self-assessment tool that SMEs can use to
evaluate and improve their PMS. The main assumption is that SMEs require tools
specifically designed and tailored on their characteristics and needs. For this reason,
SME characteristics and current performance measurement practice have been
reviewed and analyzed to derive the main characteristics of an effective PMS for a
SME. Then the different means proposed in literature for self-assessment have been
considered and maturity grids have been identified as the most suitable approach for a
SME context. Finally a self-assessment tool based on scorecards describing three
maturity stages for each PMS characteristic has been developed.
Since the proposed tool codifies what can be regarded as best practice and makes it
accessible by companies in a simple way, it seems to be a valid instrument that could
support SMEs in the process of continuous improvement of their performance
measurement practice.
The application of the tool in real settings will offer the possibility of verifying its
applicability and effectiveness as long as provide an opportunity of refinement and
improvement.

References
Bititci, U.S., Carrie, A.S. and McDevitt, L. (1997), “Integrated performance measurement systems:
an audit and development guide”, The TQM Magazine, Vol. 9 No. 1, pp. 46-53.
IJPPM Caplice, C. and Sheffi, Y. (1995), “A review and evaluation of lofistics performance measurement
systems”, The International Journal of Logistics Management, Vol. 6 No. 1, pp. 61-74.
59,2
Carpinetti, L.C., Galdámez, E.V.C. and Gerolamo, M.C. (2008), “A measurement system for
managing performance of industrial clusters. A conceptual model and research cases”,
International Journal of Productivity and Performance Management, Vol. 57 No. 5,
pp. 405-19.
198 Chiesa, V., Coughlan, P. and Voss, C. (1996), “Development of a technical innovation audit”,
Journal of Product Innovation Management, Vol. 13 No. 2, pp. 105-36.
Cocca, P. and Alberti, M. (2008), “PMS maturity level and driving forces: an empirical
investigation in Italian SMEs”, paper presented at the 15th International Annual EurOMA
Conference Tradition and Innovation in Operations Management, 15-18 June, Groningen,
The Netherlands.
Cocca, P. and Alberti, M. (2009), “SMEs’ three-step pyramid: a new performance measurement
framework for SMEs”, paper presented at the 16th International Annual EurOMA
Conference “Implementation – Realizing Operations Management Knowledge”, Göteborg,
14-17 June.
Cocca, P., Tomasoni, G. and Rossi, D. (2007), “Performance measurement frameworks:
a conceptual model”, in Lalwani, C., Mangan, J., Butcher, T. and Coronado Mondragon, A.
(Eds), Logistics Research Network 2007 Conference Proceedings, 5-7 September, Hull, UK,
The Chartered Institute of Logistics and Transport (UK), Corby, pp. 410-5.
Dixon, J.R., Nanni, A.J. and Vollman, T.E. (1990), The New Performance Challenge – Measuring
Operations for World-class Competition, Dow Jones-Irwin,, Homewood, IL.
Dooley, K., Subra, A. and Anderson, J. (2001), “Maturity and its impact on new product
development project performance”, Research in Engineering Design, Vol. 13, pp. 23-9.
EC (2005), “SME definition 2005”, available at: http://ec.europa.eu/enterprise/enterprise_policy/
sme_definition/index_en.htm (accessed 23 July 2009).
Eccles, R. (1991), “The performance measurement manifesto”, Harvard Business Review,
January-February, pp. 131-7.
European Foundation for Quality Management (EFQM) (2002), The EFQM Excellence Model,
EFQM, Brussels.
Folan, P. and Browne, J. (2005), “A review of performance measurement: towards performance
management”, Computers in Industry, Vol. 56, pp. 663-80.
Garengo, P. (2009), “A performance measurement system for SMEs taking part in quality award
programmes”, Total Quality Management, Vol. 20 No. 1, pp. 91-105.
Garengo, P., Biazzo, S. and Bernardi, G. (2007), “Design and implementing a performance
measurement system in SMEs: a bottom up approach”, paper presented at the 14th
International Annual EurOMA Conference “Managing Operations in Expanding Europe”,
17-20 June, Ankara.
Garengo, P., Biazzo, S. and Bititci, U. (2005), “Performance measurement systems in SMEs: a
review for a research agenda”, International Journal of Management Reviews, Vol. 7 No. 1,
pp. 25-47.
Hong, P. and Jeong, J. (2006), “Supply chain management practices of SMEs: from a business
growth perspective”, Journal of Enterprise Information Management, Vol. 19 No. 3,
pp. 292-302.
Hudson, M. (2001), “Introducing integrated performance measurement into small and medium
sized enterprises”, PhD thesis, University of Plymouth, Plymouth.
Hudson, M., Smart, A. and Bourne, M. (2001), “Theory and practice in SME performance Performance
measurement system”, International Journal of Operations & Production Management,
Vol. 21 No. 8, pp. 1096-115. measurement
Kaplan, R. and Norton, D. (1992), “The balanced scorecard: the measures that drive systems in SMEs
performance”, Harvard Business Review, January-February, pp. 71-9.
Kennerley, M. and Neely, A. (2002), “A framework of the factors affecting the evolution of
performance measurement systems”, International Journal of Operations & Production 199
Management, Vol. 22 No. 11, pp. 1222-45.
Lynch, R.L. and Cross, K.F. (1991), Measure Up – The Essential Guide to Measuring Business
Performance, Mandarin, London.
Malcolm Baldrige National Quality Award (MBNQA) (2009), Criteria for Performance Excellence,
United States Department of Commerce, Technology Administration, National Institute of
Standards and Technology, Gaithersburg, MD.
Medori, D. and Steeple, D. (2000), “A framework for auditing and enhancing performance
measurement systems”, International Journal of Operations & Production Management,
Vol. 20 No. 5, pp. 520-53.
Moultrie, J., Clarkson, P.J. and Probert, D. (2007), “Development of a design audit tool for SMEs”,
Journal of Product Innovation Management, Vol. 24, pp. 335-68.
Najmi, M., Rigas, J. and Fan, I.S. (2005), “A framework to review performance measurement
systems”, Business Process Management Journal, Vol. 11 No. 2, pp. 109-22.
Neely, A., Adams, C. and Kennerley, M. (2002), The Performance Prism: The Scorecard for
Measuring and Managing Business Success, FT Prentice Hall, London.
Neely, A., Richards, H., Mills, J., Platts, K. and Bourne, M. (1997), “Designing performance
measures: a structured approach”, International Journal of Operations & Production
Management, Vol. 17 No. 11, pp. 1131-52.
Pansiri, J. and Temtime, Z.T. (2008), “Assessing managerial skills in SMEs for capacity
building”, Journal of Management Development, Vol. 27 No. 2, pp. 251-60.
Singh, R.K., Garg, S.K. and Deshmukh, S.G. (2008), “Strategy development by SMEs for
competitiveness: a review”, Benchmarking: An International Journal, Vol. 15 No. 5,
pp. 525-47.
Sousa, S., Aspinwall, E. and Rodrigues, A. (2006), “Performance measurement in English small
and medium enterprises: survey results”, Benchmarking: An International Journal, Vol. 13
Nos 1/2, pp. 120-34.
St-Pierre, J. and Delisle, S. (2006), “An expert diagnosis system for the benchmarking of SMEs’
performance”, Benchmarking: An International Journal, Vol. 13 Nos 1/2, pp. 106-19.
Tangen, S. (2004), Evaluation and Revision of Performance Measurement Systems, Doctoral
thesis, Woxén Centrum, Department of Production Engineering Royal Institute of
Technology Stockholm.
Tangen, S. (2005), “Analysing the requirements of performance measurement systems”,
Measuring Business Excellence, Vol. 9 No. 4, pp. 46-54.
Van Aken, E.M., Letens, G., Coleman, G.D., Farris, J. and Van Goubergen, D. (2005), “Assessing
maturity and effectiveness of enterprise performance measurement systems”,
International Journal of Productivity and Performance Management, Vol. 54 Nos 5/6,
pp. 400-18.
van der Wiele, T., Dale, B., Williams, R., Kolb, F., Luzon, D.M., Schmidt, A. and Wallace, M.
(1995), “State-of-the-art study on self-assessment”, TQM Magazine, Vol. 7 No. 4, pp. 13-17.
IJPPM Voss, C.A., Chiesa, V. and Coughlan, P. (1994), “Developing and testing benchmarking and
self-assessment frameworks in manufacturing”, International Journal of Operations &
59,2 Production Management, Vol. 14 No. 3, pp. 83-100.
Wettstein, T. and Kueng, P. (2002) “A maturity model for performance measurement systems” in
Brebbia, C.A. and Pascolo, P. (Eds), Management Information Systems 2002: GIS and
Remote Sensing, WIT Press, Southampton.
200
About the authors
Paola Cocca is a post-doctoral research fellow at the Department of Mechanical and Industrial
Engineering, Brescia University. She holds a first-class degree in Industrial Engineering and a
PhD in “Design and management of production-logistic integrated systems” from Brescia
University. For some years she has been carrying out research and teaching activities at Brescia
University and has also developed international collaborations. Her research focuses broadly on
operations management, in particular on performance measurement systems, risk management,
waste management and safety and health at work. Paola Cocca is the corresponding author and
can be contacted at: [email protected]
Marco Alberti is Full Professor of Industrial Plants at the Department of Mechanical and
Industrial Engineering, Brescia University. He obtained a first-class degree in Chemical
Engineering from the Engineering Faculty of the Politecnico di Milano. He has over 30 years of
research/teaching and industrial experience. He has guided a number of students for their
undergraduate projects, Master’s dissertations and PhD degrees. His major research interests
concern the design and management of industrial plants, with a wide range of topics that cover
industrial plant and process safety and ergonomics, as well as environmental compatibility
management and performance measrement.

To purchase reprints of this article please e-mail: [email protected]

Or visit our web site for further details: www.emeraldinsight.com/reprints