KM Measurement PDF
KM Measurement PDF
Rifat O. Shannak
Chairman of MIS Department, Faculty of Business, University of Jordan
Amman, Jordan
E-mail: [email protected]
Tel: +962777427177; Fax: +96265829977
Abstract
This is a theoretical paper that has the purpose of reviewing previous experiences
based on surveying the extant literature with the aim of delving into the subject of
Knowledge Management (KM) Performance. It is an attempt to try to identify performance
indicators for measuring KM systems. The paper presents important works related to direct
KM performance and another way of evaluating KM performance based on the use of KM
strategy. It discusses the different experiences of the large international companies in
assessing KM performance and proposes a categorization matrix that classifies the
performance indicators for potential use in KM performance measurements. Based on the
revealed finding, the paper proceeds to present suggestions for future research in the area.
1. Introduction
Knowledge Management (KM) has been the subject of much discussion over the past decade.
Organizations are told that they will not survive in the modern Knowledge era unless they have a
strategy for managing and leveraging value from their intellectual assets, and many KM lifecycles and
strategies have been proposed. (Arthur, 1996)
As the term "Knowledge Management" has been applied to a very broad spectrum of activities
designed to manage, exchange and create or enhance intellectual assets within an organization, there is
no widespread agreement on what KM actually is. IT applications that are termed "KM applications"
that range from the development of highly codified help desk systems to the provision of video
conferencing to facilitate the exchange of ideas between people. (Haggie and Kingston, 2003)
“As it can be thought that KM is the deliberate design of processes, tools, structures, etc. with
the intent to increase, renew, share, or improve the use of knowledge represented in any of the three
elements [Structural, Human and Social] of intellectual capital.” Seemann, et.al., 1999)
This justifies the existence of KM projects. The challenge lies ability to assess a person’s
individual knowledge and make sure it is fully taken advantage of, and that an environment is created
to encourage people to take part in the sharing process.
Therefore, and in order for organizations to best utilize their KM systems, they should be able
to measure their performance; a subject that also has been both discussed and emphasized (Strassmann,
1999). This is easy if one knows the current performance level, and by checking if it is increasing or
not, then it becomes possible to track the changes. As also knowledge is comprised of the soft (human
Measuring Knowledge Management Performance 243
and culture) and the hard (technology) aspects, it is the integration between both that is critical to
optimize the effects of KM.
As of today, there are a few methods for such measurements; Balanced Business Scorecard,
EFQM (European Foundation for Quality Management) and the Skandia Navigator. These methods
give some attention to knowledge related items such as innovation, patents and intellectual capital.
2.2. Technological, Human and Financial resources based Measurement of the Performance of a
KM System
As mentioned in the introduction, if one knows the current level of performance, then it is possible to
control it. It can be simply stated that KM systems and their effects are justified to top management by
tracking the progress on these projects.
One view states that in order to ensure an overall organizational performance, the organization
needs to manage and measure its technological, human and financial resources. Measuring this
performance validates if the effort was worth the while or not. (Tseng and Lee, 2009)
This point of view states that KM projects have both tangible and intangible results; the first
lends itself to quantifiable measurement, while the latter results in valuable outcomes that are not
readily quantifiable which makes them really difficult to measure. (Arthur, 1996)
The traditional methods of measuring the performance of a KM system have been financially
based. Measurement regarding competence, employee satisfaction and technological development, to
mention but a few, do not seem to be used as frequently as measurements of financial areas such as
cost effectiveness, product efficiency and distribution of sales. (Bontis et al., 2003)
It has been argued that this result does not stem from the organization's lack of interest, but
rather the fact that organizations have not yet been able to develop metrics that adequately fit
qualitative areas. As often measurement in such areas involves limited information and impaired
judgement. It is probable that the organizations attempt to measure these areas, but because of
insufficient measurement systems, it appears as if they are focused on the financial areas.
The need therefore is dire for having what is called performance indicators, which may not
necessarily show an improving overall organizational performance, but they do show whether the
knowledge activity is increasing or not. These determine the status of the project and whether it has
established a level of satisfaction or it needs for some actions for improvement. (Robertson, 2003)
A performance indicator is a variable, parameter, measure, statistical measure, a proxy for a
measure, and a sub-index among others. Generally, there are four ways to express the performance
indicator:
1. An indicator of how many times an event takes place,
2. A ratio, i.e. how many times an event takes place compared to how many times it could
have taken place in the given time period,
3. A percentage,
4. A Boolean variable, i.e. did the indicator generate what it was supposed to generate or not.
There are some features which should exist in the performance indicators; for example they
should have relevance for project goals, and provisional, since there may appear a need to eventually
change the performance indicator. The indicator also needs to be understandable, valid, sufficiently
flexible, and in line with the organization and its business goals, as well as the purpose it was
developed for.
In addition to this, the indicators may also be qualitative or quantitative. Qualitative indicators
indicate improvements by measuring attitudes, beliefs and culture. Quantitative indicators, on the other
hand, indicate participation, for instance the number of communities or the number of people using a
database.
the measurement easier. Linking this knowledge with the items drafted in the strategy is of high
essence in measuring the performance of the KM system. (McAdam, R and McCreedy, S, 1999;
Lopez, 2000)
Practitioners such as Nonaka and Takeuchi classify “knowledge” based on a combination of
knowledge accessibility (i.e. where is the knowledge stored or located and in what form) and
knowledge transformation (i.e. the flow of knowledge from one place to another and from one form to
another) (Nonaka, 1991; Nonaka and Takeuchi, 1995). They also have different classifications of
knowledge (explicit vs. tacit, individual vs. collective, along with knowledge transformation processes
i.e.: socialization, externalization, combination and internalization).
Wiig (1993) proposed his Knowledge Management model with a principle which states
that,knowledge can be useful if it is well oraganized.There are some useful dimensions to be noted in
Wiigs KM model.They are:
• Completeness
• Connectedness
• Congruency
• Perspective and purpose
Boisot (1998) proposes 2 key points they are:
1. The more easily data is converted to information the more easily it is diffused.
2. The less the data is structured requires a shared context for its diffusion,the more
diffusable it becomes.
Boisot's I-Space model is visualised as a 3 dimensional cube with following dimensions:
1. Codified-Uncodified
2. Abstract-Concrete
3. Diffused-Undiffused
Boisot (1998) introduced an extra dimension (abstraction, in the sense that knowledge can
become generalised to different situations).
What seems clear from both Boisot's model and that of Nonaka & Takeuchi is that the process
of growing and developing knowledge assets within organizations is always changing. Organizations
are living organisms that must constantly adapt to their environment. This means that the KM strategy
identified as appropriate at one moment in time will need to change as knowledge moves through the
organizational learning cycle to a new phase. The rate at which this cycle operates will vary from one
sector to another, so that in some rapidly evolving sectors new knowledge is being created and applied
in rapid succession, while in some more established sectors, the cycle time of innovation is much
slower.
On another token, Wiig (1993) and the APQC (American Productivity and Quality Center)
(Vestal, 2002) identified six emerging KM strategies in a study of organizations considered to reflect
the different natures and strengths of the organizations involved:
1. Knowledge Strategy as Business Strategy; A comprehensive, enterprise-wide approach to
KM, where frequently knowledge is seen as the product.
2. Intellectual Asset Management Strategy; Focuses on assets already within the company that
can be exploited more fully or enhanced.
3. Personal Knowledge Asset Responsibility Strategy; Encourages and supports individual
employees to develop their skills and knowledge as well as to share their knowledge with
each other.
4. Knowledge Creation Strategy; Emphasizes the innovation and creation of new knowledge
through R&D. Adopted by market leaders who shape the future direction of their sector.
5. Knowledge Transfer Strategy; Transfers knowledge and best practices in order to improve
operational quality and efficiency.
6. Customer-Focused Knowledge Strategy; Aims to understand customers and their needs and
so provide them with exactly what they want.
246 Rifat O. Shannak
Treacy and Wiersema (1994) proposed three generic "value disciplines" in their model, as a
way to focus an organization's activities. Successful organizations concentrate their efforts on a
particular area and excel at it, rather than trying to be all things to all people and failing to excel at
anything.
A. Operational Excellence: Superb operations and execution often by providing a reasonable
quality at a very law price. The focus is on efficiency, streamlining operations, supply chain
management, no-frills, volume counts, most large international corporations are working out of
this discipline.
B. Product Leadership: Very strong innovation and brand marketing, operating in dynamic
markets. The focus is on development, innovation, design, time-to-market, high margins in a
short timeframe.
C. Customer Intimacy: Excel in customer attention and customer service. Tailor their products and
services to individual or almost individual customers. Focus on customer relationship
management, deliver products and services on time and above customer expectations, lifetime
value concepts, reliability, being close to the customer.
These value disciplines reflect the fact that 'value' is determined as a trade-off between
convenience, quality and price.
Some organizations will concentrate on their relationship with their customers (to increase
customer satisfaction and retention by better understanding the customer's needs and preferences).
Other organizations will focus on their products (constantly developing new ideas and getting them to
market quickly). The third group of organizations focus primarily on themselves and their internal
processes (sharing best practices between different units, reducing costs and improving efficiency).
3.1. Ericsson
The purpose of the KM system was to take advantage of knowledge acquired in one market to the other
markets. The idea is to find supporting structures and to be able to repeat successful solutions in more
than one market. The following are the performance indicators identified:
1. Number of best practices identified.
2. Number of contributions.
3. Number of contributions in the re-use database related to the total number of projects
delivered in the market unit.
4. Number of contributions proven to have led to new/repetitive business.
5. Number of contributions rated as re-usable.
6. Number of individuals in the community and each individual's activity.
1. Active involvement.
2. Number of participating employees.
3. Number of postings/contributions.
4. Number of downloads.
5. Number of calls to support function.
6. Support ratings.
7. Unique log-ins.
3.3. KPMG
The purpose of the KM system is to take advantage of the experience within the organization, both in
Sweden and globally. The overriding objectives are to maximize value creation and realization for the
organization and for their clients, by making universally and instantly available best practices,
experiences, insights and connections to the right people.
1. Adoption curve to see knowledge culture progression.
2. Generated business.
3. Individual contributions to the further development of the organization and its employees.
4. Statistics of awareness.
5. Number of contacts gained.
6. Attitude of knowledge sharing.
7. Efficiency & visibility on the market.
8. Employee satisfaction.
9. Re-use of information and/or experience.
3.4. Schlumberger
The purpose of the KM system is to capture, manage and share knowledge within the large and
geographically spread organization. The aim of doing so is to become more productive and efficient.
Schlumberger has identified nine activities to measure extent of use and the level of effectiveness of
these activities which are listed in table 1.
3.5. Siemens
The purpose of the KM system was for the company to focus on its most valuable assets: its knowledge
base. The organization needed something to help overcome geography. As it is truly global, the
employees must share their knowledge by other than (or in addition to) informal face-to-face
communication. The following are the performance indicators identified:
1. Customer success.
2. Employee satisfaction.
3. Indicators for incentives.
4. Innovation index.
5. Number of entries in a database.
6. Number of knowledge-shares a year.
7. Number of orders.
8. Quality of the information.
9. Re-use of knowledge.
10. Usefulness of the knowledge.
3.6. Xerox
The main purpose of the KM system is to maintain a competitive advantage by using the collected
knowledge of all employees, archives of patents and processes as well as all documents stored in
various formats in all locations. The focus of the knowledge management strategy lies in creating a
knowledge-sharing culture, which will lead to accelerated learning and innovation. Managers at Xerox
argued that it is important to make the expertise available to everybody which makes the solution to
complex problems faster. The following are the performance indicators identified:
1. Access of knowledge.
2. Arrangement of knowledge.
3. Number of calls to administration.
4. Support from management.
5. Usability.
Performance Indicator
Arrangement and
application of
access to knowledge Accesses classification of attitude
knowledge
knowledge
if knowledge is
availability of to what extent is the
arranged in such a way
knowledge in number of site accesses knowledge applied of knowledge sharing
that it facilitates finding
information systems throughout organization
what is sought for
awareness of Channels enabling
baseline for maturity calls to helpdesk community
evaluation creative outlets
to what extent the
awareness of being information systems
number of calls to
evaluated may comparing results to a enable channels for
helpdesk, preferably number of members
contribute to increased previously set baseline creative outlets, for
decreasing
participation instance flexible forums
enabling attachments
readability usability re-use re-use contributions
number of re-used
contrib. related to total number of contributions
rated by other users rated by other users rated by other users
number of projects in systems/communities
delivered
best practices documents Individual experience success stories
co-operation with
culture Customer satisfaction Customer success customer time increase
external experts
the amount of
adoption curve to see increased success with
cooperation after having to what extent the are more time to spend with
the progression of the customers compared to
managed knowledge in customer satisfied customers
knowledge culture previously
organization
distribution of efficiency & visibility
downloads effect of knowledge Employee retention
knowledge on the market
any effects of
knowledge within the number of employees
Facilitated distribution
organization, for staying in the
of knowledge within number of downloads increase or decrease
instance increased organization compared
organization
visibility or increased to previously
customer value
Employee satisfaction Expertise available Expert participation Innovation index Job performance
number of employees
the degree of expertise an index showing increased job
being satisfied with number of participating
available in the increased innovation in performance due to
their work situation experts within a field
knowledge system organization managed knowledge
compared to previously
Knowledge activity
job satisfaction Knowledge shares leverage effect Motivation incentives
report
increased satisfaction stating an increased or number of knowledge if the project contributes
based on number of
with job situation due to decreased knowledge shares per measurement to small efforts giving
knowledge-shares a year
facilitated routines activity interval large effects
networking with
motivation incentives Motivation incentives orders participation
external partners
number of number of users
based on usefulness of increased possibilities
based on knowledge orders,preferably an participating in
the knowledge for networking with
activity report increase due to managed knowledge sharing
contribution external partners
knowledge activities
Profitability quality of knowledge query guidance relationships responses
new business gained, for
instance stemming from
usefulness of the number of relationships
contributions in
knowledge found, for sufficient instructions in established due to number of responses to
knowledge base or
instance possibilities of systems knowledge systems & a request
contacts generated or by
re-use networking
interviewing sales
people
response time return on investments re-use Revenue gained/saved self assessment
decrease of response revenue gained due to estimate of how much
times due to facilitation savings from of knowledge managing organization's the individual has
of finding information knowledge gained from the
250 Rifat O. Shannak
managed knowledge
within the organization
participating in
communities of
practice, ie estimated
savings * percent
attributable to the
community * degree of
certainty
support from Support incidents speed of knowledge
sharing index support ratings
management saved transfer
index showing an a more efficient
increased or decreased degree of support from knowledge transfer
number of support decreasing need for
amount of knowledge management regarding within the organization
function ratings support function
sharing within the managed knowledge due to managed
organization knowledge
time, money time to meet customer
tickets usability usefulness
&personnel time saved needs
number of tickets increased amount of
degree of user to what extent the
submitted, preferably a savings due to managed time to spend making
friendliness of systems systems respond to user
decrease due to knowledge sure customers are
and search engine needs
knowledge systems satisfied with solution
usefulness of value added per
user acceptance user satisfaction work methods
knowledge employee
amount of users amount of user increased value due to
how useful a knowledge
accepting knowledge satisfaction with being able to take part more efficient ways of
is to other individuals
activities as part of their systems, routines, and of collective knowledge working
within organization
daily work knowledge available of organization
6. Conclusion
It is desirable to find performance measurements for all areas in order to see the whole picture. When
measuring performance of a KM system, a number of performance indicators may apply to the system
developed, which gives rise to a need to choose which measures that are the most appropriate for the
project. Even though there may be a wide range of measures, it may not always be cost-effective and
efficient to use all identified in the performance indicator matrix.
From here stems the importance of categorizing the indicators in the above matrix in order to
simplify things. Different approaches to measuring the performance of a KM system agree on
Measuring Knowledge Management Performance 251
classifying the effect of such on three main domains namely; process, human and IT. Table 3
summarizes the performance indicators as classified into a categorization matrix with the three primary
categories or dimensions.
References
[1] Arthur, W. B. (1996) Increasing returns and the new world of business. Harvard Business
Review, July-August, pp. 100-109.
[2] Boisot, M (1998). Knowledge assets, Oxford:Oxford University Press.
[3] Bontis, N, Dragonetti, N, Jacobsen, J and Roos, G. (2003). “THE KNOWLEDGE TOOLBOX:
A Review of the Tools Available To Measure and Manage Intangible Resources”, European
Management Journal, Vol. 17, No. 4, pp. 1- 23.
[4] Davenport, T, De Long, D and Beer, M. (1999). “Successful Knowledge Management
Projects”, in The Knowledge Management Yearbook 1999-2000 (Ed. Cortada J and Woods, J),
pp. 89 – 107.
[5] Haggie, K and Kingston, J (2003). "Choosing Your Knowledge Management Strategy",
Journal of Knowledge Management Practice, 4 (June, 2003),
http://www.tlainc.com/jkmpv4.htm checked on (7/8/2009).
[6] Lopez, Kimberly and Project Team. (2000). American Productivity & Quality Center,
“Measurement for Knowledge Management”, Consortium Learning Forum: Best-Practice
Report.
[7] McAdam, R and McCreedy, S (1999). “A critical review of Knowledge Management models”,
The Learning Organization, Vol. 6, Issue 3, pp. 91 – 101.
[8] Nonaka, I. (1991). “The Knowledge Creating Company”, Harvard Business Review,
November-December, 96-104.
[9] Nonaka I. and H. Takeuchi. (1995). The Knowledge-Creating Company: How Japanese
Companies Create the Dynamics of Innovation, Oxford University Press.
[10] Robertson, J (2003). "Metrics for knowledge management and content management",February-
2003, at http://www.steptwo.com.au/papers/kmc_metrics/index.html checked on (7/8/2009).
[11] Seemann P., D. De Long, S. Stucky and E. Guthrie, 1999, Building Intangible Assets: A
Strategic Framework for Investing in Intellectual Capital, Second International Conference on
the Practical Applications of Knowledge Management (PAKeM99), 21-23 April.
[12] Strassmann, Paul. (1999). “Measuring and Managing Knowledge Capital”, Knowledge
Executive Report, This article is a preview of Paul Strassmann's book Knowledge Capital
(1999).
[13] Treacy, M and Wiersema, F. (1994). The Discipline of Market Leaders, Addison-Wesley
Measuring Knowledge Management Performance 253
[14] Tseng, Ya-Fen and Lee, Tzai-Zang. (2009). “Comparing appropriate decision support of human
resource practices on organizational performance with DEA/AHP model”, Expert Systems
with Applications, Volume 36,Issue 3, pp. 6548-6558.
[15] Vestal, W (2002). “Measuring Knowledge Management”, APQC (American Productivity &
Quality Center), USA.
[16] Wiig, K (1993). Knowledge management foundations. Thinking about thinking: how people
and oraganisations create and use knowledge .Arlington TX, Schema press.