2016 49th Hawaii International Conference on System Sciences

Digital Knowledge Objects and Digital Knowledge Object Clusters:

Unit Holdings in a Learning Health System Knowledge Repository

Allen J. Flynn Wei Shi Rocky Fischer Charles P. Friedman

Medical School Cancer Center Cancer Center Medical School
School of Information University of Michigan University of Michigan School of Information
University of Michigan [email protected] [email protected] School of Public Health
[email protected] University of Michigan
[email protected]

Abstract Digital Knowledge Objects are computable objects

comprised of (a) knowledge payload “cores”, (b)
A high-functioning Learning Health System internal interface “layers” (c) descriptive metadata
requires extraordinary knowledge management and a “wrappers”, and (d) instances of semantic links to other
profound capacity to formulate health advice. To meet digital knowledge objects. As such, DKOs form the
these requirements, existing digital libraries of static nodes in multimodal, multiplex, hybrid semantic
narrative documents will have to give way to networks.2 Such networks are comprised of standalone
repositories of biomedical knowledge represented in DKOs and also of DKO clusters. Standalone DKOs are
more transactional forms. This paper introduces units of discrete knowledge. DKO clusters are units of
Digital Knowledge Objects (DKOs) and DKO clusters. interconnected knowledge.2 Here the term “unit
These two “unit holdings” within a new sort of digital holdings” is introduced to describe the relationship
knowledge repository permit serializing, managing, between DKOs (or DKO clusters) and the knowledge
and disseminating many types of knowledge, and repository within which they are stored. Just as books
afford a means of formulating individualized health are the holdings stored a physical library, DKOs are
advice automatically using stored digital knowledge. the holdings stored in a new sort of digital knowledge
repository envisioned here. Unlike books, however, the
content in DKOs is subject to change throughout the
DKO life cycle and DKOs can be semantically linked
1. Introduction directly to one another in innumerable ways. Thus,
DKOs and DKO clusters are two identifiable
To be high-functioning, a Learning Health System constituent units in the hybrid semantic networks that
(LHS) needs to be capable of providing clear, relevant, are stored within digital knowledge repositories. For
reliable health advice to all who are in need of it.1 This this reason, individual DKOs or DKO clusters are
advice must be based on up-to-date biomedical referred to hereon as unit holdings.
knowledge and provided just in time when it is needed. Specific DKOs and DKO clusters contain
To draw near to such an advice-giving capability is one knowledge payloads of various types. These payloads
of the grand challenges of bringing about virtuous are represented in multiple formats with varying
cycles of ongoing health improvement within an LHS.1 degrees of explicitness. Ultra-large collections of
As a small step towards the ultimate objective of DKOs and DKO clusters are envisioned as a partial
generating clear, relevant, reliable, timely health advice solution to the knowledge management challenges that
for all people who require it, this paper specifies a pertain to a high-functioning LHS.1
novel design for Digital Knowledge Objects (DKOs).
The authors designed DKOs for the dual purposes of
recontextualizing empirically derived biomedical
knowledge and formulating individualized advice.
DKOs are essentially containers of biomedical
knowledge artifacts that can be interacted with directly
or indirectly, by computers or by people, to gain advice
and learn how to improve human health.

1530-1605/16 $31.00 © 2016 IEEE 3308

DOI 10.1109/HICSS.2016.413
 represented as digital documents contained within
DKOs. Other biomedical knowledge can be
represented as relations among DKOs. Relations
among DKOs comprise semantic network or “graph”
representations of knowledge. Throughout this paper,
the knowledge represented the first way within DKOs
is referred to as knowledge payloads while the
(d) semantic link knowledge represented the second way, by relating
DKOs, is referred to as interconnected knowledge.2
(c) metadata wrapper The main motivation to represent knowledge in
these two different ways is to meet the knowledge
(b) internal interface layer management and advice formulation requirements of a
high-functioning LHS. Knowledge management and
(a) knowledge payload core the need for health advice are addressed next. Then, to
complete this section, knowledge payloads are
Figure 1. Components of Digital generally described and an example payload is shared.
Knowledge Object (DKOs)
2.1. A Learning Health System requires a
To meet the demand for up to date health advice scalable knowledge management capability
based on valid knowledge arising from analyses of
reliable data, a LHS will require digital repositories of
To enable the continuous learning cycles that are
knowledge represented in transactional forms.3 Ultra-
intended to take place within a LHS6, it is necessary
large repositories of DKOs and DKO clusters offer
not only to represent knowledge in one or more
users the potential to formulate enough clear, relevant,
transactional forms, but also to manage ultra-large
reliable, timely health advice to meet rapidly growing
stores of digital knowledge well. Consider the learning
global demand. Unit holdings of DKOs stored in
cycle diagram shown below in Figure 2.
digital knowledge repositories can be linked to create
immense, dynamic semantic knowledge networks.
Open source knowledge repository software
platforms affording capabilities to create and manage
large DKO networks already exist. One such platform,
the Flexible and Extensible Digital Object and
Repository Architecture (FEDORA) platform, made
available via the Fedora Commons system, offers a
flexible digital object model with actionable objects.
Fedora Commons facilitates object interaction with
external software systems through application program
interfaces (APIs), and it includes a native semantic-
web based relationship building capability for
interrelating digital objects of all sorts to form
semantic networks.4,5
The focus of this paper is on the design of two key
unit holdings in digital knowledge repositories Figure 2. Learning Cycle Diagram
intended to support LHS stakeholder needs, DKOs and Highlighting the Steps of Storing
DKO clusters. In addition to describing a general Knowledge and Formulating Advice
design for DKOs, the design for one special type of
DKO cluster is also discussed in detail. The learning cycle diagram above, or a variant thereon,
is a feature of any LHS capable of rapid learning. The
cycle depicted has eight sequential functions. To
2. Background and Motivation
understand the cycle, begin at point 1 where data are
collected. Once data are collected, they are assembled
This paper is based on the idea that a digital and then analyzed at points 2 and 3, respectively. At
knowledge repository for a learning health system must point 4, people interpret the results of data analyses to
represent biomedical knowledge in two fundamentally assess what knowledge has been gained. Knowledge
different ways. Some biomedical knowledge can be gained is then represented, stored, and used to

3309
formulate advice at points 5 and 6. These two functions 20%.8 Together these studies indicate that many
are the knowledge management functions of the questions are raised during clinical encounters but
learning cycle and so they appear in a grey curved never answered, suggesting a need for more and better
triangle. After advice is formulated, it is delivered at health advice and mechanisms of advice delivery.
point 7 to inform health decisions made by people. Information overload occurs when the burden of
Finally, the decisions taken and outcomes related to processing large amounts of information cannot be
them are new data that can be recorded at point 8. By overcome.9 Markman describes how the number of
collecting these newly recorded data, the learning cycle relevant inputs needed to choose an appropriate
may then repeat. A high-functioning, large-scale LHS oncology treatment plan is now overwhelming
must be able to operate many thousands of these practitioners’ ability to identify and assimilate them.10
learning cycles simultaneously.1 To overcome the problem of information overload in
Considering the learning cycle in Figure 2, a high- the health domain, Stead et al. envision complex
functioning learning health system needs a knowledge assistive computer systems that work to coordinate the
repository to store, manage, and curate accumulating thinking of multiple experts by providing them with
knowledge, and to make that knowledge accessible to “patient centered cognitive support.”11 Patient centered
LHS stakeholders. In addition, such a repository must cognitive support is an advanced type of highly
allow for knowledge to be stored in a manner that individualized and contextualized advice.
scales up efficiently and effectively, even when a LHS Health professionals are not immune to cognitive
grows very large. biases. Multiple studies indicate that physicians are
At any given time, a LHS must be self-aware. susceptible to probability neglect and outcome bias.12,13
Stakeholders in a LHS need to “know what the system Advice could be used to counteract these human
knows”, i.e., they need a means to inventory all the cognitive biases by providing information on
knowledge stored in its repository. The design of probabilities and population distributions.14
DKOs makes it possible to establish ultra-large Finally, current rates of preventable medical errors
repositories of biomedical knowledge artifacts, to may represent the most compelling evidence of the
store, search, sort, and maintain the biomedical need for more and better health advice. After
knowledge as the payloads of DKOs or relations compiling a number of studies of medical error, the
among them, and to access and apply stored knowledge report of the Institute of Medicine from 2000, To Err is
for the purpose of formulating advice. For this reason, Human, called for health care organizations to (1)
DKOs, DKO clusters, and the semantic networks they “avoid reliance on memory” by adopting “protocols
form are all important parts in required knowledge and checklists”, (2) to “use constraints and forcing
management components for a LHS. functions”, and (3) to “improve access to accurate,
timely information.”15 Protocols or guidelines,
2.2. Demand for well-informed health advice checklists, and production rules describing constraints
far outstrips current supply and forcing functions all appear here as different DKO
payload types (Table 1). Online repositories of DKOs
holding DKOnets are intended to improve access to
Motivating work to design and develop DKOs and
biomedical knowledge and to afford a capability to
semantic networks of DKOs is a concern that too often
formulate much more advice based on that knowledge.
needed advice is not available to improve health and
ensure the safety and quality of health care. Evidence
to support the assertion that there is not enough clear, 2.3. Knowledge payloads in DKOs
relevant, reliable, timely health advice is abundant.
Research shows that (a) health professionals Certainly not all knowledge can be removed from
experience gaps in their knowledge at the point of care, the minds of individual human experts.16 Nevertheless,
(b) health information overload and cognitive biases a huge and rapidly growing quantity of published
both need to be mitigated, and (c) there are many biomedical knowledge is available to be drawn upon to
preventable medical errors that might be avoided formulate health advice.17 This published knowledge
altogether with clear, relevant, reliable, timely advice. can be contained in many types of digital documents
In 1985, Covell et al. found that only 30% of that describe or explain some physical or intellectual
physician information needs about “patient phenomenon.18 This paper focuses on a scalable
management” were being met in everyday practice.7 In infrastructure for managing instances of published
2007, González estimated that the percentage of biomedical knowledge for the particular purpose of
primary care physician information needs being met formulating advice. Here we are particularly interested
when questions arise during practice had fallen to in micropublications of knowledge artifacts, and
especially in representing scientific claims, rules, and

3310
models.19 These types of micropublications make example has been selected because (a) there are
potentially useful knowledge payloads for DKOs. published measures that pertain to OCD, (b) treatment
Digital Knowledge Objects carry knowledge recommendations for OCD vary, suggesting that there
payloads as byte streams in their “cores.” In the is more to learn about treating OCD, thus making OCD
context of this paper, we define knowledge payloads in a realistic case of potential interest to stakeholders in a
general as analytic outputs from empirical studies LHS, and (c) a “pipeline” of three interconnected
found to be valid in some context by an expert DKOs with three different knowledge payloads can
community.20 The important role expert communities ultimately be elaborated using this example. The three
play by validating knowledge within a high- interconnected DKOs in this OCD example have
functioning LHS is acknowledged and supported here. knowledge payloads explaining, (i) how to measure
There are as many potential DKO knowledge OCD symptom severity, (ii) for which human
payloads as there are instances of published biomedical population the OCD symptom severity scale is
knowledge. Table 1 below lists examples of some applicable and valid, and (iii) how to use symptom
knowledge payload types. Each type of knowledge severity scores to guide OCD treatment.
payload has to be represented digitally in a byte stream A detailed example of the first knowledge payload
to become part of a DKO. In addition, a single DKO mentioned above that explains how to measure the
can carry the same knowledge payload encoded in severity of OCD is given next. The Yale-Brown
multiple ways. Obsessive-Compulsive Scale (Y-BOCS) is used to
In general, the preferred encodings for knowledge measure the severity of OCD illness. Table 2 shows the
payloads are those that are as transactional as possible, complete 10-item Y-BOCS scale of human
given the payload type. This means that the preferred obsessiveness and compulsiveness.21 Y-BOCS is an
forms of representing knowledge are those that are evidence-based, validated assessment scale taken from
strongly typed, homogenously structured, and the published biomedical literature.22
independent of presentation format.3 While the greatest
degree of transactionality is usually desired for the None Mild Moderate Severe Extreme
purpose of automating advice formulation, it is Obsessions
1 Time spent on 0 1 2 3 4
possible to encode knowledge in DKOs as simple
2 Interference by 0 1 2 3 4
plaintext, images, or waveforms. 3 Distress of 0 1 2 3 4
Definitely Partial Yields
Algorithm 4 Resistance to 0 1 2 3 4
Checklist Complete Much Moderate Little None
Clinical calculator 5 Control over 0 1 2 3 4
Clinical pathway None Mild Moderate Severe Extreme
Decision model Compulsions
Guideline 6 Time spent on 0 1 2 3 4
Narrative description 7 Interference by 0 1 2 3 4
Message tailoring model 8 Distress of 0 1 2 3 4
Order set Definitely Partial Yields
Production rule 9 Resistance to 0 1 2 3 4
Predictive model Complete Much Moderate Little None
Report format model 10 Control over 0 1 2 3 4
Scale Table 2. Yale-Brown Obsessive-
Template Compulsive (Y-BOCS) Scale
User model
Table 1. Some Types of Knowledge The Y-BOCS scale shown above could be transformed
Payloads Carried by DKOs into a byte stream and made into the knowledge
payload of a DKO in various ways. First, a digital
In the next subsection, attention is paid to a image of the Y-BOCS scale, as depicted in Table 2,
specific knowledge payload of the scale type. could be used and the scale could be encoded in one of
many image formats, e.g., gif, png, or jpg. As an
2.4. Example of a knowledge payload alternative yet still simple representation, the Y-BOCS
scale could be represented in a plaintext format.
A specific knowledge representation example is Neither the image or plaintext formats are highly
introduced in this subsection and used throughout the transactional. For a more transactional format, an
rest of the paper. The disease domain for this example XML-based format could be used such as the Triple-S
is obsessive-compulsive disorder or OCD. This XML 2.0 format for survey data and variables. In

3311
Table 3 below, the first item from the 10-item 3. Digital Knowledge Object Design
Y-BOCS scale is represented in Triple-S XML.
When the Y-BOCS scale is represented in Triple-S Besides the previously described knowledge
XML, it conforms to a known standard representation, payloads in DKOs, which are comprised of byte stream
its values can be strongly typed, and other software representations of biomedical knowledge artifacts, the
code can be used to present the scale in various ways. design of DKOs also includes three other components,
Hence, the Y-BOCS scale, when represented in Triple- as shown in Figure 1. These three additional
S XML, is more transactional than it otherwise would components are (1) the internal interface layer within
be in either an image or plaintext representation. By DKOs, (2) the metadata wrapper about DKOs, and (3)
more transactional, what is meant is that it is easier for the semantic links between DKOs used to create
a machine to automatically process the Y-BOCS if it is semantic networks of DKOs. These three components
encoded in XML. When used as the knowledge of the design of DKOs are discussed in detail next.
payload for a DKO, external software applications can
identify, parse, transform, and manipulate the XML
formatted Y-BOCS scale in ways that may facilitate its 3.1. Internal Interface Layer
dissemination and use as knowledge in the world. In
this case, the Y-BOCS formatted in Triple-S XML One of the key design requirements for DKOs is
could be automatically consumed by end-user health that the knowledge within them can be made
applications and the users of those applications could executable, either directly, by methods of accepting
more easily access and utilize the Y-BOCS scale to data as inputs, performing computations informed by
assess OCD disease severity. The need to have more knowledge payloads, and outputting results; or
published biomedical knowledge available in indirectly, by dissemination of the knowledge payloads
transactional forms for the purpose of automatically of the DKOs so they can be put to use during further
formulating advice has been previously reviewed. automated or manual downstream processing.
For direct execution of knowledge-based functions,
<?xml version=1.0?> a single DKO can have multiple methods. In the case
<sss version=2.0> of the Fedora Commons repository, clients can
<date>1 June 2015</date> discover the execution methods available for any DKO
<time>16:01</time> using a LIST service request.5
<origin>Program I, Macintosh OS 10.8.5</origin> To do indirect, downstream execution on
<user>A Flynn</user> computable knowledge disseminated from a DKO,
<survey> access to one or more representations of knowledge
<title>Yale-Brown Obsessive Compulsive Scale</title>
<record ident=Obsessions>
payloads is provided by a set of basic service requests.5
<variable ident=Item1 type=quantity> What results from any of these access service requests
<name>Degree of</name> is part or whole dissemination of the knowledge
<label>Time spent on</label> payload stored in the “core” of a DKO.
<position start=1> Sets of service requests can be associated with
<values range from 0 to 4> specific payload types for the purpose of affording
<value><value code = 0>None </value> methods to clients by payload type (Table 1). These
<value><value code = 1>Mild </value> sets of service requests comprise the internal interface
<value><value code = 2>Moderate </value> layer of any DKO.4 Take, for example, the “scale”
<value><value code = 3>Severe </value>
<value><value code = 4>Extreme </value>
knowledge payload type, of which the Y-BOCS is an
</values> example (Table 2). For DKOs with knowledge
</position> payloads of type scale, the following example set of
</variable> five service requests could provide useful knowledge
</record> dissemination methods to clients:
</sss>
Table 3. First item of the Y-BOCS (1) GET number of items
scale represented in Triple-S XML (2) GET full item names
as an example of a DKO payload (3) GET item labels
(4) GET executable web app of scale
(5) VIEW scale

A client call to these five service requests in the

internal interface layer of the DKO with a knowledge

3312
payload encoding the Y-BOCS scale in Triple-S XML Element Set or Metadata
could provide the following results in return: Ontology Element
Unique Identifier
(1) 10 Payload Type
(2) Obsessions: Degree of, Degree of, Degree Version
Dublin Core Title
of, Resistance degree, Completeness of
Dublin Core Subject
control; Compulsions: Degree of, Degree Dublin Core Description
of, Degree of, Resistance degree, Dublin Core Type
Completeness of control; Dublin Core Source
(3) Time spent on, Interference by, Distress Dublin Core Coverage
of, Resistance to, Control over Dublin Core Creator
(4) ID334.060115.ybocs.js* Dublin Core Publisher
(5) ID334.060115.ybocs.jpg* Dublin Core Contributor
Dublin Core Rights
* Filenames with extensions, .js and .jpg, represent an
Dublin Core Date
Javascript file and an image file disseminated from a DKO Dublin Core Format
carrying the Y-BOCS scale as its knowledge payload. Dublin Core Language
Dublin Core Provenance
Dublin Core Rights Holder
These five basic knowledge dissemination examples purl.org/mp/ Class
demonstrate how executable logic may be added to MeSH Subject Heading
DKOs as various service requests built into the internal SNOMED CT Clinical Term
interface layer. ICD-9 Code
The ability to interact directly with a DKO, or to ICD-10 Code
obtain a copy of part or all of the knowledge payload Premis Significant Properties Type
from any DKO via service requests, can be managed Premis Significant Properties Value
via access control mechanisms within a learning health Premis Fixity
Premis Size in bytes
system knowledge repostiory.5
Premis Format Name
Premis Format Designation
3.2. Metadata Wrapper Premis Format Version
Table 4. Mandatory Metadata
By design, all DKOs are “wrapped” with a Elements for DKOs
common set of metadata elements. Within the structure
of every DKO, two types of metadata elements are 3.3. Semantic Links
instantiated and saved. The first is a mandatory,
standard set of elements for all DKOs. Beyond this Individual DKOs can be linked to other DKOs via
mandatory set of metadata, additional metadata semantic web-based relationships encoded as subject-
elements can be added to any DKO. predicate-object triples between pairs of DKOs using
The mandatory set of DKO metadata includes the any World Wide Web Consortium (W3C)-defined
elements in Table 4. Most of the mandatory DKO Resource Description Framework (RDF) language.
metadata elements are associated with existing element In this subsection an example of the Simple
sets or ontologies, and many can be constrained to Knowledge Organization System (SKOS) vocabulary
controlled vocabularies. Some metadata elements may is used. (SKOS is a W3C standard available on the
be instantiated more than once, such as any of the Web at w3.org/TR/skos-reference/.) In particular, for
Dublin Core elements. Other metadata elements may this example the symmetric, non-transitive skos:related
be gleaned from automated scans of knowledge property is instantiated between two DKOs. The
payload content and added to DKOs as “coded skos:related property simply signifies that the
keywords”, e.g., Medical Subject Headings (MeSH) knowledge payload in one DKO is related to the
vocabulary terms. knowledge payload in another DKO, and vice versa.
Assume the Y-BOCS scale carrying DKO that has
been discussed earlier in this paper has a unique
identifier of “ID334.” Further assume that a second
DKO, with a unique identifier, “ID809”, has for its
knowledge payload a treatment algorithm for
obsessive-compulsive disorder (OCD) from the
American Psychiatric Association (APA).23 It is

3313
reasonable to suggest that an obsessiveness- 4.1. User Model DKO, “U”
compulsiveness scale payload is related to an OCD
treatment algorithm payload. Figure 3 depicts a The domain of application for a particular
skos:related relationship between these two DKOs. biomedical knowledge artifact specifies the population
The symmetric triples are ID334:is_related_to:ID809 of individuals to which the knowledge applies. A
and ID809:is_related_to:ID334. domain of application can be thought of as a type of
knowledge payload in its own right and represented in
as the knowledge payload of a DKO. In any UGDi
skos:related cluster, the “U” DKO carries a user model as its
knowledge payload. By definition, a UGDi cluster has
ID334 only one “U” DKO.
Y-BOCS Sometimes domains of application are described in
terms of inclusion and exclusion criteria. In the case of
the Y-BOCS obsessiveness-compulsiveness scale, the
ID809 scale is, “primarily meant for use in adults or older
APAOCD children.”21 Therefore, in this example, simple age-
based criteria combined with symptomatology could be
encoded as a user model payload in a “U” DKO.
Figure 3. W3C RDF skos:related Assume the “U” DKO to be associated with the DKO
Relationship Between Two DKOs carrying the Y-BOCS scale carries the following
compound rule in italics as its knowledge payload:
Many different types of relationships among DKOs
need to be supported within a digital knowledge USE THE Y-BOCS SCALE IF
repository for a LHS. The focus of the next section is (chronological age ≥ 16 years)
on one particular kind of DKO cluster made possible AND
by two other types of semantic links. (at least three of the following criteria are found:
1. repetitive behaviors, 2. anxiety
3. depression, 4. eating disorders
4. The UGDi Cluster Design
5. hypochondriasis, 6. paraphilia
7. trichotillomania, 8. self-injury)
This section describes a general instance of
interconnected knowledge, the UGDi cluster. In this example, the “U” DKO carries as its knowledge
Innumerable other DKO clusters besides UGDi payload a so-called user model comprised of the
clusters can be designed and created using semantic compound age and symptomatology rule above.
links. The knowledge representation example outlined
above for obsessive-compulsive disorder or OCD is 4.2. General Knowledge DKO, “G”
carried on through this section.
As previously mentioned, within a knowledge
A general knowledge DKO is simply a DKO that
repository, the design of DKOs permits DKOs to be
does not have a user model or a decision model for its
semantically linked. Semantic linking makes it possible
payload. Any of the other types of knowledge payloads
to design different kinds of DKO clusters to help meet
can be carried in a general knowledge DKO (Table 1).
the needs of knowledge consumers for health advice.
In any UGDi cluster of DKOs, the “G” stands for such
One potentially helpful kind of DKO cluster
a DKO. By definition, a UGDi cluster has only one
includes one DKO with a user model for its payload
“G” DKO.
(called the “U” DKO), one DKO with a payload that is
The DKO described in Section 2.4 that carries as its
not a user model or a decision model (called the “G”
knowledge payload the Y-BOCS scale is a general
DKO), and one or more DKOs with decision models as
knowledge DKO. It is the “G” DKO used to develop
their payloads (called the “D1…Di” DKOs.)
this specific example of a UGDi cluster.
UGDi clusters, formed by semantically linking a
“U”, “G”, and one or more “D” DKOs, are a
4.3. Decision Model DKOs, “D1…Di”
potentially helpful type of unit holding.
Decision models are a type of published biomedical
knowledge artifacts.24 Unlike decision trees that relate
multiple decisions to one another, decision models
apply to individual decisions and relate (a) conditions

3314
or considerations and (b) alternatives or options with cito:uses_data_from properties from the Citation
(c) a set of rules for making some decision. Semantic Typing Ontology (cito) to semantically link the “G”
decision tables combining these components, a, b, and and “D1” DKOs to form the completed UGDi cluster
c, have been developed and used to represent decision shown in Figure 4. Note that in this example the
models for computation.25 properties specified are not symmetric and are instead
A UGDi cluster may involve an unlimited quantity inverse properties.
of decision models, denoted D1…Di. In the example
being developed here, knowledge about obsessive-
compulsive disorder, specifically the Y-BOCS
assessment scale and a rule-based user model for the “U”
Y-BOCS assessment scale are included. To form a Y-BOCS
user
UGDi cluster, at least one DKO with a decision model model
for its knowledge payload must also be specified.
For the purpose of describing an actual UGDi
cito:qualifies cito:is_qualified_by
cluster, a very simple decision model about when to
treat OCD, based on a published meta-analysis26, is
specified in the form of three rules given in italics: “G”
Y-BOCS
scale
I. IF Y-BOCS score < 12 THEN do NOT treat

II. IF Y-BOCS score ≥ 12 and < 16 THEN consider cito:provides_data_for cito:uses_data_from

treatment
“D1”
III. IF Y-BOCS score ≥ 16 then definitely treat OCD
decision
In this example, there is only one DKO with a decision model
model as its knowledge payload, the “D1” DKO. Its
payload is a decision model in the form of the three Figure 4. Example of UGDi cluster
rules above, I-III. All of the DKOs needed to form a of DKOs for OCD
UGDi cluster have now been described in subsections
4.1, 4.2, and 4.3.
4.5. Potential of the UGDi cluster
4.4. UGDi cluster relationships
The purpose for elaborating the UGDi cluster in
this paper is specifically to relate its potential. In a
Now that the three payload types found in the
LHS knowledge repository, knowledge is not only
DKOs comprising a UGDi cluster have been reviewed
represented as the payloads carried within DKOs but it
and examples given, the description of the design of a
is also represented explicitly as interconnected
UGDi cluster is completed by specifying the semantic
knowledge via semantic links, or specified
links between the DKOs that comprise the cluster.
relationships, among DKOs. UGDi clusters within
In a UGDi cluster, the semantic link between the
semantic networks are a unique type of unit holding
one “U” DKO and the one “G” DKO can be specified
within a digital knowledge repository for a LHS.
using the cito:qualifies and cito:is_qualified_by
In particular, the UGDi cluster is important and
properties from the Citation Typing Ontology
potentially helpful because it can encode a “pipeline”
(purl.org/spar/cito). In a UGDi cluster, a user model is
of interconnected knowledge. In the OCD example, the
knowledge that qualifies (or places conditions or
UGDi cluster in Figure 4 relates knowledge of how to
restrictions upon) some general knowledge. Likewise,
measure OCD severity using the Y-BOCS scale to
the semantic links between a “G” DKO and one or
knowledge about the population of individuals for
more “D” DKOs may also be specified using
whom the Y-BOCS scale measure is valid to
properties from the Citation Typing Ontology.
knowledge of how to apply the results of the Y-BOCS
In the example elaborated here, because the general
scale severity measurement to an OCD treatment
knowledge “G” DKO carries the Y-BOCS scale as its
decision.
knowledge payload and the “D1” DKO carries a
In general, the “U” DKO carrying a user model as
decision model comprised of three treatment rules that
its knowledge payload affords an opportunity to
make use of the Y-BOCS scale scores, it is reasonable
computationally disseminate knowledge to those for
to specify the cito:provides_data_for and

3315
whom it applies. The “D1” DKO affords an opportunity requests for any given payload type has been written, it
to disseminate applicable knowledge in an appropriate can easily be referenced and reused by any DKOs
decision-making context. carrying that same knowledge payload type.
In the OCD example provided, if an end user Finally, external relationships among DKOs are
documented a Y-BOCS score for an adult with at least specified using semantic web subject-predicate-object
three OCD symptoms, the event of documenting the triples. While it is possible to define and publish new
Y-BOCS score could be used as a trigger to interact RDF terms and vocabularies, it saves time and
with a digital knowledge repository and to receive in promotes standardization to use existing RDF
return automatically formulated treatment advice based vocabularies as much as possible. An information
on the knowledge payload in the “D1” DKO. resource available to find existing RDF vocabularies is
the Linked Open Vocabularies website hosted by the
5. The DKO Construction Method Open Knowledge Foundation at lov.okfn.org.

In order to construct DKOs, a knowledge repository 6. Conclusions and Future Work

platform is required. Our DKO construction method
uses the open source Fedora Commons system, version Two unit holdings in large repositories of digital
3.x as a platform. Fedora Commons is available for health knowledge, DKOs and DKO clusters, have been
download on the Web at fedorarepository.org. described. DKOs can carry computable representations
Before a DKO can be constructed, the knowledge of biomedical knowledge related to any health problem
to be carried in the payload of the DKO has to be of interest in support of any learning goal. In a
represented in one or more forms that can be serialized Learning Health System, important learning goals,
as digital byte streams. For DKOs to be more than such as learning to prevent falls in nursing homes, or
containers for text documents and images in common learning how to improve adherence to chronic
formats, such as the portable document format (PDF) medication regimens, may be more easily met when
or graphic interchange format (GIF), respectively, reliable knowledge is managed in transactional forms
some other means of knowledge representation must be within ultra-large knowledge repositories.
used. Knowledge representation techniques will vary Whereas today, the process of building DKOs and
by payload type and by application. For clinical DKO clusters to create semantic networks of DKOs
calculators and for some predictive models, within digital knowledge repositories is laborious,
representation may be trivial if the knowledge to be future work to create DKO viewers and authoring
encoded is already expressed in the language of tools, and to write and publish transferable and
mathematics. On the other hand, some clinical reusable DKO interface service request code, may
guideline and clinical pathway payloads may be very assist to bring about the necessary ability to efficiently
difficult to represent completely for automated and effectively create and relate DKOs as a means of
processing due to ambiguities or unknowns. managing biomedical knowledge on an ultra-large
Once the knowledge payload for a DKO is scale and as a way of meeting the actual health advice
available as a byte stream, a new DKO can be created formulation needs of stakeholders in learning health
using built-in Fedora Commons functions. At this systems.
point, the Fedora Commons platform will assign a
unique ID to the new DKO automatically. 7. References
The next step is to add the mandatory metadata to
the DKO. There are multiple ways to do this. When
[1] Friedman C, Rubin J, Brown J, et al. Toward a science of
building DKOs it is possible to add metadata
learning systems: a research agenda for the high-functioning
datastreams directly in Fedora Commons for the Learning Health System. J Am Med Informatics Assoc.
mandatory metadata elements shown in Table 4. In the 2014:43-50.
DKO design described here, the specified mandatory
elements are those metadata elements that support a [2] Sowa JF. Semantic networks. Enclycopedia Artif Intell.
coordinated approach to knowledge management using 1992.
Fedora Commons.
After the metadata for a DKO is specified, work [3] Glushko RJ. The Discipline of Organizing. MIT Press;
can begin on the code for service requests that form the 2013.
internal interface layer of the DKO. Because sets of
service requests can be associated with a particular
payload type, once the code for a set of service

3316
[4] Payette S, Lagoze C. Flexible and Extensible Digital [18] Day RF, Martnet L, Anghelescu HGB. Suzanne Briet’s
Object and Repository Architecture (FEDORA). Proc What Is Documentation?; 2006.
Second Eur Conf Res Adv Technol Digit Libr Lect Notes
Comput Sci. 1998;1513(41):41-59. [19] Clark T, Ciccarese PN, Goble C a. Micropublications: a
semantic model for claims, evidence, arguments and
[5] Lagoze C, Payette S, Shin E, Wilper C. Fedora: an annotations in biomedical communications. J Biomed
architechture for complex objects and their relationships. Int Semantics. 2014;5(1):28.
J Digit Libr. 2006;6(2):124-138.
[20] Slutsky JR. Moving closer to a rapid-learning health care
[6] Etheredge LM. A rapid-learning health system. Health system. Health Aff. 2007;26. doi:10.1377/hlthaff.26.2.w122.
Aff. 2007;26.
[21] Goodman WK, Price LH, Rasmussen SA, et al. The
[7] Covell DG, Uman GC, Manning PR. Information needs in Yale-Brown Obsessive Compulsive Scale. Arch Gen
office practice: are they being met? Ann Intern Med. Psychiatry. 1989;46:1006-1011.
1985;103(4):596-599.
[22] Goodman WK, Price LH, Rasmussen SA, et al. The
[8] González-González A. Information needs and Yale-Brown Obsessive Compulsive Scale II. Validity. Arch
information-seeking behavior of primary care physicians. Gen Psychiatry. 1989.
Ann Fam …. 2007:345-352.
[23] Koran LM, Hanna GL, Hollander E, Nestadt G, Simpson
[9] Clarke MA, Belden JL, Koopman RJ, et al. Information HB. Treatment of Patients With Obsessive-Compulsive
needs and information-seeking behaviour analysis of primary Disorder.; 2007.
care physicians and nurses: a literature review. Health Info
Libr J. 2013;30(3):178-190. [24] Khandker RK, Dulski JD, Kilpatrick JB, Ellis RP,
Mitchell JB, Baine WB. A decision model and cost-
[10] Markman M. Information overload in oncology practice effectiveness analysis of colorectal cancer screening and
and its potential negative impact on the delivery of optimal surveillance guidelines for average-risk adults. Int J Technol
patient care. Curr Oncol Rep. 2011;13(4):249-251. Assess Health Care. 2000;16(3):799-810.

[11]Stead WW, Searle JR, Fessler HE, Smith JW, Shortliffe [25] Meersman R, Tang Y. On Constructing Semantic
EH. Biomedical informatics: changing what physicians need Decision Tables. 2007;2007:34-44.
to know and how they learn. Acad Med. 2011;86(4):429-434.
[26] Eddy KT, Dutra L, Bradley R, Westen D. A
[12] Poses RM, Anthony M. Availability, Wishful Thinking, multidimensional meta-analysis of psychotherapy and
and Physicians’ Diagnostic Judgments for Patients with pharmacotherapy for obsessive-compulsive disorder. Clin
Suspected Bacteremia. Med Decis Mak. 1991;11(3):159-168. Psychol Rev. 2004;24(8):1011-1030.

[13] Gruppen L, Margolin J. Outcome bias and cognitive

dissonance in evaluating treatment decisions. Academic
Medicine. 1994..

[14] Sunstein CR. Probability neglect: Emotions, worst cases,

and law. 2002;112(1):61-107.

[15] Kohn LT, Corrigan J, Donaldson MS. To Err Is Human:

Building a Safer Health System M S Washington DC USA:
Institute of Medicine/National Academy Press; 2000.

[16] Ackerman M, Halverson C. Sharing expertise: The next

step for knowledge management. Soc Cap Inf. 2004:273-299.

[17] Larsen PO, von Ins M. The rate of growth in scientific

publication and the decline in coverage provided by science
citation index. Scientometrics. 2010;84(3):575-603.

3317