Data Management and Visualization

Introduction to Data Management

Lecture – 1

Department of Public Health(Health Informatics Program)

College of Nursing and Health Sciences
Jazan University
Outlines:
● What is data Management?

● Purpose of data Management

● Types of data Management

● Components of data Management Framework

● Data Management Policies

● What is the Data Life Cycle?

● Stages in Data Life cycle

● Data Storage

● Benefits of Data Management Systems

● Data Management Challenges

 Data management

• Data management is the practice of collecting, keeping, and using data

securely, efficiently, and cost-effectively.
• The data management process includes a combination of different
functions that collectively aim to make sure that the data in corporate
systems is accurate, available and accessible.
Goal:
• The goal of data management is to help people, organizations, and
connected things optimize the use of data within the bounds of policy
and regulation so that they can make decisions and take actions that
maximize the benefit to the organization.
Purpose of Data Management
• Managing digital data in an organization involves a broad range of tasks,
policies, procedures, and practices.
• The work of data management has a wide scope, covering factors such as
how to:
• Create, access, and update data across a diverse data tier
• Store data across multiple clouds and on premises
• Provide high availability and disaster recovery
• Use data in a growing variety of apps, analytics, and algorithms
• Ensure data privacy and security
• Archive and destroy data in accordance with retention schedules and
compliance requirements
 Importance of Data Management

• Data management helps to minimize potential errors by establishing

processes and policies for usage and building trust in the data being
used to make decisions across the organization.

• With reliable, up-to-date data, companies can respond more efficiently

to market changes and customer needs.
Types of Data Management
There are 7 types of data management
• Data management experts generally focus on specialties within the field.
These specialties can fall under one or more of the following areas:
1.Master data management: Master data management (MDM) is the
process of ensuring the organization is always working with — and making
decisions based on — a single version of current, “true” information.
Ingesting data from all of your sources and presenting it as one constant,
reliable source, as well as repropagating data into different systems, requires
the right tools.
1.Data stewardship: A data steward does not develop information
management policies, but rather deploys and enforces them across the
enterprise. As the name implies, a data steward stands watch over enterprise
data collection and movement policies, ensuring practices are implemented
and rules are enforced.
3.Data quality management: If a data steward is a kind of digital sheriff, a data quality manager
might be thought of as his court clerk. Quality management is responsible for combing collected data
for underlying problems like duplicate records, inconsistent versions, and more. Data quality
managers support the defined data management system.

4.Data security: One of the most important aspects of data management today is security. Though
emergent practices like DevSecOps incorporate security considerations at every level of application
development and data exchange, security specialists are still tasked with encryption management,
preventing unauthorized access, guarding against accidental movement or deletion, and other
frontline concerns.
5.Data governance: Data governance sets the law for an enterprise’s state of information. A data governance
framework is like a constitution that clearly outlines policies for the intake, flow, and protection of institutional
information.

Data governors oversee their network of stewards, quality management professionals, security teams, and other people
and data management processes in pursuit of a governance policy that serves a master data management approach.

6.Big data management: Big data is the catch-all term used to describe gathering, analyzing, and using massive
amounts of digital information to improve operations.

In broad terms, this area of data management specializes in intake, integrity, and storage of the tide of raw data that
other management teams use to improve operations and security or inform business intelligence.

7. Data warehousing: Information is the building block of modern business. The sheer volume of information presents
an obvious challenge: What do we do with all these blocks? Data warehouse management provides and oversees the
physical and/or cloud-based infrastructure used to aggregate raw data and analyze it in-depth to produce business
insights.
 Components of Data Management

1.Data Governance: Data Governance provides the overarching

support to data management through stewardship, policies, processes,
standards, and adherence to leading practices.
2.Data Architecture: Data Architecture provides the infrastructure for
the storage, integration, and use of data throughout the organization.
3.Metadata: Metadata allows you to use data more efficiently by
providing critical information about data attributes.
4.Data Quality: Data Quality provides the structure necessary to have
data that fulfills the needs of the business.
5.Data Lifecycle:
The Data Lifecycle follows the data throughout the company, providing
integrity from the initial introduction into the company through the final
deletion from the company.
6.Analytics:
Analytics applies statistical and visualization techniques that lead to valuable
insights that can help the company make better business decisions.
7.Data Privacy:
Data Privacy supports the needs of the business to share data internally and
externally.
 Data Management Policy
• A data management policy addresses the operating policy that focuses
on the management and governance of data assets.
• The data management policy should contain all the guidelines and
information necessary for governing enterprise data assets and should
address the management of structured, semi-structured
and unstructured data.
 What does a data Policy contain?
A comprehensive Data Management Policy should contain the following:

• An inventory of the organization’s data assets

• A strategy of effective management of the organization’s data assets
• An appropriate level of security and protection for the data including details
of which roles can access with data elements
• Categorization of the different sensitivity and confidentiality levels of the
data
• The objectives for measuring expectations and success
• Details of the laws and regulations that must be adhered to regarding the
data program
 Data Life Cycle

• The data life cycle, also called the information life cycle, refers to the
entire period of time that data exists in your system.

• This life cycle encompasses all the stages that your data goes through,
from first capture onward.
Data Life Cycle Stages

1.Generation.
2.Collection
3.Processing.
4.Storage. ...
5.Management. ...
6.Analysis. ...
7.Visualization. ...
8.Interpretation.
1. Generation: For the data life cycle to begin, data must first be generated. data is generated by your

organization, some by your customers, and some by third parties you may or may not be aware of. Every sale,

purchase, hire, communication, interaction—everything generates data ...

2. Collection: Not all of the data that's generated every day is collected or used.

• You can collect data in a variety of ways, including:

• Forms: Web forms, client or customer intake forms, vendor forms, and human resources applications are

some of the most common ways businesses generate data.

• Surveys: Surveys can be an effective way to gather vast amounts of information from a large number of

respondents.
• Interviews: Interviews and focus groups conducted with customers, users, or job
applicants offer opportunities to gather qualitative and subjective data that may be
difficult to capture through other means.

• Direct Observation: Observing how a customer interacts with your website,

application, or product can be an effective way to gather data that may not be
offered through the methods above.
3. Processing
• Once data has been collected, it must be processed. Data processing can refer to
various activities, including:
• Data wrangling, in which a data set is cleaned and transformed from its raw form
into something more accessible and usable. This is also known as data cleaning,
data munging, or data remediation.
• Data compression, in which data is transformed into a format that can be more
efficiently stored.
• Data encryption, in which data is translated into another form of code to protect it
from privacy concerns.
4. Storage
• After data has been collected and processed, it must be stored for
future use.
• This is most commonly achieved through the creation of databases or
datasets. These datasets may then be stored in the cloud, on servers, or
using another form of physical storage like a hard drive, CD, cassette,
or floppy disk.
• When determining how to best store data for your organization, it’s
important to build in a certain level of redundancy to ensure that a
copy of data will be protected and accessible, even if the original
source becomes corrupted or compromised.
5. Management
• Data management, also called database management, involves organizing,
storing, and retrieving data as necessary over the life of a data project.
• While referred to here as a “step,” it’s an ongoing process that takes place
from the beginning through the end of a project.
• Data management includes everything from storage and encryption to
implementing access logs and changelogs that track who has accessed data
and what changes they may have made.
6. Analysis
• Data analysis refers to processes that attempt to clean meaningful
insights from raw data.
• Analysts and data scientists use different tools and strategies to
conduct these analyses.
• Some of the more commonly used methods include statistical
modeling, algorithms, artificial intelligence, data mining, and machine
learning.
7. Visualization
• Data visualization refers to the process of creating graphical
representations of your information, typically through the use of one
or more visualization tools.
• Visualizing data makes it easier to quickly communicate your analysis
to a wider audience both inside and outside your organization.
• The form your visualization takes depends on the data you’re working
with, as well as the story you want to communicate.
8. Interpretation
• Finally, the interpretation phase of the data life cycle provides the
opportunity to make sense of your analysis and visualization.
• Beyond simply presenting the data, this is when you investigate it
through the lens of your expertise and understanding.
• The interpretation may not only include a description or explanation of
what the data shows but, more importantly, what the implications may
be.
Data Storage
• Data storage refers to magnetic, optical or mechanical media that
records and preserves digital information for ongoing or future .
Data storage refers to the use of recording media to retain data
using computers or other devices.
• The most prevalent forms of data storage are file storage, block
storage, and object storage, with each being ideal for different
purposes.
• There are three main types of data storage on the market: cloud-based,
server-based (also known as hyper-convergence), and traditional.
• Data storage in a digital, machine-readable medium is sometimes
called digital data
• Computer data storage is one of the core functions of a general-
purpose computer.
• Electronic documents can be stored in much less space than
paper documents.
• Barcodes and magnetic ink character recognition (MICR) are two
ways of recording machine-readable data on paper.
• Data storage describes what type of, where, and how hardware or
software holds, deletes, backs up, organizes, and secures information.
This includes keeping data in temporary or permanent storage.
• The digitization of manufacturing, known as Industry 4.0, is a good
example of how the enormous volumes of data, real-time analysis, and
quickness drive innovation and increase data storage demands.
• The simplicity of using a paper punch card to hold data has long gone
by the wayside. Today data can be stored not only on hard disks,
memory cards, and DVDs but also in the cloud, and on atoms and
DNA
• What type of thing holds the data?
• For example, data can sit on hard disks, flash drives, Non-Volatile
Memory Express (NVMe) systems, and DNA. A virtual software-
defined infrastructure also may hold data.
• Where is the data stored?
• For example, data can be stored on-premise, in server farms, on
the Internet of Things (IoT), or through a data storage service, as a
cloud provider.
• How is the data stored?
For example, solid-state drives use “electronically programmable and
erasable memory microchips” to store data. Other storage devices may
use Light Store, an environmentally friendly technology, or flash
memory, an ”electronic, non-volatile data storage medium that is erased
and reprogrammed electrically” to store data.
 Benefits of data management systems
• Data management processes help organizations to identify and resolve internal
pain points and deliver a better customer experience.
• First, data management provides businesses with a way of measuring the amount
of data in play.
• A myriad of interactions occur in the background of any business — between
network infrastructure, software applications, APIs and security protocols.
• Data management gives managers a big-picture look at business, which helps with
both perspective and planning.
• Once data is under management, it can be mined for informational gold: business
intelligence. This helps organizations in a variety of ways, including:
• Smart advertising that targets customers according to their interests and
interactions
• Holistic security that safeguards critical information
• Alignment with relevant compliance standards, saving time and
money
• Machine learning that grows more environmentally aware over time,
powering automatic and continuous improvement
• Reduced operating expenses through utilization of only the necessary
storage and compute power required for optimal performance
Data management challenges

There are four key data management challenges to anticipate:

1.The amount of data can be (at least temporarily) overwhelming. It’s hard to
overstate the volume of data that must come under management in a modern
business, so, when developing systems and processes, be ready to think big.
Really big. Specialized third-party services for integrating big data or providing it
as a platform are crucial allies.
2.Many organizations silo data. The development team may work from one data
set, the sales team from another, operations from another, and so on. Modern data
management relies on access to all this information to develop modern business
intelligence. Real-time data platform services help stream and share clean
information between teams from a single, trusted source.
3.The journey from unstructured to structured can be steep. Data often
pours into organizations in an unstructured way.
Before it can be used to generate business intelligence, data preparation has to
happen: Data must be organized, de-duplicated, and otherwise cleaned.
Data managers often rely on third-party partnerships to assist with these
processes, using tools designed for on-premises, cloud, or hybrid
environments.
4.Managing the culture is essential to managing data. All of the processes
and systems in the world produce little good if people don’t know how — and
perhaps just as importantly, why — to use them.
By making team members aware of the benefits of data management and
fostering the skills of using data correctly, managers engage team members as
essential pieces of the information process.