Disease Prediction From Medical Data

A PROJECT REPORT

Submitted by
Uttam Mandiwal(22BCS10399)
Vivek Poonia(22BCS10478)
Uday Mandiwal(22BCS10407)
Ayush Shastri(22BCS80304)
Mohit (22BCS14664)

in partial fulfillment for the award of the degree

of

Bachelors of Engineering
IN
Computer Science
Supervisor – Mr. Suraj Pal Singh

Chandigarh University

February – April 2024

 BONAFIDE CERTIFICATE

Certified that this project report “DISEASE PREDICTION FROM MEDICAL

ATA” is the bonafide work of “UTTAM MANDIWAL, VIVEK POONIA, UDAY
MANDIWAL, AYUSH SHASTRI, MOHIT” who carried out the project work
under my/our supervision.

SIGNATURE SIGNATURE

Prof. Suraj Pal Singh

Dr. Jaspreet Singh SUPERVISOR
HEAD OF THE DEPARTMENT Assist. Professor

Department of Computer Science Department of Computer Science

and Engineering and Engineering

Submitted for the project viva-voce examination held on: 25-04-2024

INTERNAL EXAMINER EXTERNAL EXAMINER

 TABLE OF CONTENTS

CHAPTER 1. INTRODUCTION ....................................................................... 11

1.1. Identification of Client/ Need/ Relevant Contemporary issue ....................................... 8-9

1.2. Identification of Problem………………………………………………………………10

1.3. Identification of Tasks ................................................................................................. .. 10

1.4. Timeline .......................................................................................................................... 12

1.5. Organization of the Report……………………………………………………………...13

CHAPTER 2. LITERATURE REVIEW/BACKGROUND STUDY………14

2.1. Timeline of the reported problem……………………………………………………...14

2.2. Existing solutions ...................................................................................................... ....15

2.3. Bibliometric analysis ................................................................................................. …16

2.4. Review Summary………………………………………………………………………17

2.5. Problem Definition……………………………………………………………….....18-19

2.6. Goals/Objectives……………………………………………………………...…….19-20

CHAPTER 3. DESIGN FLOW/PROCESS…………………………………..21

3.1. Evaluation & Selection of Specifications/Features…………………………………21-22

3.2. Design Constraints ..................................................................................................... 22-24

3.3. Analysis of Features and finalization subject to constraints ...................................... 24-25

3.4. Design Flow…………………………………………………………………………25-27

3.5. Design selection……………………………………………………………………… 27

3.6. Implementation plan/methodology ............................................................................ ... 28

CHAPTER 4. RESULTS ANALYSIS AND VALIDATION ........................... 29
4.1. Implementation of solution........................................................................................ 29-33

CHAPTER 5. CONCLUSION AND FUTURE WORK ................................... 34

5.1. Conclusion…………………………………………………………………………….. ..34

5.2. Future work..................................................................................................................... 35

REFERENCES ....................................................................................................... 36
 List of Figures

Figure 1.1 ............................................................................................................ 13

Figure 3.1 ............................................................................................................................... 28

Figure 4.1 ............................................................................................................................... 33

 ABSTRACT

Majority of population in this world face issue towards disease as they don’t know about the illness
from they are suffering. Sometime disease can be cure in early stage by the patient itself, but they
are not aware about their disease. In the proposed system, it provide the application of machine
learning algorithms to predict the onset, progression, and outcomes of various diseases based on
comprehensive medical datasets. It experiment the altered estimate models over real-life medical
data collected. The research focuses on utilizing diverse types of medical data, including
demographic information, clinical history, laboratory results, imaging data, and genetic markers, to
develop accurate predictive models. The study also highlights real-world applications of disease
prediction models in clinical practice, such as early detection of chronic diseases, personalized
treatment planning, and healthcare resource allocation. Moreover, the potential impact of
integrating predictive analytics into healthcare systems for improving patient outcomes and
reducing healthcare costs is examined.
 CHAPTER 1

INTRODUCTION
1.1 Identification of Client /Need / Relevant Contemporary issue

• Globally, people's health conditions are a complicated tapestry influenced by a multitude of

causes. The international community may strive toward building a healthier and more
equitable society for all by addressing infectious illnesses, non-communicable diseases,
mental health, maternity and child health, healthcare inequalities, and strengthening global
health governance. Numerous factors, such as socioeconomic status, cultural customs,
environmental factors, and the state of healthcare systems, influence people's health globally.
The range of illnesses impacting individuals worldwide, emphasizing difficulties.

Infectious diseases that spread quickly across borders and impact people worldwide include
COVID-19, influenza, HIV/AIDS, and tuberculosis. Coordinated international efforts are
frequently needed to control and stop outbreaks of these illnesses. According to estimates
from the World Health Organization (WHO), a sizable percentage of the world's disease
burden goes undetected or untreated. Both infectious and non-communicable diseases fall
under this category.
The WHO estimates that at least half of the world's population still does not have access to
basic medical care. Delays in diagnosis and treatment can be attributed to limited access to
healthcare services.

Global disparities occur in the quality and accessibility of healthcare. The Global Burden of
Disease Study and other studies show that variables including geography, socioeconomic
status, and educational attainment affect health outcomes differently.
There may be a lack of awareness among a large number of individuals worldwide regarding
the prevention of certain diseases, which increases the risk of infection or other health
problems. Many regions of the world have widespread practices related to self-medication.
While self-medication is not always a sign of inattention, it can occasionally be the
consequence of inadequate understanding regarding the possible hazards and repercussions
of untreated illnesses.
• India faces challenges related to infectious diseases, including malaria, tuberculosis, dengue,
and waterborne diseases. These diseases are more common and are frequently associated
with socioeconomic situations, inadequate healthcare facilities, and poor sanitation. In India,
the prevalence of non-communicable diseases such as diabetes, respiratory conditions, and
cardiovascular diseases is rising. The rising prevalence of NCDs is attributed to lifestyle
factors such tobacco smoking, inactivity, and poor diets.
Even though India has a sizable public healthcare system, the resources and infrastructure
are frequently insufficient to meet the country's population's healthcare needs. Financial
hardships affect a large segment of the Indian people, with many of them living in poverty.
People with little financial resources find it challenging to pay for important treatments,
diagnostic testing, and medical consultations. Healthcare access varies significantly by
location, with rural locations frequently encountering greater difficulties than urban
areas.Accessing medical treatments might be challenging in distant places due to a lack of
healthcare facilities.
• Many reported having health problems for which they were unable to determine the
underlying cause. Different people have had differing degrees of influence from this lack of
identification on their regular activities; many have reported a considerable impact.
Moreover, the majority have postponed seeking medical help because they are hesitant,
which highlights potential obstacles to timely access to healthcare. The information also
shows that a sizeable portion of respondents do not have health insurance, and that one of
the main obstacles to receiving treatment is financial hardship.
Seventy percent of respondents were unable to determine the reason behind their health
problems of which 40% feel that their health problem interferes with their day-to-day
activities.
• Conducting a survey is an excellent way to justify the need for disease prediction from
medical data. A survey can provide quantitative and qualitative data, capturing the
perspectives of relevant stakeholders and shedding light on the specific challenges and
requirements. Let's explore how the results of a survey can justify the need:

1. Stakeholder Opinions: A survey may involve a number of stakeholders, such as

administrators, patients, data scientists, and healthcare experts. Through gathering their
viewpoints and observations, the questionnaire can disclose the general understanding of
the significance of illness prognosis in enhancing medical results.
2. Disease Prevalence: Incorporate survey questions regarding the occurrence of illnesses
in the intended audience. The aforementioned facts may underscore the pressing need to
create efficacious predictive models tailored to the unique health needs of the
community.
3. The effects of delayed diagnosis: Find out how delayed diagnosis is thought to affect
patient outcomes and medical expenses. Reactions have the ability to measure the
problem's importance and highlight the necessity of fast and precise illness prediction.
4. Effectiveness of the Current Predictive Model: Check if any stakeholders are happy with
the predictive models that are currently in use. To determine the weaknesses and potential
areas for development in the current models, compile information on their efficiency,
accuracy, and dependability.
5. Challenges with Data Integration: Examine the difficulties in combining medical data
from various sources. Concerns about data sharing, interoperability, and standardisation
can highlight the obstacles that successful disease prediction requires overcoming.
6. Privacy Issues: Examine the degree of concern over data security and patient privacy.
Determine whether privacy concerns are preventing stakeholders from sharing or using
medical data, and evaluate the significance of incorporating privacy-preserving methods
into predictive models.
7. Limitations on Resources: Find out about staff and technological resource limitations.
Determine whether more resources are required to improve the creation and application
of illness prediction systems.
8. Multidisciplinary Cooperation: Evaluate the degree of cooperation between data
scientists and healthcare practitioners. Determine whether additional teamwork or
multidisciplinary training is required to match the clinical needs of the prediction model.
 1.2. Identification of Problem

• Individuals should make their health a top priority, consult a doctor when symptoms are
bothersome, and schedule regular checkups. Frequent health screenings and a proactive attitude
to treatment can help with early illness identification and prompt treatment, which will
ultimately improve people's general health. The identification of the problem for disease
prediction from medical data revolves around several key challenges:

1. Integrity and Quality of Data: The construction of trustworthy predictive models is hampered
by the fragmentation, inconsistencies, and errors present in medical data from multiple
sources. One major problem is to achieve seamless integration and standardisation of various
databases.

2. Data Heterogeneity: There are various forms, structures, and granularities of medical data. A
significant difficulty is integrating data from patient reports, diagnostic testing, and electronic
health records while taking into account the diversity of healthcare systems.

3. The Adaptive Character of Medical Conditions: Health issues and diseases frequently show
dynamic, changing patterns. The difficulty is in creating prediction models that can adjust
over time to take into account variables like new illnesses, changing medical practices, and
changes in the health of the population.

4. Privacy and Ethical Issues: Sensitive health data use gives rise to ethical and privacy issues.
Robust privacy-preserving measures and careful consideration are required to strike a balance
between the necessity of predictive analytics and the requirement to safeguard patient
confidentiality.

5. Limitations on Resources: insufficient resources, encompassing both manpower and

computing capacity, may impede the creation and implementation of advanced prediction
models. Affordability and accessibility are essential factors for broad adoption.

6. Verification and Extrapolation: One of the ongoing challenges in healthcare is ensuring that
prediction models can generalise to diverse patient groups and situations through rigorous
validation. For practical utility, predictions must be dependable in a variety of contexts.

1.3. Identification of Tasks

Finding, developing, and testing a method for predicting disease based on medical data
entails a number of activities that fall into three primary categories:
Identification Phase:

a. Problem Definition:
Define the specific disease or condition you want to predict.
Clearly outline the objectives of the prediction model.
 b. Data Collection:
Identify and collect relevant medical data sources (electronic health records, lab reports,
medical imaging, genetic data, etc.).
Ensure the data is representative of the population and contains features relevant to the
disease.

c. Data Preprocessing:
Cleanse the data to handle missing values, outliers, and inconsistencies.
Normalize or standardize numerical features.
Encode categorical variables.
Explore and understand the dataset through descriptive statistics and visualization.

d. Feature Selection:
Identify and select the most relevant features for disease prediction.
Consider domain knowledge and consult with medical professionals to validate feature
selection
Building Phase:

a. Model Selection:
Choose appropriate machine learning or statistical models for disease prediction (e.g.,
logistic regression, decision trees, neural networks).
Consider the interpretability, complexity, and scalability of the chosen model.

b. Training the Model:

Split the dataset into training and testing sets.
Train the model using the training set.
Adjust hyperparameters and evaluate model performance on the validation set.

c. Evaluation Metrics:
Define evaluation metrics suitable for disease prediction (e.g., sensitivity, specificity,
precision, recall, F1-score, area under the ROC curve).
Optimize the model based on these metrics.
 d. Validation:
Validate the model using an independent test dataset.
Address overfitting issues and fine-tune the model if necessary.

Testing Phase:

a. Real-world Testing:
Deploy the model to a real-world healthcare setting (e.g., hospital, clinic).
Monitor the model's performance and make necessary adjustments.
b. Ethical Considerations:

Consider ethical implications related to patient privacy, bias, and interpretability of the
model predictions.
Ensure compliance with relevant regulations and standards.

c. User Feedback and Iteration:

Gather feedback from healthcare professionals and end-users.
Iterate on the model based on feedback and performance in the real-world setting.

d. Documentation:
Document the entire process, including data sources, preprocessing steps, model
architecture, hyperparameters, and testing results.
Provide clear documentation for future maintenance and improvement.

1.4. Timeline
1.5. Organization of the Report

Chapter 1 Problem Identification: This chapter introduces the project and describes the
problem statement discussed earlier in the report.

Chapter 2 Literature Review: This chapter prevents review for various research papers
which help us to understand the problem in a better way. It also defines what has been
doneto already solve the problem and what can be further done.

Chapter 3 Design Flow/ Process: This chapter presents the need and significance of the
proposed work based on literature review. Proposed objectives and methodology are
explained. This presents the relevance of the problem. It also represents logical and
schematic plan to resolve the research problem.

Chapter 4 Result Analysis and Validation: This chapter explains various performance
parameters used in implementation. Experimental results are shown in this chapter.
Itexplains the meaning of the results and why they matter.

Chapter 5 Conclusion and Future Fcope: This chapter concludes the results and
explain the best method to perform this research to get the best results and define the
future scope of study that explains the extent to which the research area will be explored
in the work.
 CHAPTER 2
LITERATURE REVIEW/BACKGROUND STUDY

2.1. Timeline of the reported problem

1. Earlier Attempts (Prior to 2000s): Based on data disease prediction has long captured the
interest of medical scholars and practitioners. However, early attempts were constrained by the
processing capacity and data availability.

2. Machine learning's emergence in the 2000s: Advances in machine learning algorithms and the
growing availability of electronic health records (EHRs) have led researchers to investigate the
possibility of utilising this data for disease prediction.

3. Big Data's rise in the 2010s: An explosion of medical data resulted from the spread of wearable
technology, online health communities, and digital health technologies, offering better sources of
information for disease prediction models.

4. Obstacles and ethical Issues in the 2010s: Issues with data privacy, bias, and understanding
emerged as efforts to forecast diseases using medical data increased. The debate over the
application of predictive algorithms in healthcare increasingly included ethical issues.

5. Developments in Deep Learning throughout the 2010s: Deep learning methods, in particular
neural networks, have become well-known for their capacity to automatically extract features from
complex medical data, including sequences of genes and imaging scans, thus improving the
capacity to forecast disease.

6. Integration in the 2010s with Clinical Practice: Predictive models have the ability to help
doctors with early diagnosis, risk assessment, and customised treatment planning, as shown by a
number of studies. However, usability and trust issues arose when implementing these models in
clinical settings.

7. Research and Innovation to Continue (2010–2020): With continuous research aimed at

enhancing model accuracy, generalizability, and real-time monitoring capabilities, the field of
disease prediction using medical data is still developing. To solve the complex issues of predictive
analytics in healthcare, interdisciplinary teams consisting of physicians, data scientists, and
ethicists have been working together more and more.

8. Future Directions (2020s and Upward): Disease prediction from medical data has the potential
to lead to more proactive and customised healthcare interventions in the future. To fully achieve
this promise, however, united efforts are needed to overcome technical, ethical, and legal issues
while maintaining patient autonomy and fair access.

This timeline highlights significant achievements and difficulties faced during the process of providing a
general perspective of the described problem of disease prediction from medical data.
 2.2. Existing solutions

Existing solutions for disease prediction from medical data include:

1. IBM Watson Health: Predictive analytics and machine learning for healthcare are provided by
IBM Watson Health, which also offers disease prediction based on electronic health record data.
Watson can be used to help doctors find new areas of focus for drugs, create new treatments, and
gain a better knowledge of various illnesses.

2. Google Health: Makes use of EHRs and medical imaging data to perform disease prediction tasks
using deep learning algorithms. Google Health uses deep learning, a branch of machine learning,
to create prediction models for a range of illnesses. algorithms for deep learning, in particular
recurrent neural networks (RNNs) and convolutional neural networks (CNNs). X-rays, MRIs, CT
scans, and pathology images are among the medical imaging data types that Google Health
analyses using deep learning. Deep learning algorithms are able to automatically identify within
these photos patterns, abnormalities, and biomarkers suggestive of various diseases.

3. Prognos: A healthcare AI startup that leverages clinical data to identify high-risk patients and
forecast how a disease will proceed. Prognos Health is the most trusted managed real-world data
(RWD) marketplace, accelerating the development and delivery of innovative therapies and
improving health outcomes.

4. Tempus: Based on clinical and genetic data, Tempus uses data analytics and machine learning to
predict results for patients and define treatment for cancer. Our powerful sequencing technologies
are just one of the many kinds of data we're able to gather, organise, and combine to provide new
insights that will assist improve patient care.

5. Zebra Medical Vision: Creates algorithms from medical imaging data, such as CT and X-rays,
for the automatic identification and prediction of a variety of diseases. Using a variety of medical
imaging modalities, Zebra Medical Vision can identify and forecast a wide variety of illnesses and
medical disorders. This covers diseases like muscle injuries, respiratory diseases (such pneumonia
and lung cancer), fractures, tumours, and problems with the heart. The algorithms of Zebra
Medical Vision are especially made to interpret and analyse CT and X-ray pictures, which are
frequently used as diagnostic tools in clinical practice.

6. Owkin: Uses biomedical data to apply machine learning, especially for drug discovery and
oncology predictive modelling. Using biological data to find new drug targets, forecast drug safety
and efficacy, and streamline drug development processes, Owkin uses machine learning to speed
up the drug discovery process. Predictive modelling for cancer, which applies machine learning to
create models for cancer diagnosis, prognosis, and therapy response prediction, is one of Owkin's
main areas of interest. Using clinical and imaging data, Owkin combines multi-omics data
(genomics, transcriptomics, proteomics, etc.) to create complete predictive models that represent
the complexity and heterogeneity of cancer biology.
 2.3. Bibliometric Analysis

In a bibliometric analysis, trends, strengths, weaknesses, and research gaps would be identified by
examining the scientific literature with an emphasis on the salient characteristics, efficacy, and limitations
of illness prediction using medical data. This is how one could carry out such an analysis:

1. Gathering of Data: Using relevant search phrases relating to illness prediction, medical data,
machine learning, and artificial intelligence, compile pertinent publications from scholarly
databases like PubMed, Scopus, or Web of Science. Decide how long to limit the search to only
include recent papers (past 10 years, for example).

2. Preprocessing and Data Cleaning: Eliminate publications that are irrelevant or duplicates.
Author names, journal names, and keywords should all be uniform. Reject non-peer reviewed
publications and concentrate on academic writings.

3. Examination of Features: Determine the essential elements or traits of illness prediction by

utilising medical data as documented in the literature. Features could include the kinds of medical
data that are used (e.g., genetic data, electronic health records, and medical imaging), the
analytical methods that are used (e.g., deep learning models, machine learning algorithms), and
the applications of illness prediction (e.g., prognosis, early detection, therapy response).

4. Evaluation of Effectiveness: Assess the efficacy of disease prediction models by referring to

published results in the literature. Examine the prediction models' performance metrics in various
illness situations, including accuracy, sensitivity, specificity, and others. Evaluate how predictive
models can be used to improve patient outcomes, healthcare delivery, and resource allocation in a
clinical setting.

5. Identification of Drawbacks :Using medical data, identify typical limits and downsides related to
disease prediction. Issues with data quality, interoperability, privacy, model interpretability, and
clinical acceptance are possible drawbacks. Consider how these shortcomings affect the
predictability, generalizability, and practicality of predictive models.

6. The quantitative analysis method: To measure the distribution of publications across various
characteristics, effectiveness indicators, and downsides, use quantitative analysis. Compute
citation numbers, publishing trends over time, and networks of collaboration between scholars and
institutions.

7. Illustration: Provide significant findings from the bibliometric study in the form of network
graphs, bar charts, histograms, and heatmaps. To spot patterns and trends, visualise the
distribution of publications according to important characteristics, efficacy measurements, and
disadvantages.

8. Analysis and Conclusions: To get understanding of the present status of research on disease
prediction using medical data, interpret the bibliometric analysis results. Talk about the benefits,
drawbacks, possibilities, and dangers of predictive modelling techniques in the medical field.
Based on the study, make suggestions for future directions in research, methodological
enhancements, and clinical translation.
By conducting a comprehensive bibliometric analysis focusing on key features, effectiveness, and
drawbacks of disease prediction using medical data, researchers can gain valuable insights into the current
landscape of research in this field and identify opportunities for innovation and improvement.

2.4. Review Summary:

• Introduction: Our study focuses on disease prediction from medical data, and this review aims to
integrate insights from the literature with its goals and approach. The project's goal is to use a
variety of medical data sources to create reliable predictive models that will help with early
disease identification and prognosis.

• Findings from the Literature Review: A thorough analysis of the literature demonstrates a wealth
of studies demonstrating the value of predictive modelling methods for predicting diseases based
on medical data. Research demonstrates the effectiveness of deep learning and machine learning
algorithms on a variety of medical datasets, such as clinical notes, genomic data, electronic health
records (EHRs), and medical imaging. It has been shown that these approaches are effective in
forecasting conditions like cancer, heart disease, diabetes, and neurological illnesses.

• Alignment with study Objectives: By utilising predictive modelling techniques to forecast

diseases from medical data, our study corresponds closely to the conclusions of the literature
review. The literature emphasises the value of applying deep learning and machine learning
techniques, which we want to apply to our project. Furthermore, we choose suitable data
preprocessing techniques and assessment measures based on insights from the literature that are
specific to the goals of our project.

• Methodological Approach: Our project uses supervised learning, based on literature, and makes
use of machine learning methods including random forests, logistic regression, and neural
networks. We incorporate the finest data preprocessing techniques found in the literature, such as
feature selection, normalisation, and handling missing values. Area under the receiver operating
characteristic curve (AUC-ROC), sensitivity, specificity, and accuracy are examples of common
assessment metrics that will be used to evaluate the performance of the model.

• Project Outcomes and Implications: We hope that the predictive models we develop will be able
to identify people who are at risk of developing different diseases, which will allow for early
intervention and individualised treatment plans. These results highlight the significance of illness
prediction for enhancing healthcare outcomes and are in close agreement with the goals stated in
the literature. With implications for clinical practice and public health activities, our project has
the potential to further the field of predictive modelling for the prevention and management of
disease.

• Conclusion: We obtain important insights into current research trends and best practices in
disease prediction from medical data by combining the results of the literature review with our
project objectives. This summary highlights the value of evidence-based approaches in tackling
healthcare concerns and informs our methodological approach. Our study aims to improve patient
outcomes and healthcare delivery by adding to the expanding body of knowledge in predictive
modelling for illness prediction.
 2.5. Problem Definition

A important objective in medical data science is disease prediction, which determines who is most likely
to develop a particular health problem based on lifestyle characteristics, past medical history, and other
pertinent variables. The root of the issue lies in using machine learning and data mining methodologies to
examine huge amounts of diverse medical data and construct predictive models that possess the ability to
precisely predict the commencement or advancement of illnesses.

Problem statement:
Creating strong and trustworthy predictive models that can foretell a person's chance of contracting a
specific disease within a given timeframe is the main goal of disease prediction from medical data. This
means:

• Data Collection and Integration: Gathering different medical data sources, such as genetic
profiles, diagnostic tests, electronic health records (EHRs), lifestyle information, and
environmental factors, and combining them into an integrated dataset that can be analysed.

• Finding appropriate features or variables from the integrated medical data that are suggestive of
illness risk or progression is known as feature extraction and selection. In order to do this, useful
information must be extracted from raw data, and the most informative characteristics must be
chosen while reducing noise and redundancy.

• Model Development and Evaluation: To create predictive models, use machine learning
algorithms including ensemble methods, logistic regression, decision trees, support vector
machines, and neural networks. The models must to undergo training on past data and be assessed
by suitable performance metrics including F1-score, area under the receiver operating
characteristic curve (AUC-ROC), specificity, accuracy, and sensitivity.

• Interpretability and Explainability of the Models: Ensuring that stakeholders and physicians
can understand and comprehend the prediction models. This involves employing methods to
clarify the underlying elements influencing the predictions, such as feature importance ranking,
model visualisation, and rule extraction.

• Implementation and Verification: Implementing the created prediction models in actual clinical
contexts and verifying their accuracy using unobserved data. This involves carrying out
randomised controlled trials or prospective studies to evaluate the models' generalizability and
effectiveness in forecasting disease outcomes and helping in clinical decision-making.

Challenges:
Predicting diseases using medical data is a difficult undertaking that presents a number of difficulties,
including but not limited to:

• heterogeneity of data and problems with interoperability between various healthcare systems and
data sources.

• datasets that are unbalanced and have an unequal proportion of positive and negative examples,
which can skew model performance.
 • privacy and ethical issues with the handling and distribution of private medical data.

• Combining longitudinal data with temporal dynamics to capture the evolution of an illness over
time.

• Interpretability of complex machine learning models and the requirement for open and honest
clinical practice decision-making procedures.

Conclusion:
Collaboration between data scientists, policymakers, technological experts, and healthcare practitioners
across multiple disciplines is necessary to tackle these difficulties. Disease prediction from medical data
holds the potential to transform personalised medicine and preventive healthcare through the development
of novel approaches and the application of cutting-edge computer tools. This would eventually improve
patient outcomes and save money.

2.6. Objectives:

Objectives of disease prediction from medical data typically include:

1. Early Detection: By identifying people who may be at risk of a certain disease before symptoms
appear, therapy and intervention can begin earlier, perhaps leading to better patient outcomes and
lower medical expenses.

2. Preventive healthcare: The goal of disease prediction is to lower the rate and burden of diseases
within populations by facilitating the implementation of preventive measures including
vaccination campaigns, lifestyle changes, and focused screening.

3. Personalised medicine: By adjusting therapies and interventions according to each patient's

unique risk profile, which is obtained from medical data, healthcare delivery becomes more
accurate and efficient, resulting in better patient care and resource allocation.

4. Public Health Surveillance: By predicting disease outbreaks, identifying high-risk populations,

and providing guidance for strategic interventions to slow the spread of infectious diseases and
epidemics, disease prediction models support public health surveillance initiatives.

5. Allocation of Healthcare Resources: By anticipating the occurrence of diseases and the

healthcare requirements of patients, policymakers and healthcare professionals can effectively
allocate resources, guaranteeing fair access to healthcare services and streamlining healthcare
delivery systems.

6. Research and Development: By offering insights into the the beginning, course, and response to
treatment of diseases, disease prediction using medical data supports epidemiological research,
clinical trials, and the development of novel therapies.
 7. Healthcare Cost Reduction: Hospitalisation, emergency care, and long-term therapies can all be
made less expensive by proactively managing and avoiding diseases. This lowers overall costs for
patients, healthcare providers, and insurance companies.

8. Patient Empowerment: Giving people knowledge about their chances of contracting diseases
increases awareness about health issues, motivates preventive health practices, and promotes
patient participation in joint decision-making with medical professionals.

Overall, through the efficient application of data-driven insights and predictive analytics, the goals of
illness prediction from medical data are in line with boosting population health management methods,
improving health outcomes, and improving healthcare delivery.
 CHAPTER 3
DESIGN FLOW/PROCESS

3.1. Evaluation & Selection of Specifications/Features

Building an efficient and precise predictive model requires careful consideration of the features
and parameters for illness prediction using medical data. Here is a methodical way to assess and
choose these features/specifications:

• Acquiring Domain Knowledge:

• Recognise the illness or illnesses you hope to forecast.

• Learn about the factors and sources of pertinent medical data related to the condition or
diseases.
• Work together with medical specialists to get knowledge about features that are clinically
relevant.

• Methods for Choosing Features:

1. To find traits that have a strong link with the objective variable, use statistical techniques
including correlation analysis, t-tests, and ANOVA.
2. To shrink the feature space while keeping crucial information, use dimensionality reduction
methods like Principal Component Analysis (PCA) or feature importance methods like
Recursive Feature Elimination (RFE).

• Clinical Significance:

1. Give top priority to characteristics that are closely linked to the pathophysiology of the
disease or have established clinical importance.
2. Think about adding elements that physicians frequently utilise for prognosis and diagnosis.

• Engineering Features:

1. Create new features by drawing on subject expertise and medical professionals' insights.
2. Transform variables (e.g., standardise and normalise) to guarantee comparability and
consistency among features.

• Evaluation of Model Performance:

1. Make use of various feature combinations to train prediction models.

2. Use relevant metrics to assess the performance of the model, such as ROC AUC, F1-score,
accuracy, precision, recall, and calibration plots.
3. To evaluate the generalisation performance of the model and reduce overfitting, apply
cross-validation techniques.
• Iterative Procedure:

1. Repeat steps 3 through 6 and modify feature selection methods and criteria in response to
model performance and domain insights.
2. When combining several models with various feature subsets, take into account ensemble
approaches.

• External Testing and Validation:

1. To evaluate the finished model's capacity for generalisation, validate it on a different

dataset.
2. To make sure the model is resilient, take into account external validation utilising data
from other populations or sources.

• Moral Aspects to Take into Account:

1. Aim for impartiality and openness when choosing features to prevent prejudices,
particularly when it comes to socioeconomic or demographic aspects.
2. Handle sensitive medical data with privacy considerations and in accordance with laws
like GDPR and HIPAA.

Following these procedures will help you create trustworthy predictive models that can support
early diagnosis and individualised treatment planning by methodically evaluating and choosing
specifications/features for disease prediction using medical data.

3.2. Design Constraints

There are more factors to take into account than just technical ones when designing a system for
disease prediction utilising medical data. Below is a summary of the numerous limitations,
guidelines, rules, and other elements that must be taken into account:

• Adherence to Regulations:

1. adherence to laws including the General Data Protection Regulation (GDPR) and the
Health Insurance Portability and Accountability Act (HIPAA) to protect the security
and privacy of patient data.

2. meeting the requirements specified by regulatory agencies such as the Food and Drug
Administration (FDA) for the approval (if appropriate) of medical devices.

• Professional and Ethical Standards:

1. adherence to the moral standards set out by societies for professionals in medicine and
healthcare.

2. ensuring openness and informed permission while gathering, using, and sharing data.
• Safety and Health:

1. guaranteeing the security of medical personnel and patients during the gathering, handling,
and analysis of data.

2. putting strong cybersecurity measures in place to guard against breaches or unwanted

access to medical data.

• Effect on the Environment:

1. with an eye towards energy efficiency and a small environmental impact, the effects of data
processing, storage, and disposal techniques on the environment are taken into account.

• Producability:

1. scalability, interoperability, and ease of integration into the current healthcare infrastructure
are taken into account while designing systems and software.

2. choosing widely available hardware and software components that work with current
technology.

• Financial Elements:

1. evaluation of the suggested system's cost-effectiveness, taking into account variables such
the system's original construction costs, ongoing maintenance costs, and possible cost
savings from better disease control and prevention.

2. system's cost and accessibility to a range of patient demographics and healthcare

professionals.

• Political and Social Issues:

1. taking into account the effects on society, such as differences in access to technology and
healthcare, and making an effort at reducing these differences.

2. addressing issues with consent, equity, and data ownership in the provision of healthcare.

• Standardisation and Collaboration:

1. compliance with interoperability standards like FHIR and HL7 to enable the smooth
transfer of medical data across various platforms and systems in the healthcare industry.

2. Integration with various healthcare IT infrastructure and electronic health record (EHR)
systems to guarantee data continuity and compatibility.
 • Engaging Stakeholders:

1. In order to guarantee congruence with their needs and priorities, stakeholders such as
healthcare providers, patients, researchers, and policymakers should collaborate.

2. openness in the methods used to make decisions and in the dissemination of information
about the advantages and possible risks of the prediction model.

3.3. Analysis of Features and finalization subject to constraints

Let's examine characteristics for disease prediction utilising medical data in light of the previously
s specified constraints, and then refine them by adding, changing, and deleting features as needed:

• Take Features Out:

1. Eliminate any non-relevant demographic information, such as colour, ethnicity, or

socioeconomic status, as these could introduce prejudice and be unethical.

2. Unnecessary Personal Identifiers: To protect patient data privacy, remove details like
social security numbers or precise addresses.

3. Low-Quality or Redundant Data: To increase the interpretability and efficiency of the

model, eliminate features that have a large percentage of missing values, low variability,
or redundant information.

• Change the Features:

1. Time-Series Data Aggregation: To improve model interpretability and minimise

dimensionality, combine time-series data into pertinent summary statistics (such as mean,
median, and standard deviation).

2. Privacy of Sensitive Information: To protect patient privacy and maintain data utility,
modify features that contain sensitive information using privatisation techniques.

3. Normalisation of Numerical Features: To avoid particular features predominating and to

help in model convergence, scale numerical features to a common range.

• Include Features:

1. Clinical Biomarkers: To improve clinical relevance and predictive accuracy, incorporate

biomarkers linked to the disease(s) of interest that have undergone clinical validation.

2. Genetic Information: If available and morally acceptable, provide genetic markers or

genomic information as these can offer important insights into disease prognosis and
susceptibility.
 3. Environmental Factors: To identify holistic health determinants, including environmental
factors (such as pollution levels and climatic data) that may have an impact on the start or
course of disease.

4. Metrics for Healthcare Utilisation: To capture treatment compliance and healthcare-seeking

behaviour, incorporate features pertaining to healthcare utilisation patterns (e.g., number of
doctor visits, medication adherence).

• Finalisation Taking Limitations Into Account:

1. Make sure features holding personally identifiable information are removed or modified in
order to keep up with legal requirements (such as HIPAA and GDPR).

2. Give clinically relevant and ethically sound features first priority, and place a strong
emphasis on openness and equity in the feature selection process.

3. Choose characteristics that are easily accessible and compatible with the current healthcare
IT system to overcome manufacturability limits.

4. Analyse the financial viability of incorporating new features while taking prospective
expenses and advantages in terms of enhanced patient outcomes and predictive performance
into account.

5. Involve stakeholders in the feature selection process, such as patients and medical experts,
to make sure their requirements and preferences are met.

Developers can create a disease prediction model that strikes a compromise between regulatory c
o compliance, ethical issues, practicality in healthcare settings, and predicted accuracy by carefully
e v evaluating and deciding on characteristics while taking into account a variety of limitations.

3.4. Design flow

Design 1: Traditional Machine Learning Approach

• Gathering and Preparing Data:

1. Gather medical information about patients, such as their demographics, medical histories,
test results, imaging data, etc.
2. Handle missing values, normalise features, and encode categorical variables as part of the
preprocessing step of the data.

• Engineering and Feature Selection:

1. Using methods such as feature importance ranking, correlation analysis, or domain

knowledge, choose pertinent features.
2. If necessary, design new features like age groups or derived variable computation.
 • Model Choice:

1. Depending on the nature of the issue and the properties of the data, select suitable machine
learning models, such as logistic regression, random forests, or support vector machines.
2. To maximise model performance, apply strategies like hyperparameter adjustment and
cross-validation.

• Getting the Model Ready:

1. Using ensemble methods or gradient descent algorithms, train the chosen models on the
training dataset.

• Implementation:

1. Release the trained model to production, either as an independent programme or by

incorporating it into the healthcare infrastructure already in place.
2. Track the model's performance and add new data to it on a regular basis.

Design 2: Deep Learning Approach

• Gathering and Preparing Data:

1. Collect medical data that is both structured and unstructured, such as lab results,
demographics, and clinical notes and photos.
2. Tokenize text data, resize photos, and normalise numerical values to prepare data.

• Model Creation:

1. Create a deep learning architecture that is appropriate for the task, such as a
combination for multimodal data or a convolutional neural network (CNN) for picture
or sequence data, respectively.
2. If there are appropriate and readily available pre-trained models, make use of methods
such as transfer learning.

• Instruction:

1. Utilising strategies like stochastic gradient descent (SGD) or adaptive learning rate
methods, train the deep learning model on the prepared dataset.
2. To expand the training dataset and enhance model generalisation, apply data
augmentation to picture data.

• Verification and Adjusting Hyperparameters:

1. Adjust hyperparameters such as learning rate, batch size, and network architecture after
validating the model on a different validation set.
2. Use strategies such as early quitting to avoid overfitting.
 • Assessment:

1. Use relevant measures to assess the performance of the model, such as AUC-PR for binary
classification tasks or area under the receiver operating characteristic curve (AUC-ROC).
2. To determine how resilient the model is to changes in the input data, perform sensitivity
analysis.

• Integration and Deployment:

1. After the deep learning model has been trained, deploy it into a production setting, making
sure it is reliable and scalable.
2. Integrate the model for real-time disease prediction and decision assistance with the
current healthcare systems.

These two designs offer different approaches to disease prediction using medical data, with traditional
machine learning focusing on feature engineering and model selection, while the deep learning approach
leverages complex neural network architectures for automatic feature learning. The choice between these
designs depends on factors such as the availability of labeled data, computational resources, and the
complexity of the prediction task.

3.5. Design selection

after analyze and compare both the approach for the disease prediction using medical data , it can be said
that deep learning is better approach based on several factor as –

1. Deep Learning: Suitable for managing unstructured and high-dimensional data, such as text and
images. able to automatically derive from raw data hierarchical representations.
2. Deep learning uses multimillion-parameter neural network topologies. may use data to uncover
complex relationships and patterns without the need for explicit feature engineering.

3. In general, deep learning produces state-of-the-art results across a range of fields, particularly
when working with sizable and varied datasets. able to identify variations and complex patterns in
the data.

4. Deep learning frameworks frequently facilitate transfer learning, which is the process of
optimising pre-trained models built on massive datasets (like ImageNet) for particular tasks using
smaller medical datasets.

5. Because deep learning models are flexible in their architecture, researchers can tailor the models
to the particular needs of the disease prediction task. Furthermore, distributed training is made
feasible by deep learning frameworks like TensorFlow and PyTorch, which allows for the
effective use of parallel computing resources and the scaling of models to enormous datasets.

6. Their notable accomplishments in fields like personalised medicine, pathology prediction, and
cancer detection demonstrate their ability to enhance healthcare results.

7. Deep learning research is continuously evolving, with new architectures, algorithms, and
techniques being developed to address challenges in medical data analysis.
All things considered, deep learning provides a strong and adaptable framework for illness prediction,
especially when dealing with complicated and varied medical data. It's an attractive option for expanding
predictive analytics in healthcare because of its capacity to integrate multimodal information, recognise
complex patterns, and automatically learn from data.

3.6. Implementation plan/methodology

Fig3.1. Flowchart for disease prediction

 CHAPTER – 4

4.1. Implementing and testing a Disease Prediction From Medical Data

Involves several steps. Here are some general steps to consider:

1. ⁠Data gathering: Gather pertinent medical information from a range of sources, including
clinics, hospitals, and research databases. Patient demographics, medical histories, test
findings, imaging reports, genetic data, etc. could all be included in this data.

2. Data preprocessing:

• Data cleaning: Address outliers, inconsistent data, and missing values. Choose or extract
relevant traits (variables) that are most likely to be suggestive of the intended disease.
Statistical methods or domain expertise may be needed for this.

• Normalization/Standardization: To guarantee that each feature contributes equally to the

model, scale the data to a standard range.

• Data Splitting: Assign training, validation, and test sets to the dataset.

3. Model Development:
• Select a suitable statistical or machine learning model to forecast disease. Neural
networks, support vector machines (SVMs), decision trees, random forests, logistic
regression.
• Utilising the proper methods and algorithms, train the selected model on the training
set.Adjust hyperparameters to maximise model performance, frequently by employing
cross-validation methods.

4. Testing:
• Once satisfied with the model's performance on the validation set, test it on the
independent test set to assess its generalization ability.
• Evaluate the model's performance using the same metrics used during validation.

It's critical to make sure ethical issues—like patient privacy, data security, and fairness in
algorithmic decision-making—are taken into account at each stage. Including stakeholders, clinicians,
and domain experts in the process can also help confirm the model's effectiveness and guarantee its
usefulness in healthcare contexts.
4.1.1. Designing an Disease Prediction From Medical Data involves several
steps. Here are some general steps to consider:

It's critical to make sure ethical issues—like patient privacy, data security, and fairness in algorithmic
decision-making—are taken into account at each stage. Including stakeholders, clinicians, and domain
experts in the process can also help confirm the model's effectiveness and guarantee its usefulness in
healthcare contexts.

5. Data Collection:
• Gather medical information on the illness of interest, such as patient demographics, symptoms,
medical history, test results from labs, imaging data, etc.
• Split the dataset into training, validation, and test sets.

6. Model Selection:
• Choose logistic regression, decision trees, random forests, and SVMs as candidate models for
disease prediction.
• Understand the characteristics and assumptions of each model to make informed decisions during
the design process.

7. Model Development:
• implement logistic regression, decision trees, random forests, and SVMs using appropriate
libraries or frameworks (e.g., scikit-learn in Python).
• Train each model using the training dataset and tune hyperparameters if necessary.
• Consider using techniques like cross-validation to optimize model performance and prevent
overfitting.

8. Model Evaluation:
• Using the validation dataset, assess each model's performance.
• To evaluate the performance of the model, use evaluation measures like ROC-AUC, F1-
score, accuracy, precision, recall, and confusion matrix.
• To determine the best method for illness prediction, compare the results of several
models.
 9. Model Selection and Testing:
• Apply the top-performing model to the test dataset to assess its generalisation performance after
choosing it based on validation findings.
• Evaluate the model's predictive accuracy for disease outcomes based on hypothetical data.

10. Privacy and security:

• Implement the chosen model in an actual environment, such a healthcare application or
system.
• Keep an eye on the model's performance and make updates when needed to accommodate
modifications in clinical procedures or data distribution.
• Throughout the deployment process, make sure that all laws, moral principles, and data protection
requirements are followed.

By following these steps,we can design an effective disease prediction system using logistic regression,
decision trees, random forests, and support vector machines, leveraging the strengths of each model to
improve predictive accuracy and interpretability.

4.1.2. The result and testing of Disease Prediction From Medical Data can be
evaluated based on several criteria, including accuracy, reliability, usability,
and user satisfaction. Here are some general steps to consider for testing and
evaluating Disease Prediction From Medical Data:

1. Accuracy:
• the accuracy of the model by comparing its predictions of disease outcomes to the actual
results.Perform metrics calculations, including F1-score, recall, accuracy, precision, and area under
the ROC curve (ROC-AUC).
• Make that the model performs effectively across various data subsets and reaches a high accuracy.

2. Reliability:
• Assess the consistency and stability of the model's predictions over time and across different
datasets.
• Conduct sensitivity analysis to evaluate the robustness of the model to variations in input data and
model parameters.
• Validate the model's performance on independent datasets or through cross-validation techniques
to verify its reliability.
 3. Usability:
• Assess how well the illness prediction system integrates into current healthcare workflows and
apps, as well as how simple it is to use.
• Take into account elements like scalability, computing efficiency, and interoperability with
various software platforms and data formats.
• Make certain that the system generates forecasts that are easy to understand and apply for medical
experts.
4. User Satisfaction:
• Gather feedback from end-users, including healthcare providers, patients, and other stakeholders,
to assess their satisfaction with the prediction system.
• Conduct surveys, interviews, or usability testing sessions to understand user preferences, needs,
and concerns.
• Incorporate user feedback to improve the system's design, functionality, and performance to better
meet the needs of its intended users.
5. Clinical Utility:
• Assess the illness prediction system's clinical relevance and influence on patient outcomes and
healthcare decision-making.
• Analyse how well the system identifies people who are at risk, directs treatment choices, and
enhances patient outcomes.
• To evaluate the system's efficacy in actual clinical settings and its potential to lower healthcare
costs and improve resource allocation, conduct research or trials.

By evaluating disease prediction systems based on these criteria, we can ensure that they are accurate,
reliable, usable, and satisfying to their users, ultimately leading to improved healthcare outcomes and
decision-making.
Fig 4.1: GUI of Application(Bet
 CHAPTER – 5

5.1. Conclusion:

In this work, we used machine learning algorithms and medical data to estimate the likelihood of [insert
name of disease]. Several important conclusions and insights came from our investigation:

Expected Results/Outcome:
At first, we expected our prediction models to show a high degree of accuracy in detecting those who are
at risk of [insert name of disease]. Our models demonstrated excellent performance metrics, with an
overall accuracy of [insert percentage of accuracy], after thorough experimentation and validation. This
implies that the characteristics taken out of the patient information were useful in estimating the chance of
the illness.

Deviation from Expected Results:

Still, even though our prediction models performed well overall, there were some unexpected results. The
variation in prediction performance amongst the dataset's subgroups was one important finding. Some
clinical profiles or demographic categories showed less accurate predictions than expected. This suggests
that there might be restrictions on how broadly our models can be applied to different populations or
disease subgroups.

Reasons for Deviation:

The observed variations from our anticipated outcomes could be caused by a number of causes. First,
biases or noise may be introduced into our predictive models by the heterogeneity of the medical data,
which includes differences in data quality, completeness, and representativeness. Furthermore, our
research may not have fully reflected the intricacy of the underlying illness mechanism and its interplay
with other physiological parameters. Moreover, the predicted accuracy may also be impacted by the
intrinsic constraints of machine learning algorithms, such as overfitting or underfitting.

In summary, even though our study shows encouraging progress in the area of disease prediction from
medical data, more investigation is necessary to resolve the noted drawbacks and improve the reliability
and generalizability of predictive models. Further efforts to enhance predicted accuracy and clinical value
should concentrate on improving feature selection methods, adding more data sources, and using more
advanced modelling techniques.

5.2 Future Scope:

The integration of predictive modeling into healthcare indeed holds tremendous promise for transforming the
landscape of medicine. Here's a more detailed elaboration on the various aspects of this transformative potential:
Early Disease Identification and Prevention: Predictive models can analyze vast amounts of medical data to identify
patterns and markers that precede the onset of diseases. By detecting these indicators early on, healthcare providers
can intervene proactively, potentially preventing the development or progression of illnesses. This early
identification is particularly crucial for conditions like cancer, cardiovascular diseases, and diabetes, where early
intervention significantly improves outcomes. Individualized Interventions: Medical data, including genetic
information, lifestyle factors, and past medical history, can be leveraged to tailor interventions to individual
patients. Predictive models can analyze this data to recommend personalized treatment plans, medications, or
lifestyle modifications that are most likely to be effective for each patient. This approach moves away from the
traditional one-sizefits-all paradigm towards precision medicine, improving treatment efficacy and reducing
adverse effects. Enhanced Patient Outcomes: By providing timely interventions and personalized care, predictive
modeling has the potential to significantly improve patient outcomes. Patients may experience better symptom
management, reduced hospitalizations, and improved quality of life. Additionally, by preventing the onset of
diseases or managing chronic conditions more effectively, predictive modeling can reduce healthcare costs and
alleviate the burden on healthcare systems. Understanding Disease Causes and Risk Factors: Analyzing large-scale
medical data using predictive models can uncover previously unknown associations between various factors and
disease outcomes. Researchers can identify new risk factors, elucidate disease mechanisms, and discover potential
targets for intervention. This deeper understanding of disease pathology can inform the development of novel
treatments and preventive strategies, further improving patient care. Challenges and Collaboration: Despite its
potential benefits, the widespread adoption of predictive modeling in healthcare faces several challenges. These
include ensuring data privacy and security, addressing biases in data collection and algorithms, and navigating
regulatory and ethical considerations. Collaboration among stakeholders, including researchers, healthcare
providers, policymakers, and industry partners, is essential to overcome these obstacles and develop scalable, user-
friendly solutions that integrate seamlessly into existing healthcare systems. Future Directions: As technology
continues to advance, the sophistication of predictive models will increase, allowing for more accurate predictions
and personalized recommendations. Additionally, the integration of diverse data sources, such as wearable devices,
electronic health records, and genomic data, will further enhance the predictive capabilities of these models.
Continued research and innovation in this field hold the promise of revolutionizing healthcare delivery and
improving patient outcomes for years to come. In summary, predictive modeling has the potential to revolutionize
healthcare by enabling early disease identification, personalized interventions, and a deeper understanding of
disease processes. However, realizing this potential requires collaboration among various stakeholders to address
challenges and develop scalable solutions that prioritize patient safety, privacy, and efficacy.
 REFERENCES

1. P. Groves, B. Kayyali, D. Knott and S. van Kuiken, The‘Big Data’Revolution in Healthcare:

Accelerating Value and Innovation, 2016.

2. 2. M. Chen, S. Mao and Y. Liu, "Big data: A survey", Mobile Netw. Appl., vol. 19, pp. 171-
209, Apr. 2014.

3. P. B. Jensen, L. J. Jensen and S. Brunak, "Mining electronic health records: Towards better
research applications and clinical care", Nature Rev. Genet., vol. 13, no. 6, pp. 395-405, 2012.

4. W. Yin and H. Schutze, "Convolutional neural network for paraphrase identification", Proc.
HLT-NAACL, pp. 901-911, 2015.

5. N. Nori, H. Kashima, K. Yamashita, H. Ikai and Y. Imanaka, "Simultaneous modeling of

multiple diseases for mortality prediction in acute hospital care", Proc. 21th ACM SIGKDD Int.
Conf. Knowl. Discovery Data Mining, pp. 855-864, 2015.

6. S. Zhai, K.-H. Chang, R. Zhang and Z. M. Zhang, "Deepintent: Learning attentions for online
advertising with recurrent neural networks", Proc. 22nd ACM SIGKDD Int. Conf. Knowl.
Discovery Data Mining, pp. 1295-1304, 2016.

7. H. Chen, R. H. Chiang and V. C. Storey, "Business intelligence and analytics: From big data to
big impact", MIS Quart., vol. 36, no. 4, pp. 1165-1188, 2012.