Journal of Theoretical and Applied Information Technology

15th May 2020. Vol.98. No 09

© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

ANALYZING INFORMATION SYSTEM NEEDS IN

IMPLEMENTING INTERNAL QUALITY ASSURANCE
SYSTEM: A STUDY ON HIGHER VOCATIONAL
EDUCATION IN WEST SUMATERA, INDONESIA
1
YULHERNIWATI, 2JALIUS JAMA, 3GANEFRI, 4AIDIL IKHSAN
1
Department of Information Technology, Politeknik Negeri Padang, Padang, Indonesia
2
Department of Automotive Engineering, Universitas Negeri Padang, Padang, Indonesia
3
Department of Electrical Engineering, Universitas Negeri Padang, Padang, Indonesia
4
Department of Industrial Engineering, Universitas Bung Hatta, Padang, Indonesia
E-mail: [email protected], [email protected], [email protected],
4
[email protected]

ABSTRACT

The increasing demand of qualified skilled labor encouraged Indonesian government to improve the
number and quality of vocational education in Higher Education Institutions (HEIs), in the form of diploma
study programmes. However, the overall qualities of diploma study programs are still low, even though
quality assurance (QA) system has been an imperative to all HEIs. It can be seen from 1) the number of
accredited diploma study programs with “A” marks which is small (only 5.67%, nationally; and 3.85%
within West Sumatera Province) and 2) the accreditation marks have not continually improved year to year
as expected. Several studies about QA in Indonesia show that Internal Quality Assurance System (IQAS)
has not been implemented properly by most of HEIs. Some other literatures encouraged IS utilization to
support IQAS implementation, but have not indentified the specific requirement of IQAS information
system (IQAS-IS) for proper IQAS implementation. This study aimed to perform information system
needs analysis in IQAS implementation and IQAS-IS requirement specification. Surveys and interviews
were conducted to collect data about the problems in implementing IQAS, and the solutions purposed by
respondents. Then, the data analysis is conducted using fault tree analysis followed by five whys technique.
From this study, it is concluded that the root causes of the IQAS improper implementation are 1) lack of
leader’s commitment; and 2) lack of well designed information systems. Those two factors should be
developed simultaneously. This study gives contribution in IT research by identifying needs for IQAS-IS,
and its requirement specifications, they are namely: supporting IQAS implementation as an integrated
process management (a continous full cycle of processes that consist of IQAS planning, implementing,
evaluating, controlling and improving), enabling interoperability between different system, and facilitating
real time data processing and information retrieval. Thus, by proper IQAS implementation hopefully HEI’s
will gain quality improvements.
Keywords: Needs Analysis, Internal Quality Assurance System, Information System, Higher Vocational Education,
Interoperability, Integrated Process Management, Automated Data Collecting, Real Time Data
Processing

1. INTRODUCTION the overall qualities of diploma study programmes

are still low. Based on accreditation results, not
Indonesian government encouraged to improve many of them have gained ‘A’ remarks (5.67%,
the number and quality of vocational education in nationally; and 3.85% within West Sumatera
Higher Education Institutions (HEIs), in the form of Province) (processed from [2], [3]). The
diploma study programmes [1] to match the accreditation system had been utilized by
increasing demand of qualified skilled labor. Even Indonesian government to improve the quality of
though the quality assurance (QA) system has been higher education system since the beginning of 21st
an imperative for all higher education institution, century, however after almost twenty years being

1420
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

practiced, the quality hadn’t been being searching, using and exchanging information from
significantly improved. the internal and external environment among all
stakeholders in order to achieve the strategic goals
According to the regulatory policies set by the
of the higher education institutions, satisfaction of
government [4], Quality Assurance (QA) System
the needs of stakeholders and continuous
consists of 1) Internal Quality Assurance System
improvement of the quality assurance system [11].
(IQAS) that is QA conducted by HEIs; 2) External
A literature review concluded that ICTs utilization
Quality Assurance System (EQAS) as an
in IQAS is important in collecting and processing
assessment of HEIs’ IQAS implementation, held by
various data input [12]. The use of information
accreditation institutions, such as Badan Akreditasi
system in the implementation of QA process is
Nasional – Perguruan Tinggi (BAN-PT); and
concluded that it can improve the effectiveness and
verified by 3) HEIs Data Center or Pusat Data
efficiency of processes and support continuous
Perguruan Tinggi (PDPT), previously called
process improvement [13]. Kandil concluded that
EPSBED. Supposedly, IQAS has an
information system utilization for QA would reduce
implementation cycle that is in the form of plan, do,
a lot of fuss, time, and effort, and provide an
check and action (PDCA) against standards.
appropriate environment for the rapid exchange of
Several studies found improper implementation information and views about institution's
of IQAS in most of HEIs in Indonesia. According to performance [14].
Ikhfan Haris, IQAS has not fully implemented, Several studies that focused on designing or
compared to external QA (accreditation) activities developing information system support in HEIs’
[5]. Mohammad Faisal Amir noticed that IQAS implementation are presented in Table 1.
submission for BAN-PT accreditation does not Whereas in the field of external quality assurance
begin with the construction of internal QA. The system, there have been already many studies
study programmes were only busy preparing focused on utilizing information system or
documents prior to accreditation activities only, and information technology for accreditation, as in [15],
after that, there were no following-up QA activities [16], [17] and many more.
conducted [6]. Supposedly, the submission of BAN- Most studies on information system for IQAS
PT accreditation required study programmes’ long- only focusing on some part of the process
term planning. However, Ghafur found that self mangement such as the process of monitoring and
evaluation was conducted without planning the evaluation. They do not support the implementation
standards [7]. of quality assurance processes in a full and
continuous cycle as an integrated process
An alternative way that has reccommended to management. Most of the studies also have not
support the implementation of internal QA system identified the specific requirement for IQAS-IS to
(IQAS) is the utilization of information systems. support proper IQAS implementation.
Some benefits of information systems utilization in
Needs are the performance gap between the
organizational management have already current condition and the desired condition [18],
acknowledged, such as: 1) reducing costs; 2) whereas needs assesment is a systematic study of
reducing errors; 3) increasing the speed of activity,
problems or innovation, incorporating data from
4) improving planning and management control [8]. various sources in order to make effective decisions
According to Gasskov, vocational education and or recommendations about what should happen next
training (VET) institutions require information
[18]. A needs analysis is performed to analyze the
system to support effective governance and help gaps identified through needs assessment, as a root
reporting and accountability to external parties, and cause analysis for gaps [18]. Taken into account,
one of them is information system for QA. The QA
this study intends to continue the researches on
information system at VET institutions should improper IQAS implementation problem ([5], [6],
include QA standards and performance [7]), to identify the causes of the problem.
benchmarks, and enable tracking of the
Furthermore, while several studies of IQAS
achievement of standards and performance [9]. A information system (IQAS-IS) have been conducted
proper management information system can be a for the last decade as presented in Table 1, yet they
valuable tool in the integrated higher education
have effected on significant quality improvements.
management, that also serves the continuous Therefore, this study focuses on improper IQAS
improvement of quality assurance, budgeting, and implementation problem and analyzes the needs of
human resources management [10]. HEIs should information system that would be able to support
pay special attention to the establishment of IQAS implementation effectively. This study is
information systems for gathering, storing,

1421
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

limited to higher vocational education in West City - -

Sawahlunto
Sumatera, with respondents including coordinator City 4 1
of study programs and chairperson of QA system Solok
78 24
body in HEIs. This study does not cover the Total
accreditation process as it is part of the external
quality assurance system. 2.2 Method
This study consisted of three steps, namely 1)
The next section describes the material and
identifying needs; 2) analyzing causes and effects
method of the study. Section 3 reviews the results
relationships, 3) analyzing information system
of the study and discusses some analysis to identify
needs and make recommendations for IQAS
the information system needs, and finally, this
implementation successfullness and IQAS IS
article is concluded in section 4.
requirement specifications.
2. MATERIAL AND METHOD The needs defining process conducted by means
of interview and survey. This study was triggered
2.1 Material by some reviews found in literatures showing
Respondents participated in this study were improper IQAS implementation. Based on that, a
several higher education institutions (HEIs) structured interview was conducted with the
managing diploma study programmes located in chairperson of QA unit, about how IQAS had been
West Sumatera Province. The HEIS participated in implemented, what were the problems and the
this study, maintaining study programmes located alternative solutions. The interview results also
in several cities and districts of West Sumatera described how information system is utilized. Then,
province as described in Table 2. The respondents semi-closed questionnaire was arranged to obtain
were selected by using stratified sampling method, influencing factors. It was comprised of questions
so that the number of samples is obtained. HEIs regarding to problem and alternative solution in
samples were 24 by 78 and diploma study IQAS implementation along with the options, and it
programmes samples were 56 by 160. was also allowing the respondents to give
additional answers. This questionnaire was filled
Table 2:.Area Of Respondents
out by the coordinators of the study programs.
Number of HEIs The next process was conducted using Fault Tree
maintaining Diploma
Name of City /
Study Programs
Analysis (FTA). FTA was used to model events
Area District that causing the undesired events (T) which was
Population Sample
“Improper implementation of IQAS” and the cause-
Agam District 2 1
consequences relationship between those events.
Dharmasraya District - - Then, to find the root causes, Five Whys technique
Mentawai District - - was used. The last step was conducted based on
Island root cause analysis. Information system causal
Lima Puluh District 1 1 factors of QA system successfullness were
Kota analyzed and recommendations were made. Based
Padang District 2 1
Pariaman
on these, the information system’s specification
Pasaman District - - requirements are formulated.
Pasaman Barat District 1 1
3. RESULTS AND DISCUSSIONS
Pesisir Selatan District - - As this study issued, some new policies and
Solok Selatan District - - regulations regarding quality assurance system are
enforced, including the new accreditation
Solok District 3 1
instruments with 9 (nine) evaluation criterias.
Sijunjung District - - Although the surveys and interviews of this study
Tanah Datar District 2 1 are conducted while HEIs still implementing the
previous accreditation instruments with 7 (seven)
City 16 3
Bukittinggi standards of evaluation criteria, the results of this
City 40 10 study are still relevance. It is because the main
Padang
City 2 2 regulations applied are still the same, like the
Padang Panjang quality assurance system structures, cycle of
City 3 1 processes, etc. They differ mainly in the matter of
Pariaman
City 2 1 enhanced roles, evaluation criteria classification,
Payakumbuh

1422
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

outcome-based evaluation indicators, and documented as the results and the evidences of the
evaluation rubrics, while referring to the new 24 standards implementation. However, only 16,67%
(twenty four) national higher education standards. of HEIs participated in this study have implemented
3.1. IQAS Business Process most of the standards according to schedule and
As stated in the regulations, supposedly, have documented them, and 29,17% have
IQAS implementation cycle consists of processes, implemented a few standards according to schedule
namely IQAS planning (including standards and have documented them, and the rest of HEIs
determination or quality level setting), standards (54,17%) have not implemented standards
implementation, standards implementation according to schedule and have not documented
evaluating, standards implementation controlling, them.
and standards improving. IQAS implemention 3.1.4. Have the standards implementations
should involve various stakeholders and many kind been being evaluated on schedule and
of documentations through continous cycles of being documented?
sequential processes to gain continous quality Next, the standards implementation evaluation was
improvement. However, the interviews results conducted by measuring standards implementation
describe how HEIs have implemented IQAS in progress, measuring standards compliance, and
their regular basis, and it has found that in general, measuring results based on customer satisfaction
IQAS implementations are far from the ideal surveys. This process includes HEI’s internal audit.
concepts. Table 3 presents the interviews’ results All of this process should be documented.
related to IQAS business processes in HEIs. However, only 12,5% of HEIs participated in this
3.1.1. Have the standards been being study had evaluated most of the standards
determined? implementations on schedule and have documented
In IQAS planning, it is necessary to determine and them, whereas 33,33% have evaluated a few of the
document IQAS’ policies, standards, manuals and standards implementations on schedule and have
forms; of all aspects of HEI’s business processes, documented them. The rest of them ( 54,17%) have
both in HEI’s level and in study programme’s level. not yet evaluated standards on schedule and/or
37,50% of HEIs participated in this study have documented them.
standards and they have exeeded the minimum 3.1.5. Have the standards implementation
standards determined by government previously in been being controlled on schedule and
the form of derived standards or additional being documented?
standards such as international standards. 41,67% 12,5% of HEIs participated in this study had
have minimum standards ,whereas the rest of HEIs performed standards implementation controlling for
(20,83%) have not set any standards. most of standards implementations on schedule and
3.1.2. Have the determinated standards been had documented them, whereas 33,33% had
being followed by establishing evaluated a few of the standards implementations
additional documents (manuals / on schedule and have documented them. The rest of
standard operating procedures/ work them 54,17% have not yet performed standard
instructions, forms, implementing implementation control on schedule and have not
organization)? documented them.
20,83% of HEIs participated in this study have 3.1.6. Have the standards been being
been supported by additional documents for most improved periodically?
of the document’s types or standards, whereas 4,17% of HEIs participated in this study had
41,67% supported by additional documents for a improved most of their standards periodically or
few document’s types or standards.The rest of them achieved their increased target, whereas 41,67%
(37,50%) have not been supported any additional had improved a few of their standards or achieved
documents. their increased target and the rest of them 54,17%
3.1.3. Have the standards been being had not improved their standards.
implemented according to schedule and Thus, it can be concluded from the interviews
being documented? that in most of HEIs, IQAS has not being
Then, it continues with standards implementation. implemented as a full cycle of sequential processes.
Standards implementation related to performing all Although the standards are set, bad implementation
HEI’s business process activities regarding to the occured, there are no long term plannnings to
standards that have been set up. All the improve the standards from time to time. Therefore,
documentation of every activities of a particular quality asurrance system is going no where. There
standard that have been performed should be is no possitive effect in implementing IQAS. It

1423
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

indicates that there are some problems in 3.3. Solutions Suggested By Respondents
implementing IQAS. Thus, the interviews resulted The respondents suggest some opportunities in
in IQAS implementation need namely “all solving the problems. Figure 2 presents solutions
sequential processes of internal quality assurance sugested by respondents by selecting provided
are performed as a full cycle, continously”. answer, whereas Table 4 presents additional
3.2. Problems in Implementing IQAS suggestions from respondents’ own opinions. The
The surveys show that from the solutions recommended by more than half of
perspective of respondents there are several respondents are: 1) The addition of human
problems in implementing IQAS that may resources with the specified qualifications (76%);
contribute to improper IQAS implementation. The 2) Trainings (for management level: 74%, for staff
top five problems identified by respondents are 1) level: 58% ); 3) Better cooperation and
Inadequate number of supporting human resources collaboration with other units (68.%); 4) Facilities
(46%); 2) Not well managed documentation (46%); for better documenting, data processing and
3) Difficulties in gathering data from other units information retrieval (55%); 5) Representative
(44%). 4) Busyness with academic activities (43%); facilities and equipments (53%).
and 5) Data need to be processed first (30%). 3.4. Fault Tree Analysis (FTA) Results
Figure. 1. presented those results that derived from FTA is constructed by placing all possible causes
selecting the provided answers. It shows the of improper IQAS implementation problem
percentage of number of respondents who chose the identified by the surveys. They become the basic
option devided by total of respondents. Besides events that contributed to fault in FTA. They are
those answers, respondents can give additional grouped to some categories of events namely:
answers based on their own opinions regarding “Inadequate competent human resources”, “Lack of
problems in implementing IQAS, as presented in supporting facilities”, “Lack of leaders’
Table 4. commitment” and “Unavailable information”. The
Related to documentation issues, the surveys list of events of possible causes in FTA is presented
results show that respondents stated that in Table 6.
‘documentation is not well managed’ (46%), and 3.5. Root cause of Improper IQAS
they are also facing problems such as ‘inadequate implementation problem
facilities’ (26%) like representatives and dedicated The root causes are identified by further
rooms, filling cabinets for storing documents / analysis on FTA using Five Whys technique, to
archieves, etc. Whereas at some points in IQAS generate further causes to the problems and relate
implementation cycle, 22% respondents stated that these to each other. Table 7. presents examples of
they are facing the problem of ‘not knowing the finding the root cause by using Five Ways
current quality achievement’, and usually, to obtain technique.
that sort of information they find difficulties in Several events are namely: “Lack of
collecting data from other units (outside of their socialization about quality standard and policy”
study programmes) (46%). Then, after the data (E6), “Unclear and sudden information about
have been collected, usually they can’t immediately monitoring and evaluation schedules and activities
use the data without processing them (30%). (E7)”, “Different perception during monitoring and
However, there are also respondents stated about evaluation between auditees and auditors” (E8 )
‘not knowing the procedures’ (26%),‘busy with caused by “lack of sosialization”. Several events
academics activities’ (43%), ‘inadequate human are namely “Lack of discipline” (E4) and “Busy
resources’ (46%) or ‘lack of qualified human with academic activities” (E5) and lack of
resources’ (20%) that led to not conducting IQAS sosialization caused by the “lack of staff’s
processes properly. The two last processes in the commitment”. Several events are namely: “Not
cycle namely standards implementation controlling understanding the procedures (E3)”, “Inappropriate
and standards improving are suffered those qualification of human resources” (E1), and
problems faced by the previous processes. The “Inadequate number of human resources” (E2)
main problems in these two phases are “Inadequate fundings” (E12), “Inadequate
unavailability of accurate, timely, and adequate facilities” (E11), “Slow information distribution /
information about standards compliance and quality Lack of coordination” (E9), “Lack of supporting
achievements. It makes these two processes become policy and regulation” (E10) and lack of staff’s
not being conducted smoothly and effectively. commitment caused by the lack of leader’s
commitment. It is concluded that the root cause of
these events is the lack of leader’s commitment.

1424
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Several events are namely: “Not knowing 3. facilitating real time data processing and
current quality achievement” (E13), “Raw data information retrieval, involving :
needs processing” (E14), “Different data format - IQAS documents (policies, standards,
needs processing” (E15) caused by “Unhandled manuals, and forms) and other HEI’s
data processing“. Whereas events such as supported documents;
“Difficulties in gathering data from other units” - all IQAS processes’ documentations as
(E16), and “Not well managed documentation” IQAS implementation evidences.
(E17) caused by “Unintegrated data”. These events - information about standards
shows that information systems that has been used implementation progress, standards
(if any) are not well designed to support the IQAS compliance or quality achievement, and
implementation effectively. quality assurance results based on
The resulting fault tree (Figure 3.) shows that customer satisfaction surveys.
the root causes of the improper IQAS - information for IQAS implementation
implementation problem are: 1) lack of leader’s coordination purpose such as meetings,
commitment, and 2) lack of well designed task’s assignment, task’s schedules, and
information system support. task’s results
3.6. Needs of Information Systems in IQAS - valid and updated data of HEI’s data
implementation center in HEI’s level to be used in
The interview results also describe how HEIs accreditation purpose;
has utlized information system in IQAS These requirements should be fulfilled by
implementation. It is found that most of HEIs addressing some technical barrier. It is due to the
(95,83%) have already utilized information system technology architecture or system development
in managing their data and information for each of method which is not allowing easiness in
their business process separately, such as academic, integration and collaborations among different
library, finance, human resource, research and systems, platforms and formats.
comunitity services, etc, and only 4,17% have not
yet utilizing information system in managing their 4. CONCLUSIONS
business processes. But those information systems
are not integrated each other and/or with From the study, we learn that the need of IQAS
information system dedicated for IQAS (if any). implementation in HEIs is “performing IQAS in a
Therefore, the data collected in each of these ISs is full cycle of sequential processes, continously”.
only available for the HEIS’s unit/ study The root causes of this improper IQAS
programmes which are the user of the particular implementation problem is lack of leader’s
information system, without providing any efficient commitment, and lack of well designed information
mechanism for data sharing with other units / study system support. Information system requirment
programmes. Only a few number of HEIs (12,50%) specification for IQAS implementation are
have already utilized information system or characterized by the ability of supporting IQAS
applications dedicated for IQAS implementation, process in full cycle continously, enabling
but it is still limited to monitoring and evaluation interoperability between different technology, and
process. None of the HEIs has utilized information facilitating real time data processing and
system to support IQAS as a full cycle of sequential information retrieval.
processes. This study gives some recommendations for
The needs analysis using FTA followed by IQAS implementation successfullness. First, HEIs’
FiveWhy’s technique, shows the needs of well leaders should improve their commitment as it is
designed information system support for effective the key driver in IQAS implementation. Second,
IQAS implementation. In overall, the needed Quality asurance officials and IT managers in HEIs
information system supports should satisfiy several should sit together and discuss about detail
requirements, namely: information system requirements and contraints in
1. handling IQAS implementation as a supporting IQAS implementation. Third, as new
continous cycle consist of sequential policies and regulations regarding to quality
processes. assurance system enforced by government, this is
2. enabling interoperability between study the right time to redesign information system
programmes and other related units or according to the proposed characterististic in this
information systems within HEIs to improve study, while adapting to those new regulations.
data integration by efficient data collecting. Finally, along with IQAS-IS implementation, the

1425
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

corresponding regulations and controlling system "Profile of Study Program," Data Base for
should be enforced. Higher Education Ministry of Research,
This study has given contribution in IT research Technology and Higher Education. [On
by identifying and analyzing the needs of line]. Available: http://forlap.dikti.go.id.
information system utilization for more efficient [Accessed: 10-Feb-2018].
and effective internal quality assurance system. It [3] “Direktori Hasil Akreditasi Program Studi,”
also formulate requirement spesification for the BAN-PT. .
IQAS IS proposed. "Directory of Study Program Accreditation
The limitations of current knowledge are some Results," BAN-PT. [On line]. Available:
technical barriers that are not allowing easy https://banpt.or.id/dirirect/prodi/pencarian_
integration and collaborations among different prodi. [Accessed: 10-Feb-2018].
systems, platforms and formats. As this study has
[4] “Undang-Undang Republik Indonesia
encouraged to develop well designed information
Nomor 12 Tahun 2012 Tentang Pendidikan
system, therefore, the next study will adopt
Tinggi.” Republik Indonesia, 2012.
interoperability capabilities in the future IQAS-IS
design, to support data collecting automation "Law of the Republic of Indonesia, Act
among all related units in HEI despite of the Number 12 of 2012 concerning Higher
various technology that they have been used. It also Education." Republic of Indonesia, 2012.
allow real time data processing and information [5] I. Haris, “Assessment on the
retrieving. The proposed information system will be implementation of internal quality
modelled as an integrated IQAS-IS by using an assurance at higher education (an
enterprise architecture framework. In overall, the Indonesian report),” J. Educ. Instr. Stud.
importance of this study is as the entry point for World, vol. 3, no. 4, pp. 41–49, 2013.
other studies that will define objectives, design and [6] M. Faisal Amir, “Membangun Kesiapan
develop, demonstrate, and evaluate the information Sistem Penjaminan Mutu Eksternal (SPME)
system utilization solution for improper IQAS bagi Perguruan Tinggi Di Indonesia,” Jan-
implementation problems. 2010.
M. Faisal Amir, "Building an External
ACKNOWLEDGEMENT Quality Assurance System (SPME)
Readiness for Higher Education in
This study is supported by Ministry of
Indonesia,” Jan-2010.
Research, Technology and Higher Education of the
Republic of Indonesia through Doctoral [7] A. H. S. Ghafur, Manajemen Penjaminan
Dissertation Research Grant. We would like to Mutu Perguruan Tinggi di Indonesia, suatu
show our gratitude to all HEIs that participated in analisis kebijakan. Jakarta: Bumi Aksara,
this study. We also thank our family, friends, 2008.
colleagues and students for all the supports. A. H. S. Ghafur, Management of Higher
Education Quality Assurance in Indonesia,
a policy analysis. Jakarta: Bumi Aksara,
REFRENCES: 2008.
[1] “Soal Jumlah, Mutu dan Inovasi Pendidikan [8] S. Aswati, N. Mulyani, Y. Siagian, and A.
Tinggi Vokasi,” Kementrian Riset, Z. Syah, “Peranan Sistem Informasi Dalam
Teknologi dan Pendidikan Tinggi, Nov- Perguruan Tinggi,” J. Teknol. dan Sist. Inf.,
2016. . vol. 1, no. 2, pp. 79–86, 2015.
"The Problem of Quantity, Quality and S. Aswati, N. Mulyani, Y. Siagian, and A.
Innovation in Vocational Higher Z. Syah, “The Role of Information Systems
Education," Ministry of Research, in Higher Education,” J. Teknol. dan Sist.
Technology and Higher Education, Nov. Inf., vol. 1, no. 2, pp. 79–86, 2015.
2016. [On line]. Available: [9] V. Gasskov, Vacational Education and
http://kelembagaan.ristekdikti.go.id/index.p Training Institutions. 2004.
hp/2016/11/03/jumlah-dan-mutu- [10] J. Kettunen, “Integrated Management
pendidikan-tinggi-vokasi-akan-ditingkat/. Approaches in Higher Education,” US-
[Accessed: 03-03-2017]. China Educ. Rev. A, vol. 5, no. 11, pp. 707–
[2] “Profil Prodi,” Pangkalan Data Pendidikan 717, 2016.
Tinggi Kementerian Riset, Teknologi Dan [11] T. Č. Metzinger and D. Boras, “Information
Pendidikan Tinggi. .

1426
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Management in Quality Assurance System The Case of the TEI of Athens,” Int. J.
at Higher Education Institutions in Integr. Inf. Manag., vol. 02, pp. 41–46,
Croatia,” Recent Adv. Inf. Sci., pp. 277– 2015.
282, 2013. [21] T. C. Kahveci, Ö. Uygun, U. Yurtsever, and
[12] A. S. Haris, H. Washizaki, and Y. S. İlyas, “Quality Assurance in Higher
Fukazawa, “Utilization of ICTs in quality Education Institutions Using Strategic
assurance and accreditation of higher Information Systems,” Procedia - Soc.
education: Systematic literature review,” Behav. Sci., vol. 55, pp. 161–167, 2012.
Proc. 2017 IEEE Int. Conf. Teaching, [22] X. Jiang, “Reflection on Quality Assurance
Assess. Learn. Eng. TALE 2017, vol. 2018- System of Higher Vocational Education
Janua, pp. 354–359, 2018. under Big Data Era,” SHS Web Conf., vol.
[13] T. T. Hamza, A. Hassan, and M. A. 14, p. 01009, 2015.
Alhadidy, “Developing a Service Oriented [23] V. Cerverón-lleó, J. Cabotà-soro, F.
Process Management System for University Grimaldo-moreno, and R. Ferrís-castell,
Quality Assurance,” Int. J. Eng. Sci. “BPM for quality assurance systems in
Technol., vol. 2, no. 1, pp. 7–17, 2010. higher education,” J. Teach. Educ., vol. 03,
[14] M. S. Kandil, A. E. Hassan, A. S. Asem, no. 02, pp. 175–183, 2014.
and M. E. Ibrahim, “Prototype of Web2- [24] S. Alkhafaji, “Higher Education Institution
based system for Quality Assurance Quality Assurance Management System - ‘
Evaluation Process in Higher education Modeling and Design ,’” Excel Int. J.
Institutions,” Int. J. Electr. Comput. Sci., Multidiscip. Manag. Stud., vol. 2, no.
vol. 10, no. 02, pp. 1–8, Feb. 2010. September, 2015.
[15] A. I. Ahmed, “BIG DATA FOR [25] R. Mokhtar, A. A. Rahman, and S. H.
ACCREDITATION : A CASE STUDY OF Othman, “Towards Model Driven
SAUDI UNIVERSITIES,” J. Theor. Appl. Architecture in Academic Quality
Inf. Technol., vol. 91, no. 1, pp. 130–138, Assurance Information System
2016. Development,” 2016 3rd Int. Conf.
[16] E. Mohamed, M. Noura, D. Ashraf, and E. Commun. Comput. Eng., vol. 9, no. 1, pp.
H. Aboul, “Intelligent information system 1–10, 2016.
to ensure quality in higher education [26] A. Darmawan and M. S. Hasibuan,
institutions , towards an automated e- “Analisis Dan Perancangan Aplikasi Sistem
university,” Int. J. Comput. Intell. Stud., Informasi Audit Mutu Internal dan
vol. 6, pp. 115–148, 2017. Dokumentasi Penjaminan Mutu Perguruan
[17] M. B. Legowo, B. Indiarto, and D. Tinggi,” J. Generic, vol. 9, no. 2, pp. 342–
Prayitno, “Jurnal Penelitian Pos dan 347, 2014.
Informatika BAGI INSTITUSI A. Darmawan and M. S. Hasibuan,
PERGURUAN TINGGI "Analysis and Design of Internal Quality
BERSERTIFIKASI ISO MODELING OF Audit Information System Applications and
QUALITY ASSURANCE Higher Education Quality Assurance
INFORMATION SYSTEM FOR ISO- Documentation," J. Generic, vol. 9, no. 2,
CERTIFIED HIGHER EDUCATION pp. 342–347, 2014.
INSTITUTIONS Abstrak,” J. Penelit. Pos [27] Y. Nuraeni, “Perancangan Sistem Informasi
dan Inform., vol. 8, no. 2, pp. 155–168, Penjaminan Mutu Perguruan Tinggi Bidang
2018. Sumber Daya Manusia,” J. Sist. Inf., vol. 6,
[18] R. Watkins, M. Meiers, and Y. Visser, A no. 1, pp. 32–43, 2010.
Guide to Assessing Needs: Essential tools Y. Nuraeni, "Design of Higher Education
for collecting information, making Quality Assurance Information Systems in
decisions, and achieving development the Field of Human Resources," J. Sist. Inf.,
results. 2012. Vol. 6, no. 1, pp. 32–43, 2010.
[19] Z. Alzoabi, F. Diko, and M. Alnoukari, [28] E. Hariyanti and E. Purwanti, “Perancangan
Enhancing Education Quality Assurance Sistem Dashboard Untuk Monitoring
Using Data Mining, vol. 1. 2008. Indikator Kinerja Universitas,” Semin. Nas.
[20] A. Tsolakidis, M. Chalaris, and I. Chalaris, Sist. Inf. Indones., no. September, pp. 1–6,
“Quality Assurance Information System- 2014.

1427
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

E. Hariyanti and E. Purwanti, "Designing a Performance Dashboard Method," J.

Dashboard System for Monitoring Teknol. Inf. Din., Vol. 18, no. 1, pp. 1–8,
University Performance Indicators," Semin. 2013.
Nas. System. Inf. Indonesia, no. September, [32] Adelin and H. Effendi, “Aplikasi Audit
pp. 1–6, 2014. Mutu Akademik Internal dengan
[29] F. Fathya, N. Fatimah, A. Widjajarto, P. Pendekatan Extreme Programming,” J. TI
Studi, S. Informasi, and F. R. Industri, Atma Luhur, vol. 4, no. 1, pp. 13–24, 2017.
“Penerapan Business Intelligence Pada Adelin and H. Effendi, "Application of
Aplikasi Dashboard Monitoring Internal Academic Quality Audit with
Performansi Mahasiswa Dan Lulusan Extreme Programming Approach," J. TI
Berdasarkan Standar 3 Ban-Pt Program Atma Luhur, vol. 4, no. 1, pp. 13-24, 2017.
Studi Sarjana Menggunakan Metode [33] A. P. Atmaja and A. Azis, “Sistem
Scrum,” J. Rekayasa Sist. Ind., vol. 1, no. Informasi Terintegrasi Evaluasi Kegiatan
01, pp. 144–151, 2014. Mengajar Dosen Sebagai Implementasi
F. Fathya, N. Fatimah, A. Widjajarto, P. Sistem Penjaminan Mutu Internal,” J.
Studi, S. Informasi, and FR Industri, MATRIX, vol. 9, no. 1, 2019.
"Implementation of Business Intelligence in A. P. Atmaja and A. Azis, "Integrated
Student Performance Dashboard Information Systems Evaluate Lecturer
Monitoring and Graduate Applications Teaching Activities as Implementation of
Based on Standard 3 BAN-PT Internal Quality Assurance Systems," J.
Undergraduate Study Programs Using MATRIX, vol. 9, no. 1, 2019.
Methods Scrum, "J. System Engineering.
[34] V. Rindengan, A. Lumenta, and Y.
Ind., Vol. 1, no. 01, pp. 144–151, 2014.
Rindengan, “Rancang Bangun Aplikasi
[30] D. Rolliawati, A. Yusuf, and A. S. Audit Mutu Akademik Internal Universitas
Hamdani, “Desain Prototipe Sistem Sam Ratulangi Berbasis Web,” E-journal
Informasi Penjaminan Mutu Internal Tek. Inform., vol. 6, no. 1, pp. 1–6, 2015.
Berbasis Standar Nasional Pendidikan
V. Rindengan, A. Lumenta, and Y.
Tinggi,” in Seminar Nasional Inovasi dan
Rindengan, "Designing the Application of
Aplikasi Teknologi di Industri 2018, 2018,
Web-Based Internal Academic Quality
pp. 167–173.
Audit for Sam Ratulangi University," Tech
D. Rolliawati, A. Yusuf, and A. S. E-journal. Inform., Vol. 6, no. 1, pp. 1–6,
Hamdani, "Prototype Design of Internal 2015.
Quality Assurance Information Systems
[35] M. Safi’i and Vidy, “Perancangan Sistem
Based on National Higher Education
Informasi Badan Penjaminan Mutu,” Metik,
Standards," in the National Seminar on
vol. 1, no. 2, pp. 1–7, 2017.
Innovation and Application of Technology
in Industry 2018, 2018, pp. 167–173. M. Safi'i and Vidy, "Designing a Quality
Assurance Agency Information System,"
[31] A. P. Utomo, H. Murti, and R. S. Rejeki,
Metik, vol. 1, no. 2, pp. 1–7, 2017.
“Sistem Monitoring dan Evaluasi Kinerja
Program Studi dengan Metode Performance .
Dashboard,” J. Teknol. Inf. Din., vol. 18,
no. 1, pp. 1–8, 2013.
A. P. Utomo, H. Murti, and R. S. Rejeki,
"Monitoring and Evaluation System for
Study Program Performance with the

1428
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Table 1: Research Map of Information System Support Development for IQAS Implementation in Higher Education
Research Main idea
[13] Develop an information system to automate process management of quality assurance in
university which is related to modeling, design, execution, and monitoring activities. The
approach used is service oriented architecture (SOA).
[14] Develop an information system prototype for process evaluation of quality assurance in a
higher education institution, which focuses on self-evaluation stage and external evaluation
stage.
[19] Build an automated system for quality assurance in higher education using data mining.
[20] Propose a framework for the quality assurance of a higher educational institute including all
the relevant roles and responsibilities of the administration hierarchy, It is based on a four
layers architecture, which consists of data collection, data mining, decision support, and the
monitoring of the KPI’s.
[21] Propose a quality assurance framework supported by strategic information system. It
integrates strategic management, process management and monitoring measuring
management system as well as take into consideration international, national, regional,
external factors.
[22] Construct of data management platform for quality assurance system using big data
technology.
[23] Develop and evaluate a computer platform using Business Process Management (BPM)
tools in Quality Assurance System implementation in a university. It proved efficient. It
concluded that Quality Assurance Systems, and the overall functioning of organizations,
may be supported by Business Process Management (BPM) software systems, in which: a
workflow engine should be used; process development should be documented with
evidences that should be stored and organized in a structured content repository. The users’
experiences prove that a computer supported quality system based on BPM facilitates the
compliance of quality processes and allow further improvement of the whole quality system
and its processes.
[24] Identify the general requirements of a quality assurance management system data
management tool, develop user interface and a model for HEIs. It also provides different
user levels to use the quality assurance management system to maintain confidentiality and
creditability.
[16] Develop a framework for an intelligent information system that manages the quality
assurance in higher education’s institutions, in oder to provide an automation tool that
avoids unnecessary and redundant tasks associated to quality in higher education
institutions. It helps all higher education stockholders to handle and monitor their tasks, and
to apply their quality’s standards and to make sure that they are being maintained and
enhanced.
[25] Develop a metamodel using Model Driven Architecture (MDA) in (AQA) that can be
configured to implement academic quality assurance application software based on
information system (IS) platform.
[26] Develop a web-based information system to facilitate internal quality audits based on the
ISO 9001: 2008 clauses and national education standards by providing information,
guidelines, assessment criteria, answers, reports, responses and monitoring of findings
[27] Develop human resource quality assurance information systems in higher education
institutions.
[28] Design a dashboard system for performance monitoring and evaluation based on key
performance indicators (KPI) of the university.
[29] Develop dashboards to support monitoring activities of students and graduates by each study
program using business intelligence and also provide predictive information to support
decision making for study programmes’ top level management.
[30] Develop a web-based internal quality assurance information system software prototype
according to the latest national higher education standards.
[31] Develop study programme dashboard performance to provide information to study
programme managers, faculties and universities on the quality achievements of study

1429
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

programmes, covering all criteria in the accreditation standards for undergraduate

programmes
[32] Build an information system for Internal Academic Quality Audit that can integrate data
owned by clients with academic quality documents needed for academic audit needs.
[33] Develop an online internal quality assurance information system that allows evaluation of
the teaching and learning process of lecturers, and management of evaluation results as part
of the IQAS activity cycle at the end of each semester.
[34] Develop information systems to facilitate the implementation of internal academic quality
audits so that document checking and data processing become faster and easier to do.
[35] Develop prototype of web responsive quality assurance information system that provide
services in the form of monitoring and evaluation supporting document management.

Table 3. Interview’s Results Related To IQAS Business Processes

Answers

Questions Yes
Yes
(beyond
(minimum / No
minimum
a few of)
/most of)
Have the standards been being determined? 37,50% 41,67% 20,83%
Have the determinated standards been being followed by the 20,83% 41,67% 37,50%
establishing additional documents (manuals, standard
operating procedures, work instructions, forms, implementing
organization)?
Have the standards been being implemented according to 16,67% 29,17% 54,17%
schedule and being documented?
Have the standards implementations been being evaluated on 12,50% 33,33% 54,17%
schedule and being documented?
Have the standards implementation been being controlled on 12,50% 33,33% 54,17%
schedule and being documented?
Have the standards been being improved periodically? 4,17% 41,67% 54,17%

1430
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Figure 1: Survey Results About Problems In Implementing Iqas (By Selecting Provided Answer)

Table 4. Additional answer (respondent’s own opinion) regarding problems in implementing IQAS
Additional answer (respondent’s own opinion) Percentage
Lack of leader’s commitment for financing 2%
Lack of leader’s commitment for infrastructure 2%
Lack of socialization about quality standards 2%
Lack of socialization about quality policy 2%
Lack of socialization about monitoring and evaluation activities 2%
Lack of information about standards implementations because data has not yet processed 2%
Supporting policy, regulations and guidelines are still weak 4%
Slow information distribution / lack of coordination 2%
Lack of socialization about monitoring and evaluation schedules 4%
Lack of lecturer’s discipline in administration 2%
Different perceptions about audit in monitoring and evaluation between auditee and 2%
auditor

1431
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Figure 2. Suggested solutions to improper IQAS implementation problems (by selecting provided answer)

Table 5. Additional suggestions for IQAS implementation problems

Solutions Percentage
Standardizing formats of data and document 2%
Allocating time, budgets and efforts for data processing 2%
Online and integrated data 2%
Provide easiness for study programmes to get data / information directly and fast 2%
Provide online IQAS documents for all study programmes/units 2%
Commitment in conducting IQAS 2%
Involvement of all academic community in conducting IQAS 4%
Commitment and support from leaders in conducting IQAS 4%
Socialization of standard operating procedures and rules in IQAS implementation 2%
Maximizing the work of IQAS unit staffs in study programmes 2%
Socialization and training for IQAS unit staffs in study programmes 2%
Routine meetings for IQAS from study programme level to HEI level 2%

1432
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Table 6. Events of improper IQAS implementation problem

Symbol Description
T Improper IQAS implementation
G1 Incompetent human resources
G2 Lack of leader’s encouragement
G3 Lack of supporting facilities
G4 Unavailable information
E1 Inappropriate qualification of human resources
E2 Inadequate number of human resources
E3 Not understanding the procedures
E4 Lack of discipline
E5 Busy with academic activities
E6 Lack of socialization about quality standard and policy
Unclear and sudden information about monitoring and evaluation schedules and
E7
activities
E8 Different perception during monitoring and evaluation between auditees and auditors
E9 Slow information distribution / Lack of coordination
E10 Lack of supporting policy and regulation
E11 Inadequate facilities
E12 Inadequate fundings
E13 Not knowing current quality achievement
E14 Different data format needs processing
E15 Raw data needs processing
E16 Difficulties in gathering data from other units
E17 Not well-managed documentation

Table 7. Example of finding the root cause by using Five Whys technique
Event as Possible Cause 5 Whys
E7 Unclear or sudden Why?: Lack of socialization
information about Why?: Changing schedule
monitoring and Why?: Bad planning
evaluation schedules Why?: Lack of staff’s commitment
Why?: Lack of leader’s commitment
E16 Difficulties in Why?: Unintegrated data that are spread across units within HEI
gathering data from Why?: Diversity and incompatibility of technology,
other units Why?: System is built separately,
Why?: System development method does not consider
collaborations
Why?: Lack of well designed information system

1433
 Journal of Theoretical and Applied Information Technology
15th May 2020. Vol.98. No 09
© 2005 – ongoing JATIT & LLS

ISSN: 1992-8645 www.jatit.org E-ISSN: 1817-3195

Figure 3: Fault Tree Analysis followed by FiveWhys technique of Improper IQAS implementation problem

1434