Follow-up of the

implementation of audit
recommendations
Best practices guide, issued by the project group

January 2021
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 2

Quality and Transparency Statement of the leader of the EUROSAI Project Group on
“Follow-up of the implementation of audit recommendations”

This is to certify that the best practices guide in the follow-up of the implementation of audit
recommendations has been developed following the Quality and Transparency process stated in
the “QUALITY AND TRANSPARENCY PROTOCOL FOR EUROSAI PRODUCTS AND
DOCUMENTS ” as detailed below:

i. Representation of the membership of the PG elaborating the product:

 SAI Belgium (lead): Mr. Steven Bernagie
 SAI Germany: Mr. Jens Rößler
 SAI Lithuania: Ms. Lina Venckūnaitė-Barauskienė
 SAI The Netherlands: Mr. Geert Jan Mol
 SAI Spain: Ms. Beatriz Sanchez Almendros

The best practices guide was drawn up and approved by mutual agreement between the
PG members.

An open invitation to join the PG was made during the second EUROSAI Strategic Goal
1 meeting in Tirana (October 30-31 2018). All EUROSAI members were welcome to join
as respondents to the survey.

There were no external stakeholders involved in this PG.

ii. Terms of Reference: approved on November 27 2018

iii. Openness and transparency: progress report at third EUROSAI Strategic Goal 1 meeting
in Liberec (October 9-10 2019) and at fourth EUROSAI Strategic Goal 1 e-meeting
(December 3 2020).
The best practices guide in the follow-up of the implementation of audit
recommendations will be sent by e-mail to all EUROSAI members, and made available
to EUROSAI community and external stakeholders on the EUROSAI website (database
of products) and EUROSAI OP website.
iv. Work method: information in report gathered through a survey, sent on 18 July 2019 to
all 50 EUROSAI members
v. Exposure to comments: draft version of the best practices guide has been submitted for
peer review within the PG members’ SAIs. Results of peer reviews were incorporated in
the best practices guide by mutual agreement between the PG members.

Philippe Roland
Senior President

February 2021
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 3

SUMMARY

1 Introduction 4
1.1 EUROSAI Project 4
1.2 Survey 4

2. Good practices that influence the implementation of audit recommendations 5

2.1 Impact and follow-up of the implementation of recommendations 5
2.2. Factors that influence the impact of SAIs 6
2.3. Good practices in/for the implementation of audit recommendations 7

3. Results of the survey 9

3.1. Audit report quality 9
3.2. Constructive relationship between auditor and auditee 12
3.3. Existence of follow-up system 17
3.4. Parliamentary involvement 22
3.5. Report the results of the follow-up system 24
3.6. Use of follow-up results for performance monitoring and risk assessment 27

4. Closing remarks 30

Annex 1: survey 31

Annex 2: survey answers 39

 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 4

1 Introduction

1.1 EUROSAI Project

The SAIs of Belgium (lead), Germany, Lithuania, The Netherlands and Spain launched the project ‘Follow-
up of the implementation of audit recommendations’ in January 2019. The project supports the
implementation of Strategic Goal 1 ‘Supporting effective, innovative and relevant audits by promoting and
brokering professional cooperation’ and especially Objective 1.3 ‘To facilitate the sharing of knowledge
and experience within EUROSAI and with external stakeholders and partners’ of the EUROSAI Strategic
Plan 2017-2023.

The objective of the project group was to gather information through a survey about the various systems
of follow-up of the implementation of audit recommendations by EUROSAI members.

1.2 Survey

The information on the follow-up systems applied by the EUROSAI members was gathered through a
survey, which was sent on 18 July 2019 to all 50 EUROSAI members (the SAIs of 49 European states and
the European Court of Auditors - ECA).

As follow-up systems can differ in the ways recommendations are made, the first part of the survey
includes questions on the nature of the recommendations formulated by the SAIs. The main part of the
survey deals with the different ways in which follow-up can be conducted. Finally, the last part of the
survey investigates if SAIs make global reports of aggregated follow-up results, and includes questions
about the use of follow-up results for performance monitoring or risk assessment.

The survey is shown in annex 1. Annex 2 presents the survey answers.

33 SAIs, including ECA, have replied to the survey. We would like to explicitly thank them for their
cooperation.

Respondents of the survey

This report describes the results of the survey and derives several good practices on follow-up procedures
from scientific research complemented by interesting case examples from the SAIs.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 5

2. Good practices that influence the implementation of audit

recommendations

2.1 Impact and follow-up of the implementation of recommendations

Impact of SAIs can be defined as the effect or influence of an audit or SAI on the auditees, the government,
parliament or society. Or, to put it another way, impact is the change resulting from an audit i.

According to the audit principles defined in INTOSAI-P 12 The Value and Benefits of SAIs – making a
difference to the lives of citizens, SAIs need to demonstrate their ongoing relevance to citizens, parliament
and other stakeholders. In this respect, SAIs should evaluate the changes resulting from their audit
activities.

Evaluating impact can be done in various ways. SAIs might for example
 evaluate their perception through surveys amongst the auditees, legislature or society;
 keep statistics on the number of media articles related to audit reports; or
 estimate the financial impact generated by the audit (savings or additional revenues) on the
audited organisation or on society.

This variety is also related to different kinds of impact, such as conceptual impact, which refers to changes
in learning processes, mentality and behaviour at the auditee or society; strategic impact, which refers to
the use of audit reports by stakeholders in political or public debates; and instrumental impact, which is
a more direct impact by implementing the audit recommendationsii.

Indeed, as most of the audit reports connect their findings with suggestions for solving the problems,
these recommendations are an appropriate instrument for measuring the impact resulting from the audit.
Although this instrument could possibly be used to evaluate other kinds of impact, such as conceptual
impact1, most SAIs will probably use the implementation of recommendations to measure instrumental
impact. Even more, as the implementation of recommendations is easier to measure than other impact
indicators, most of the SAIs might limit their impact measurement to this method. Therefore, the
INTOSAI Capacity Building Committee (CBC) guide How to increase the use and impact of audit reports
focuses on the follow-up of the implementation process to increase the impact of audits. Besides
evaluating the SAI’s performance and making audit reports more effective, follow-up also assists
government and legislature and creates incentives for learning and development.

Even this limitation to evaluating the implementation of audit recommendations can result in different
practices amongst SAIs:
 The follow-up exercise varies depending on how information about the implementation was
obtained: either it relies on data reported by the auditee, or the data is briefly verified by the
auditor, or the auditor performs a thorough check-up carrying out a follow-up audit.
 Likewise, the timing and frequency of the follow-up can vary considerably among SAIs, and the
same goes for the way SAIs report the follow-up results.
 A quantitative approach, that usually consists of displaying the percentage of implemented (or
accepted) recommendations, is very different from a qualitative approach, where the information
on the audit implementation is presented in a descriptive way.

The main aim of the survey was to collect the various systems of follow-up of implementation of audit
recommendations by EUROSAI members. By asking questions which cover the process chronologically
starting from the drafting of the recommendations until the use of the follow-up results in the
performance monitoring and risk assessment processes, the survey also tried to identify good practices in
this process.

1 In the case that recommendations are picked up by other organisations than the auditee.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 6

2.2. Factors that influence the impact of SAIs

Many scientific researchers have investigated the factors that influence the impact of audit reports iii, or
rather influence the implementation of audit recommendations.

Factors that are relevant in the context of the survey questions are:
 Constructive relationship between auditor and auditee;
 Audit report quality;
 Existence of a follow-up mechanism;
 Parliamentary involvement;
 Dissemination of the follow-up report.

Relationship between auditor and auditee

It is clear that a good relationship between auditor and auditee 2 increases the support base for
implementing the recommendations. The survey questions on the discussion or the acceptance of the
recommendations between the auditor and the auditee can be placed in this context. This also applies to
the questions on the auditee’s drafting of an action plan that specifies the measures to implement the
recommendations. The same goes for some practices mentioned in the survey answers where auditees
have been engaged in the follow-up mechanism, for example by filling in a database with actions to
implement the recommendations.

Audit report quality

A high-quality report that clearly shows and underpins the findings and recommendations will stimulate
the report’s impact. This quality is strongly influenced by the quality of the formulated recommendations,
which is dealt with in the first part of the survey.

Existence of a follow-up mechanism

It is also worth noting that the presence of a follow-up mechanism itself can be seen as a factor that
influences the impact of audits. Following up the implementation of the audit recommendations,
repeatedly if necessary, may put pressure on the auditees to act in a timely manner, and thus might
increase the audit impact. Questions on the existence of a follow-up mechanism, including the procedure,
the scope and the methods of measurement, form the most important part of the survey.

Parliamentary involvement
Likewise, the involvement of parliament in the follow-up process of SAIs, for example by holding a
parliamentary debate on the issue, may increase the pressure on the auditee. One survey question asks if
parliament plays a role in the implementation of recommendations.

Dissemination of the follow-up report

Scientific research also mentions another crucial factor for raising the impact of audit reports, which are
the efforts to disseminate the report to a wide audience (parliament, citizens, media, civil society). In our
opinion, this argumentation holds also for the communication of the follow-up results. Indeed, also ISSAI
300 Performance Audit Principles states that ‘follow-up should be reported appropriately in order to
provide feedback to the legislature’ (art. 42).

Reporting the follow-up results, preferably in a readable and clear way for a broad audience, may again
raise the pressure on the auditee in case of insufficient implementation. On the other hand, publishing
positive follow-up results rewards the well-performing auditees and might strengthen the good
relationship between auditor and auditee.

Several survey questions cover the reporting of the follow-up of individual audits, as well the reporting of
the aggregated follow-up results of all audits during a specific period.

2 Where we mention the relationship of the auditee with the auditor, we also refer to the relationship with the SAI in general.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 7

2.3. Good practices in/for the implementation of audit recommendations

This report presents several examples of good practices on follow-up systems.

In this context, a good practice could be defined as a procedure (legal or other), method or other practice
that can contribute to the appropriate and timely implementation of audit recommendations issued by
the SAI. However, since the survey answers do not give any information on the actual results of the
practices and procedures as mentioned, we can rely on the scientific research on this question.

From this point of view, the influencing factors on impact, as mentioned above, can be reformulated in a
more detailed manner as a chronological listing of good practices throughout the audit process. We also
formulated a last good practice on the use of the follow-up results for the performance monitoring system
and the risk assessment of the SAI. Indeed, as the follow-up of the implementation of recommendations
is a practical method for evaluating the impact of the SAI’s activity, the results of this follow-up should be
used as an indicator for the SAI’s performance. Furthermore, follow-up results can be used as an input for
the risk assessment of the SAI, whether on a strategic or operational level.

Good practices and corresponding factors that influence impact

Factors that influence the impact of Good practice

SAIs

Audit report quality (chapter 3.1)  Write relevant, operational and targeted
recommendations.
 Classify the recommendations in order of
importance.

Constructive relationship between  Seek acceptance of the recommendations by

auditor and auditee (chapter 3.2) the auditee, thereby increasing the support
base.
 Engage the auditee or the government in the
follow-up of the implementation of
recommendations.
 Ask the auditee for an action plan that specifies
the measures to implement the
recommendations, including deadlines.
 Check the appropriateness of the action plan.
 In the absence of an action plan, set clear and
realistic deadlines for the implementation of
the recommendations, if possible in agreement
with the auditee.

Existence of follow-up system  Set clear and realistic deadlines for the
(chapter 3.3) implementation of recommendations.
 Provide for an effective recommendation
monitoring and follow-up system that checks
the implementation in a timely manner.
 Repeat this follow-up, or even consider a
follow-up audit, in case there is no adequate
implementation or insufficient information
about the implementation.

Parliamentary involvement  Engage parliament in the follow-up of the

(chapter 3.4) implementation of recommendations.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 8

Factors that influence the impact of Good practice

SAIs

Report the results of the follow-up  Report the results of the follow-up system,
system (chapter 3.5) preferably in an aggregated way by publishing
a global report or database on the follow-up
results during a specific period.

Use of the follow-up results for the  Use the results of the follow-up system for risk
performance monitoring system and assessment or a performance monitoring
the risk assessment (chapter 3.6) system.

The next chapter analyses the results of the survey for each of these factors and good practices, and shows
some interesting examples provided by the respondents. In doing so, we sought to take into account the
different legal mandates and institutional environments of the SAIs, by selecting examples that can be
picked up by most EUROSAI members. Some of the information supplied in the survey answers was
complemented by asking the SAIs concerned for additional information, or by checking documentation
available on the respondents’ websites.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 9

3. Results of the survey

3.1. Audit report quality

Write relevant, operational and targeted recommendations. Where appropriate, classify the
recommendations in order of importance.

The formulation of recommendations is one of the key tasks in the audit process, as high-quality
recommendations provide a foundation to improve the performance of the audited entity, project or
processes. An inadequate formulation of conclusions and recommendations can diminish the result of the
auditor’s work, just as it could undermine confidence in the SAI. Furthermore, the formulation of
recommendations determines the impact the audit will have.

The survey shows that all SAIs issue recommendations in their reports: 28 SAIs (out of 33) report that it is
done in all reports and five SAIs report that it is done in some of the reports. 26 SAIs report that they issue
recommendations for all types of audit (financial, compliance and performance audits), three SAIs issue
recommendations in performance and financial audits, two SAIs in performance and compliance audits.
Two SAIs issue recommendations in performance audits only.

A review of academic literature on this matter, together with some methodological documents of SAIs
and examples provided in the survey answers, provides an overview of the basic requirements of how high-
quality recommendations could be formulated.

Before formulating recommendations, auditors should consider:

 What is the shortcoming, irregularity or misstatement identified;
 What are the root causes of the shortcoming, irregularity or misstatement identified;
 What corrective measures should be taken;
 Whether these measures taken could lead to a desired change.

First, recommendations should be linked with and express the main ideas and insights of the audit. Audit
teams should always make sure that none of the important findings are omitted from the applicable
recommendation, and vice versa, every recommendation has a significant finding as a basis.

Moreover, as SAIs should aim to issue recommendations that lead to real changes, recommendations
should not be formulated too formally, but practically and specifically enough so that they can be
implemented (and measurable so that their implementation can be monitored)3. For the same reasons,
recommendations should be realistic and feasible for the auditees.

On the other hand, SAIs must avoid putting themselves in a situation where they have to audit solutions
that they have proposed themselves. For that reason, recommendations that are too precise on the exact
mode of implementation could create complications. As suggested by one SAI, recommendations should
be formulated in a style where they describe what the auditee should do, and not how they should do it.

Finally, recommendations should be addressed to the entities which are in the position to implement
them. So, a good practice might be that recommendations are distinguished according to their addressee:
the auditee, the government (for strategic and policy issues) or parliament (for legislative and
accountability issues). Another SAI suggests that recommendations should be divided in parts according
to the addressee.

On the basis of the analysis set out above, high-quality recommendations can be formulated. The wording
of recommendations is also important, as it affects whether and how well they are going to be achieved.
In this respect, recommendations should be clear, reasoned and logical. It is also important that

3
The specificity of the recommendations is also depending on the subject dealt with: with management issues recommendations can
be more general, with technical issues recommendations can be more technical.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 10

recommendations are visible, for instance by presenting them – possibly numbered – at the end of the
report.

The formulation of high-quality recommendations is not easy. Methodological guidelines could help
auditors to formulate such recommendations. Thus, having a manual for the formulation of
recommendations could be perceived as a good practice. Our survey showed that 26 SAIs (out of 33) report
that they have manuals for formulating recommendations.

Chart 1
Does your SAI have a manual/guideline for formulating recommendations?

7no
no
77
33
respondents
33
33
respondents
respondents

26
yes
yes
26
26

In the survey we asked the SAIs whether the recommendations they formulate are specific or more
general. 29 SAIs (out of 33) report that they formulate recommendations which can be both general and
specific in nature. Probably this answer could be explained in the requirement mentioned above that
recommendations should not be too abstract on the one hand, but not too precise on the exact mode of
implementation on the other hand. Another explanation could be that performance audits which focus
on government policy effectiveness must be cautious about too specific recommendations, as they could
interfere with the autonomy of the government’s intentions and decisions.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 11

Chart 2
Are the recommendations specific or more general?
Total respondents: 33

combination of both 29

general 3

specific 1

Moreover, another important aspect is that, when it comes to formulating and presenting audit
recommendations, SAIs may find it useful to classify and/or prioritise them. High-importance
recommendations usually have a high impact on one or more stakeholders, or have a significant financial
scale, while low-importance recommendations are only relevant to a particular entity or do not affect a
significant number or group of the public, or have a low financial scale. One SAI shares the practice of
grouping the recommendations by the importance of the expected changes to the audited area4. Other
SAIs distinguish systemic from standard recommendations, where systemic recommendations require
more time and effort to implement them, or separate the recommendations that ask for short term actions
from those which demand long-term actions.

Classifications may help the auditee to better understand the auditor’s point of view and to prioritise the
actions for implementation. Furthermore, classification draws the attention of the legislature and society.
On the other hand, lower rated recommendations might be neglected by the auditee. Therefore, it is
important to communicate effectively to the auditee, so that all recommendations will be implemented.

From the auditor’s perspective, the decision on the actions to be taken in case of non-implementation,
and the timing of this, may also depend on the ranking of the recommendations.

4 For that reason, this SAI is planning to measure the importance of the expected changes.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 12

Our survey shows that few SAIs classify recommendations: only six SAIs (out of 33) use some kind of
classification.

Chart 3
Is there a classification of the recommendations?

yes
6

33
respondents

no
27

A particular form of classification is the use of management letters in performance audits, mentioned by
one SAI. Such a report usually contains remarks or recommendations considered as too specific or
technical to be included in a public audit report. In this regard, observations that are less important for
the public or parliament can yet be communicated to the auditee and provide a basis for further follow-
up.

In summary of the above, the formulation of recommendations is an important task of SAIs which could
result in a greater impact of audits if they are formulated and discussed effectively. The discussion element
is further addressed in the next chapter of the report.

3.2. Constructive relationship between auditor and auditee

Seek acceptance of the recommendations by the auditee, thereby increasing the support basis. Engage the
auditee or the government in the follow-up of the implementation of recommendations. Ask the auditee for
an action plan that specifies the measures to implement the recommendations, including deadlines. Check
the appropriateness of the action plan. In the absence of an action plan, set clear and realistic deadlines for
the implementation of the recommendations, if possible in agreement with the auditee.

Even high-quality recommendations will not lead to any change if they are not accepted and implemented
by the auditee. Therefore, effective communication and cooperation between the auditor and the auditee
is crucial, as it increases the support basis for implementing the recommendations.

Audit findings should be discussed with the auditee before commencing with the formulation of
conclusions and recommendations. Indeed, recommendations will not be properly implemented if the
auditee has any doubts regarding their effect or the cost/efficiency ratio5 of their implementation.
Therefore, the discussion with the auditee should aim to identify the key issues, the desired results and
potential measures to implement the recommendations. At the same time, auditor’s and SAI’s

5
Which means that the auditee could agree that a problem has been detected, but accepts the risk, as risk avoidance would be too
expensive (risk acceptance).
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 13

independence from government ensures that recommendations can still be issued, even if they are not
accepted by the auditee.

Our survey shows that 27 of 33 respondents base their recommendations on some kind of dialogue with
the auditee. In 23 SAIs, auditees are explicitly asked or are expected to accept the recommendations.

Chart 4
Are the recommendations based on a dialogue with the auditee?

no
6

33
respondents

yes
27

Chart 5
Is the auditee asked or expected to accept the recommendations?

no
10
33
respondents

yes
23

The difference between these two questions might differ slightly in practice. For example, the application
of a contradictory procedure with the auditee before the final editing of the report, applied by most SAIs,
could be interpreted by some respondents as the application of a dialogue, while other SAIs could consider
this as an expectation to accept the recommendations. Therefore, both answers are treated equally here.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 14

Still, some respondents provided supplementary information and showed some examples of good
practices. Some SAIs ask the auditees explicitly to indicate whether they agree, partly agree or disagree
with each recommendation, where rejected recommendations shall be discussed by the SAI on a higher
administrative level.

Another issue is that SAIs with a jurisdictional model often issue recommendations with a legally binding
character, mainly in compliance audits. Here recommendations are negotiated with the auditee to avoid
non-feasible or non-realistic recommendations. In case of a non-agreement, auditees can even get the
chance to formulate a complaint against the audit report.

Following the completion of the audit, SAIs should continue to maintain communication and constructive
relationships with the auditee.

The engagement of the auditee in the follow-up mechanism certainly forms a prime example of a
constructive relationship, and can be implemented by the use of an action plan. This plan can be a written
document, drawn up by the auditee following the audit report and usually validated by the SAI, in which
the auditee communicates the proposed measures to implement the recommendations, along with a
timetable. Several SAIs ask that the individuals responsible for the implementation of the measures are
also mentioned.

The use of an action plan also has the following advantages for SAIs:
 It ensures that the recommendations are correctly understood by the auditee and that the
measures to implement the recommendations are feasible for the auditee and appropriate for the
SAI;
 It provides a good input for the planning and the scope of the follow-up process, starting from
the timetable of the action plan;
 It puts pressure on the auditee to implement the recommendations;
 It serves as a learning tool: the results of the action plan can provide valuable information about
the reasons why recommendations might not be (completely) implemented. It could also help to
improve the quality system of the SAI and audit practices and procedures.

The survey shows that 20 respondents out of 33 use action plans as part of their follow up processes. 11
respondents of these request action plans for all audit reports. In this group, we also find SAIs with a
jurisdictional model, where action plans mostly are legally required and auditees can be punished if they
fail to submit the plan. In that case, auditees have to send their action plan within a determined date after
the audit report has been issued.

Regarding those SAIs who responded ‘sometimes’, the demand for an action plan can be related to certain
circumstances, such as in case of major irregularities, recommendations with a long-term
implementation, recommendations that concern corrective actions or in financial audits when the SAI
raises an objection due to shortcomings in financial management.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 15

The graphic below shows the result of the survey.

Chart 6
Does your SAI ask the auditee for an “action plan” that specifies the measures to implement the
recommendations?

yes
no 11
13 33
respondents

sometimes
9

Some survey answers show additional good practices in the application of the action plan. One SAI
demands the action plan at the moment of the draft report, followed by the publication of the plan in the
final audit report. This publication increases the pressure on the auditee to implement the announced
measures in a timely manner.

At some SAIs, action plans are used as a self-reporting tool, in which auditees are supposed to
communicate regularly on the outputs of their proposed measures and the current state of the
implementation of the recommendations. This pushes the auditee to work regularly towards the
implementation of the recommendations.

For one SAI, action plans are developed in agreement with the government, and hereafter approved by
government. This certainly will raise the status of the plan.

The existence of action plans influences other subsequent actions within the follow-up system. Indeed,
the demand for action plans could encourage the application of other best practices such as:
 Provide for an effective recommendation follow-up system: depending on the feedback given by
the auditee through the action plan, scope and frequency of follow-up procedures can be planned
more effectively (see 3.3);
 Report the results of the follow-up system, preferably in an aggregated way by publishing a global
report or database on the follow-up results during a specific period: action plans can be used as a
basis for reporting the follow-up results.

In the absence of an action plan, setting clear and realistic deadlines can be considered to be an alternative
good practice. Defining deadlines in agreement with the auditee contributes to a good relationship and
generates support for the implementation of the recommendations6.

6 Setting deadlines might me more suitable when recommendations are directed at the audited administration concerning
management issues. In case of recommendations directed at the government, dealing with policy issues, defining deadlines might
be less common.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 16

Out of 33 respondents, 13 SAIs state to set deadlines for the implementation. Six SAIs do this occasionally.
The occasional application often depends on the importance of the recommendations or an assessment
by the audit team.

Chart 7
Are there deadlines set for the implementation of recommendations?

yes
no
14 33 13
respondents

sometimes
6

Out of 19 SAIs who set deadlines, nine state that they determine them in agreement with the auditee,
three let the auditees set their deadlines. Deadlines are fixed independently by six SAIs. This mostly
applies to SAIs with legally binding recommendations. One respondent states that deadlines are fixed by
the Public Accounts Committee of Parliament (PAC).

Chart 8
If there are deadlines, by whom are these determined?
Total of respondents applying deadlines (yes and sometimes): 19

SAI and auditee 9

SAI only 6

auditee only 3

PAC 1
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 17

One SAI answers that for some recommendations an immediate implementation is demanded. This could
be beneficial for recommendations that can be implemented in a very short period, as a further follow-up
is no longer needed.

The engagement of the auditee in the follow-up mechanism can be shown by other practices as well. Some
SAIs involve the government in the planning of the follow-up actions. For example, in one country, the
government issues an annual report with government actions to address the recommendations indicated
in the SAI’s report. In another country, ministers’ yearly policy notes respond on the SAI’s
recommendations. Another SAI mentions that all audit reports are discussed by the government, followed
by the approval of proper measures to be implemented by the auditee.

Other SAIs provide a database accessible to the auditees, in which actions taken in response to the
recommendations can be incorporated. This database also contains underlying information, such as policy
notes, budgets, etc. The results of this self-reporting tool can be published on the SAI’s website or in an
annual report.

Even though the engagement of the auditee forms an important step in ensuring an effective follow-up
process, the application of a follow-up system by the SAI itself should be seen as the main phase in this
process. This is covered in the next chapter.

3.3. Existence of follow-up system

Provide for an effective recommendation monitoring and follow-up system that checks the implementation
in a timely manner. Repeat this follow-up, or even consider a follow-up audit, in case there is no adequate
implementation or insufficient information about the implementation.

The Mexico Declaration on SAI Independence (INTOSAI-P 10) brings forward eight core principles as
essential requirements of proper public sector auditing. Principle 7 concerns the existence of effective
follow-up mechanisms on SAI recommendations. It states that “SAIs have their own internal follow-up
system to ensure that the audited entities properly address their observations and recommendations […].”

Of 33 respondents 30 answer ‘yes’ to the question if their follow-up systems have been laid down in a
procedure.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 18

Chart 9
Is there a formal policy or procedure for follow-up?

no
3

33
respondents

yes
30

Although almost all respondents have their follow-up procedures, differences may exist in the scope of
the follow-up, the timing and frequency and the methods of measurement.

Regarding the scope of the follow-up systems, 24 of 33 respondents indicate that all recommendations of
audit reports are followed up. Four SAIs follow up recommendations on a case-by-case basis and apply a
risk-based assessment, depending on the estimate of the importance of the follow-up by the audit panel.
So, for example, follow-up could be excluded in case of major changes in the audited entity or in the
legislation concerned. The remaining five SAIs use other methods, mostly by limiting follow-up to prior
recommendations or recommendations with a financial risk, or to recommendations that demand
corrective action by the auditee.

Chart 10
Which recommendations of an audit report are followed up?
Total respondents: 33

all 24

other methods 5

determined case by case 4

Only seven respondents state that their follow-up of recommendations is solely based on a review of
documentation or a response by the auditee in an action plan or a questionnaire reply, more than half of
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 19

the respondents or 23 SAIs complete this with an effective follow-up audit. Three SAIs only use the
instrument of follow-up audits.

Chart 11
What is the information about the implementation of recommendations in audit reports based
upon?
Total respondents: 33

limited review and follow-up audit 23

limited review 7

follow-up audit 3

As follow-up audits demand a more thorough investigation, this method is not applied to all audit reports
or recommendations. SAIs who restrict the follow-up to this method select the most important audits or
recommendations. SAIs that apply both the limited reviews and the follow-up audits make the application
of a less or more thorough method conditional on the significance and the nature of recommendations.
A good practice explicitly mentioned by six respondents, consists of a follow-up process in two phases,
where in a first phase, a limited follow-up enquiry is conducted for all recommendations. The results of
this enquiry are then used to evaluate the need for follow-up audits in a second phase, where the
implementation of the most important recommendations is checked.

Regarding the timing of the follow-up procedure, 17 respondents apply a fixed timescale. As for financial
audits, all recommendations of previous annual reports on accounts are followed up in subsequent annual
reports. Follow-up of the implementation of recommendations in performance or compliance audits
generally occurs one year after the publication of the audit report. Four respondents, however, use a longer
term of two years. three respondents mention a three-year term.

Nine SAIs schedule a follow-up in accordance with deadlines laid down in an action plan or the audit
report. Seven respondents apply an ‘ad hoc’ term, which can depend on an annual audit plan or the
decision of the audit teams. One SAI mentions an ‘ad hoc’ term because all audit recommendations are
followed up as part of the next audit on the same topic.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 20

Chart 12
At what time is the follow-up of the recommendations carried out?
Total respondents: 33

fixed timescales 17

deadlines 9

ad hoc 7

A repeated follow-up mechanism, that keeps the pressure on until all recommendations have been
implemented, can be considered to be good practice. 16 SAIs state that they follow up repeatedly, but for
two of them this is restricted to financial audits. Most of them do this on an annual basis until all
(important) recommendations are implemented, four SAIs state that this annual follow-up runs until five
or six years after the audit report, one SAI repeats the follow-up only once.

11 respondents answer that the frequency of the follow-up mechanism is decided (mostly by the audit
panel) on a case-by-case basis, depending on the importance of the recommendations. Six SAIs do not
repeat their first follow-up exercise.

Chart 13
How often is the follow-up of recommendations carried out?
Total respondents: 33

repeatedly 16

case by case 11

once 6

There are two main ways to measure the impact of audit recommendations: using a qualitative approach
or a quantitative approach. The quantitative approach usually consists of displaying the number or
percentage of implemented recommendations. Although this method constitutes a simple and clear
indicator of impact, it also has some disadvantages. Firstly, this method usually does not take into account
the different types of recommendations on which it reports, so that the outcome does not indicate
whether or not the implemented recommendations are the most significant ones. Secondly, the use of
this method could encourage auditors to produce recommendations that are easy to implement, thus
increasing the impact ratio of their audits.

The qualitative method presents the information on the audit implementation in a descriptive way.
Whereas this approach provides a broader and more nuanced view, with other ways of impact captured,
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 21

it has the disadvantage that no aggregated data on impact can be shown to be used for internal
performance monitoring (see 3.6) or external reporting (see 3.5).

As each approach has drawbacks, the application of both methods can be considered to be good practice.
More than half of the respondents, or 21 of 33 respondents, apply both approaches, 11 apply a qualitative
approach and one SAI only uses a quantitative approach.

Chart 14
Does your SAI carry out the follow-up of recommendations in individual reports in a
quantitative or qualitative manner?
Total respondents: 33

combination of both 21

qualitative 11

quantitative 1

Regarding the quantitative approach, a distinction between the significant and less significant
recommendations can be considered to be good practice. Nevertheless, only six SAIs use this classification.

An interesting extension of this quantitative approach is an estimation of the financial impact, which is a
calculation of the total savings or additional revenues arising from the implementation of the
recommendations versus the cost of the audit operation or the cost of the implementation of the
recommendations. This methodology is used by only three SAIs, and is under development at one SAI.
This is presumably caused by the high difficulty of its calculation. According to one of the answering SAIs,
hypotheses are used for this calculation.

Regarding the qualitative approach, 27 SAIs use a description of implemented and non-implemented
recommendations, 16 of them add to this an evaluation of the causal connection between the
recommendations and the actions planned or taken by the auditee, 11 SAIs also evaluate the long-term
outcomes of the audit. Three SAIs supplement this with other methods, such as a description of the impact
on media and stakeholders.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 22

Chart 15
What is the nature of qualitative information?
Total respondents: 33, multiple answers were possible

description of the (non-)implemented

recommendations 27

evaluation of the causal connection

between the audit recommendations and 16
the actions planned or taken

evaluation of long term outcomes of the

audit report 11

other 3

In summary of this chapter, nearly all respondents state that they apply a follow-up procedure. 26 SAIs do
more than a mere review of documentation or of the responses received by the auditee. A repeated follow-
up mechanism, that puts pressure on the auditee, exists at 16 SAIs. Another good thing is that the
application of both quantitative and qualitative measurement methods is done by more than half of the
respondents. However, innovative approaches such as an estimation of the financial impact of the
recommendations or an evaluation of the long-term outcomes of the audit, are applied by only a small
minority.

A further issue in the follow-up process is the question which actions SAIs must undertake if the follow-
up procedure reveals that the auditee has failed to implement the recommendations.
Some SAIs, mostly with a jurisdictional model, can impose fines or report it to the prosecuting authorities,
but the majority have to consider other methods. They can use parliamentary pressure or communicate
the follow-up results to a broader audience. Both approaches are treated in the following chapters.

3.4. Parliamentary involvement

Engage parliament in the follow-up of the implementation of recommendations.

SAIs often inform their parliaments on the implementation rate of their recommendations. They may also
wish to bring contentious issues or matters of relative importance to the attention of parliament or its
specific committees. INTOSAI principally recommends such a step if it is considered appropriate by the
SAIiv.

Some scientific studies suggest that parliamentary pressure can play a crucial role in the implementation
of audit recommendations by increasing pressure on the auditee and the government, although others
have found the impact to be rather marginalv. Probably different paths that effect impact should act
together to increase the impactvi. For instance, media coverage on contentious issues is expected to have
an impact on legislature as well, especially if the relevant legislative body chooses not to implement the
SAI’s recommendation for political reasons vii.

26 of 33 respondents confirm that their respective legislative bodies play a role in the implementation of
audit recommendations. The figure below shows the respective ratio on parliamentary involvement across
all respondents.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 23

Chart 16
Does parliament play a role in the implementation of recommendations?

no
7
yes
33 13
respondents

sometimes
13

Parliamentary involvement, however, varies considerably in respondents’ countries. In some cases, it may
be limited to the (regular) receipt of information, covering the level of compliance to the SAI’s
recommendations on the part of the auditee. We have, however, not always received sufficient
information, in how far that information is relevant to subsequent parliamentary discussions and related
decisions.

Some SAIs explicitly mention parliamentary (committee) hearings, mostly with the participation of the
government or auditee and the SAI. The auditee is usually asked whether or not recommendations have
been implemented, and, if not so, for the reasons for non-implementation. This parliamentary hearing
normally takes place after the publication of the audit report. One SAI mentions an additional
parliamentary hearing, as it proposes each year to parliament to organise specific follow-up debates on
previous audit reports whose recommendations are not sufficiently implemented.

Whether or not those parliamentary sessions have consequences (e.g. government agreement to
implement contentious recommendations or parliamentary directives) remains unclear as well. Some
SAIs, however, state that parliamentary discussions lead to some level of parliamentary recommendations,
endorsements or demands for the implementation of recommendations. Here, we could not always
properly determine in how far those demands are binding for the auditee. Only in the minority of cases,
respondents have explicitly stated that adherence to parliamentary committee resolutions or decisions
are – in practice – binding for the auditee.

At one SAI, parliamentary involvement is more developed, in the sense that the parliamentary committee
plays a more active role in the follow-up process. In this case, it is the Public Accounts Committee (PAC)’s
role to ask the responsible minister to comment on the audit report, immediately following its publication.
The minister communicates the planned measures both to the SAI and PAC, based on which the SAI
investigates at a later stage whether all issues have been appropriately addressed or whether there are still
outstanding matters for follow-up. A global report of aggregated follow-up results is drawn up by the PAC
as well.

A different manner of parliamentary impact is mentioned by another SAI, where the auditee is expected
to report to parliament on the degree of compliance to the SAI’s recommendations as part of the process
leading to the approval of its annual budget. Similarly, one SAI mentioned the possibility that, based on
findings and recommendations from the annual report, parliament may respond by decreasing the budget
of one or more budget users. Finally, the practice in one SAI where the ministers’ yearly policy notes with
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 24

their response on the SAI’s recommendations are submitted to parliament, may be seen as a combined
engagement of both auditee and parliament.

On the one hand, binding parliamentary decisions or resolutions may be the most effective instrument to
guarantee implementation of SAI recommendations. Nevertheless, non-binding recommendations,
discussions or even pure reporting-mechanisms may also have a considerable effect, especially if such
communication is made public. Parliamentary discussions and media attention may work hand in hand
to facilitate the implementation of audit recommendations. Auditees may principally wish to avoid this
level of attention7.

In summary, if national regulations permit so, the involvement of parliament may be a powerful tool to
enhance the effectiveness of audits. This works especially well if combined with an efficient reporting
mechanism that communicates issues to the wider group of stakeholders. This aspect is further addressed
in the next chapter.

3.5. Report the results of the follow-up system

Report the results of the follow-up system, preferably in an aggregated way by publishing a global report or
database on the follow-up results during a specific period.

A clear and effective communication of audit results is essential for a SAI to fulfil its mandate.
Stakeholders or audiences of audit information, however, are to be found beyond the auditee - which is
usually part of government or its wider administration - itself. Parliament or specific committees thereof
are also crucial recipients of audit information. The media, NGOs and the general public form part of the
wider group of stakeholders.

Consequently, according to INTOSAI, the results of SAIs’ follow-up exercises should be communicated
publicly, unless regulations stipulate otherwiseviii. ISSAI 3000 promotes an appropriate reporting
mechanism including the support of all SAI stakeholdersix.

In light of the above, a communication or reporting system on the results of the follow-up of audit
recommendations may fulfil various purposes, such as:

 Internal aggregation of information on follow-up results, e.g. for risk assessments and/or audit
planning.
 External presentation of information on follow-up results to various stakeholders, e. g.
parliamentarians, journalists or the broader public, either for transparency reasons or to raise the
pressure on the auditee.
 Assessment of the SAI’s performance and accountability-reporting towards the SAI’s
stakeholders.

Almost every respondent indicates some method of communication of follow-up results, although in some
cases, this is limited to the reporting of follow-up audits. The communication of the results of follow-up
activities can be addressed to the auditee itself, but various SAIs share these results with other parts of
the respective department or to related government bodies. In some cases, parliament receives individual
reports on a regular basis. Furthermore, relevant information may be made public on SAIs’ websites.
Respondents’ individual approaches vary considerably, also depending on the audit subject (e. g. financial,
compliance or performance audit).

A substantial number of respondents have specific reporting mechanisms in place that go beyond the
distribution of individual reports related to the follow-up of audit recommendations.

7 At the same time, publishing positive follow-up results rewards the well-performing auditees (see chapter 2.1).
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 25

The figure below shows the spread across all respondents regarding the availability of a ‘global report’ that
is either used for the internal or external presentation of follow-up activities or results. For the purposes
of this document, a ‘global report’ constitutes some form of aggregation (either online or in print) of
relevant information covering a specific time frame (e.g. within a calendar year).

Chart 17
Does your SAI make a global report on the follow-up of recommendations/reports and is it
communicated to external stakeholders?
Total respondents: 33

yes, and communicated 20

no 10

yes, but not communicated 3

The level of information contained in these reports may vary. The figure below shows the type of
information covered in respondents’ global reports. Various SAIs selected more than one option. As
expected, most of the SAIs report on the follow-up of recommendations. Some reports also include
information on the follow-up of the audit findings or a further evaluation of the impact.

Chart 18
What kind of information does this report contain?
Total respondents: 33, multiple answers possible, distribution per category

follow-up of recommendations 22

follow-up of findings, conclusions 10

evaluation of impact 6

The figure below shows the manner in which information is presented. Multiple selections were possible.
Most SAIs use an aggregation of the follow-up results. Some SAIs complement this with an aggregation
on the level of ministries or departments or provide additional detailed information of follow-up results
of individual audits. The qualitative technique of case studies, where examples of follow-up results of
individual audits are described, is rather rarely applied.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 26

Chart 19
How is this report presented?
Total respondents: 33, multiple answers possible

aggregation of follow-up
results of individual audits 14

aggregation on level
of ministries or departments 8

detailed information of follow-up

results of individual audits 7

case studies
(narrative techniques) 3

Other 2

Good practice in this field mainly depends on the potential ‘range of influence’: a broader distribution of
information on the implementation of audit recommendations is to be preferred over a sole internal use
or a smaller distribution of information. As demonstrated in the figure above, 20 of 33 respondents
communicate their global report externally. This is usually done by putting this report on the website, or
by including the follow-up information in the SAI’s annual activity report. Some respondents also refer to
a database used by the auditee in order to report on the fulfilment of the recommendations. One SAI
answered that it had recently developed a database that is accessible to the auditee. The database builds
on the action plans and the corresponding deadlines. The reporting of the auditee on the fulfilment of
recommendations, including the uploading of audit evidence and documentation, is done via the database
system. Deadlines trigger automatic notifications that are sent to auditees. The analysed data are then
published online.

Good practices are also defined by the type of information presented. While most SAIs follow either a
quantitative approach, by presenting an aggregation of the follow-up results as a number or percentage
of implemented recommendations, or a qualitative approach, by providing detailed information of the
follow-up results in a descriptive way, one SAI answers that its annual public report combines both
approaches: The report provides a general statistical presentation on the implementation of all
recommendations, based on a rating mechanism designed to assess the degree of implementation 8, as
well as an in-depth analysis of the implementation status of certain selected recommendations.

The level of detail presented may also vary. One SAI for example, describes the selected recommendations
of the individual audit reports followed up. The SAI provides an assessment on the extent of
implementation of these recommendations and a short explanation supporting the audit team’s
judgment. A final conclusion for each audited entity on the overall progress in implementing the
recommendations is also issued.

Summing up, an adequate communication or reporting on audit recommendations and its

implementation supports SAIs’ justification and effectiveness. SAIs may wish to consider a combination

8
Completely implemented, implementation in progress, implementation incomplete, no implementation, no longer applicable,
refusal to implement.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 27

of a qualitative and quantitative approach. The further utility (e.g. for risk assessments) of information on
the implementation of audit recommendations is discussed in the next chapter.

3.6. Use of follow-up results for performance monitoring and risk assessment

Use the results of the follow-up system as a performance monitoring system and for risk assessment.

 Use for performance monitoring system

According to the INTOSAI-P 20 Principles of Transparency and Accountability, SAIs may use performance
indicators to assess the value of audit work for parliament, citizens and other stakeholders.

Regarding the topic of follow-up, ISSAI 3000 states that following up on recommendations may serve the
evaluation of SAI performancex. Also, the INTOSAI guide Supreme Audit Institutions: Performance
Measurement Frameworkxi mentions that, ‘if data regarding the implementation of the SAI’s
recommendations is available, the proportion of the recommendations that are partially or fully
implemented by the audited bodies would be an interesting figure to take into consideration to assess the
credibility and legitimacy of the SAI within its broader institutional environment’. Furthermore, as these
follow-up results are easy to understand for stakeholders (parliament, society), this type of indicator
seems to be an adequate instrument for SAIs to report on performance evaluation9. Of course, SAIs’
performance cannot be assessed by implementation rates only, and must be evaluated by a large set of
indicators in different domains.

Regarding the question if SAIs use the results of follow-up in their performance monitoring system, the
survey answers show a divided reality, as 18 SAIs answered ‘yes’ and 15 answered ‘no’. Concerning those
who answered ‘no’, respondents did not give the reasons for this answer10.

Chart 20
Does the performance monitoring system of your SAI use the results of follow-ups?

no
33
15 respondents yes
18

Some of the SAIs who answered ‘yes’ gave additional information that can be used as examples of good
practices.

9 This may, however, lead to undesirable side effects, as auditors could focus on recommendations related to easy-to-fix problems
(‘low-hanging-fruit’), only to raise the SAI’s performance rate.
10 There might be no performance monitoring system at all, or this system might not take into account the follow-up results.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 28

SAIs mention indicators as statistics on fulfilled/unfulfilled measures, the recovery of damages, changes or
legal improvement, organisational measures, and administrative and disciplinary measures. Another SAI
does not specify the use of follow-up results for performance monitoring, but the statistical presentation
on the implementation of recommendations in its annual report, with different categories on the degree
of implementation (see chapter 3.5), already provides useful performance indicators.

One SAI uses a target value for performance indicators: 90% of all unresolved issues11 in our major studies
are addressed by the responsible minister within three years. These targets are also published on their
website and annual activity report.

 Use for risk assessment

According to ISSAI 300xii, ‘auditors should select audit topics through the SAI’s strategic planning process
by analysing potential topics and conducting research to identify risks and problems. […] Formal
techniques to prepare the strategic planning process, such as risk analysis or problem assessments, can
help structure the process but need to be complemented by professional judgement to avoid one-sided
assessments’.

At this strategic level SAIs want to maximise their value for society by helping to create good public
governance, by limiting negative financial and societal impacts of bad governance, and by stressing
compliance of public sector officials with laws and regulationsxiii. In this regard, audit reports should help
to improve governance, and an effective measure to valuate this impact is the degree of implementation
of recommendations, checked through the follow-up procedure. The results of these follow-ups are
therefore an important element for the risk assessment of SAIs: Do the recommendations and other results
generally lead to changes in public governance and if so to what degree? Do global results point to specific
problematic areas (for example lack of IT security), or more systemic risks (for example, absence of
internal controls)? If the degree of changes is not sufficient, this means that there are shortcomings in the
progress towards good governance. These shortcomings could be addressed in the strategic planning of
audit subjectsxiv.

Not implementing recommendations also constitutes the basis for other risks at strategic level, as it can
alter the image of the SAI in a negative way: the confidence of organisations and the public in the SAI
could diminish, or the audited organisations could be reluctant to work together with the SAI as they see
that the SAI lacks authority to impose its recommendations.

Besides this strategic risk assessment, operational plans with audit subjects12 should also be based on a
risk assessment. In this context, the fact that recommendations have not been implemented forms a risk
towards the audited organisation. On the other hand, in case of implementation, the SAI could lower the
risk factor for that organisation and focus its resources on others.

So, one survey question asks if the results of the follow-up procedures form a part of the risk assessment
of the SAI, and, if they do, if they impact risk assessment at strategic level and/or operational level.

Regarding the first question, 28 SAIs use the results of follow-ups for risk assessment, although for 7 of
them this use is not systematic.

11
Issues are considered unresolved if the measures implemented by the minister were not considered satisfactory by the Auditor
General.
12
The INTOSAI Global Stocktaking Report 2017 reveals that most SAI draw up operational (or annual) plans https://www.idi.no/en/all-
news/idi-news/item/128-global-sai-stocktaking-report-2017.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 29

Chart 21
Do the results of the follow-up procedures form a part of the risk assessment of the SAI in the
following years?

no
5

33
sometimes respondents
7 yes
21

Chart 22
In case the results are used for risk assessment: do they impact risk assessment at strategic
level and/or operational level?
Total respondents: 33

strategic and operational level 16

operational level only 12

N/A 5

All 28 use the results at the operational level, 16 of them also use them at the strategic level.

As the survey did not ask for a description of the risk assessment, and the role of follow-ups in it, no
further information is available on this. Only one SAI indicates explicitly that it applies a particular
methodology of risk assessment based on all follow-up results. Moreover, many SAIs’ responses to other
questions illustrate that follow-up procedures may result in follow-up audits. Where it is not clear if this
process is included in an official (operational) risk assessment that leads to an operational plan, it still
shows an implicit use of follow-up results for operational planning.

In summary of the above, we can conclude that the use of the follow-up results as a performance measure
for the SAI’s action is not very widespread, even if this type of measurement is easy to understand for the
general public. The use of the results of follow- up as an input for risk assessment is more general, certainly
at an operational level. As ISSAI 300 Performance Audit Principles refers on this matter to strategic
planning, progress should be made by SAIs to adhere to this principle.
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 30

4. Closing remarks

It might be possible that some aspects of good practices proposed here are difficult to adapt or implement
for individual SAIs. This may especially be true if national legislation does not permit, e.g., a stronger
involvement of parliament. Existing processes may be established in detailed procedures, from which SAIs
are not allowed to deviate. They may also be limited in their ability to communicate audit results to a
broader audience, as audit data can be classified or confidential by national regulations. Furthermore,
SAIs may wish to maintain good relationships and promote an atmosphere of trust with the auditee, and
therefore want to avoid exposing follow-up results that could be harmful for the reputation of the auditee.

Notwithstanding the above, the authors of this report are convinced that the results of this survey offer
many ideas and starting points for SAIs to learn from each other, and to move towards an efficient and
effective follow-up system that increases the impact of audit reports. An important lesson to be learned is
that such a follow-up system should go beyond the existence of a follow-up procedure and is related to
the whole audit process, from the drafting of the audit report to the use of the follow-up results in the
context of a risk assessment. Similarly, SAIs should not act alone and must engage their stakeholders in
this process: the auditee, parliament, media and the general public.

The results of the survey obviously describe a snapshot of the current situation and various areas relevant
for the follow-up of audit recommendations remain in continuous change. This is particularly true for
those good practices involving modern information technology. Here, it will be interesting to observe the
evolution of SAIs’ reporting mechanisms or auditees’ self-reporting tools. Likewise, scientific research on
impact of audit reports will lead to further insights, as some aspects should be further investigated13. So,
future studies and surveys within EUROSAI will be necessary to pick up on interesting ideas and practices
among the EUROSAI members. It would be recommended that EUROSAI should launch further studies
and activities in this area so that SAIs would have more possibilities to increase the impact on their
governments and society.

13 Current research hardly addresses various causal paths that can lead to impact. For example, is a certain factor (a constructive
relationship between auditor and auditee; pressure from the media) a necessary condition (along with other factors) for impact, or is
it only a contributing factor? (PUT V., The Impact of Performance Auditing – a practice friendly review, 2018).
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 31

Annex 1: survey

Survey on follow-up of audit recommendations

1. Identification: SAI of
Country:
Email:

A. Nature of recommendations

2. Do your SAI’s audit reports contain recommendations (as meant in ISSAI level 2: ISSAI 10 & 12)?
 Yes  Some reports  No
If answer is no: go to question 16

3. Are the recommendations related to a certain kind of audit (as meant in ISSAI level 3: ISSAI 100-
400)? [multiple answers possible]

 Performance audit  Compliance audit

 Financial audit  All

Remarks:

4. Has your SAI a manual/guideline for formulating recommendations (style of writing, content,
max. number of recommendations,…)?
 Yes (explain)  No
Explanation:

5. Are the recommendations based on a dialogue with the auditee?

 Yes  No
Remarks:

6. Is the auditee asked or expected to accept the recommendations?

 Yes (explain)  No
Explanation:

7. If the auditee is asked or expected to accept the recommendations: what are the consequences
in case of non-acceptation?
Explanation:
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 32

8. Are the recommendations specific (recommendation limited to one or more concrete aspects, ex.:
an IT-tool for project management should be implemented, a specific procedure must be
adapted) or more general (ex.: project management should be implemented, procedures should
be implemented)?
 Specific  General  Combination of both
Remarks:

9. Are the recommendations directed at the audited administration (ministry, agency, governing
board,…), the government (minister(s), cabinet member, collegium of ministers) or Parliament
(plenary of commission)?
 Administration  Government  Other (explain)
 Parliament  Multiple recipients (explain)
Explanation:

10. Is there a classification of the recommendations (ex: high/medium/low; short term/medium

term/long term)?
 Yes (explain)  No
Explanation:

11. Are there legal requirements for implementing the recommendations?

 Yes (explain)  No
Explanation:

12. Does your SAI ask the auditee for an “action plan” that specifies the measures to implement the
recommendations?

 Yes  Sometimes  No
If yes or sometimes, what should be included (actions, persons in charge, deadlines, ….):

13. Are there deadlines set for the implementation of recommendations?

 Yes (explain)  Sometimes (explain)  No
Explanation:

14. If there are deadlines: they are determined by

 SAI  Auditee  SAI and auditee (by agreement)

 Regulation  Other (explain)  Not applicable

Explanation:
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 33

15. Does parliament play a role in the implementation of recommendations?

 Yes (explain)  Sometimes (explain)  No
Explanation (ex. In case of non-implementation the status is brought to the attention of Parliament,
Parliament has the possibility to endorse recommendations,…):
Go to question 17

B. Follow-up system for individual audits

B.1 Follow-up system

16. (if question 2 answers no) Does your SAI follow up audit reports (findings, conclusions, …)? (note:
this question concerns the follow-up of the content of the report, not the wider impact)
 Yes (explain)  Sometimes (explain)  No

Explanation:
If answer is no: end of survey
If answer is yes or sometimes: go to question 19, then continue from B.3 (questions 32 through 33)

17. Does your SAI follow up audit recommendations?

 Yes  Sometimes (explain)  No

Explanation:
If answer is no: B.2 (questions 20 through 31) can be skipped

18. Does your SAI follow up audit reports besides the implementation of its recommendations
(findings, conclusions, …)? (note: this question concerns the follow-up of the content of the
report, not the wider impact)
 Yes
 No
 Sometimes (explain)
Remarks:
If answer is no AND answer on question 17 is no: end of survey

19. Is there a formal policy or procedure for follow-up?

 Yes  No
Remarks:
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 34

B.2 Follow-up of audit recommendations

20. Which recommendations of an audit report are followed up?

 All
 Sample basis (explain)
 Determined on case by case basis (explain)
 Other methods (explain)
Explanation:

21. What is the information about the implementation of recommendations in audit reports based
upon?
 Notice taken of response by auditee (ex. action plan by auditee, questionnaire reply by auditee,
….)
 Limited review of response by auditee or documentation
 Follow-up audit
 Other (explain)
Explanation:

22. At what time the follow-up of the recommendations is carried out?

 Fixed time scale (explain)
 Ad hoc (explain)
 Based upon the deadlines (cf. question 13)
 Other (explain)
Explanation:

23. How often is the follow-up of recommendations carried out?

 Once
 Repeatedly (explain)
 Decided on case by case basis (explain)
 Other (explain)
Explanation:
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 35

24. Does your SAI carry out the follow-up of recommendations in individual reports in a quantitative
or qualitative manner?
 Quantitative (consists of displaying a number or percentage) (go to questions 25 – 27)
 Qualitative (presents the information in a descriptive way) (go to question 28)
 Combination of both (explain) (continue)
Remarks:

25. In case of a quantitative manner, does it consist of: [multiple answers possible]
 Statistics on audit recommendations (number of implemented recommendations)
 Statistics on audit recommendations (percentage of implemented recommendations compared to
total of recommendations)
 Financial impact estimates (ex. Calculation of total savings and additional revenues or ratio of
estimated savings and additional revenues vs. the cost of audit operation or cost of implementing the
recommendations)
 Other (explain):
Explanation:

26. In case of a financial impact estimation (cf. question 25), does your SAI have a special
methodology for evaluating or measuring this impact?
 Yes (explain)  No  Not applicable

Explanation:

27. In case of a quantitative manner, does it take into account the different types of
recommendations and does it distinguish the significant and less significant ones?
 Yes (explain)  No
Explanation:

28. In case of a qualitative manner:

 Description of the (non-)implemented recommendations
 Evaluation of causal connection between the audit recommendations and the actions planned or
taken by the auditee
 Evaluation of the long-term outcomes of the audit report
 Other: explain
 Not applicable
Explanation:
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 36

29. In case of an evaluation of the causal connection or the long-term outcomes (cf. question 28),
does your SAI have a special methodology for the evaluation?
 Yes (explain)  No  Not applicable
Explanation:

30. Does your SAI take action if the SAI assesses that there is no adequate implementation of the
recommendations (auditee has not taken appropriate or enough action)?
 Yes (explain)
 Decided on case by case basis (explain)
 No
Explanation:

31. Is your SAI legally required to take action in case there is no adequate implementation of
recommendations?
 Yes (explain)  No
Explanation:

B.3 Communication of follow-up (of recommendations and/or content)

32. Are the results of the follow-up of individual audits communicated?

 Yes  Sometimes (explain)  No

Explanation:

33. To whom are the results communicated?

 Internally (explain)  Externally (explain)  Not applicable

Explanation:

C. Follow-up system in general

These questions are for SAI’s who follow up recommendations as well SAI’s who follow up the content
of reports (findings, conclusions,…). Therefore the words ‘recommendations/reports’ are used in these
questions.
34. Who is responsible for the overall process of the follow-up of recommendations/reports and
summarizing the global information?
 Special department in the SAI
 Appointed auditor or other employee
 Other (explain)
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 37

Explanation:

35. Does your SAI make a global report (all or part of follow-ups during a specific period) on the
follow-up of recommendations/reports?
 Yes  No
Remarks:

36. In case of a global report: is this report communicated outside the SAI?
 Yes  No  Not applicable
If yes: to whom?

37. In case of a global report: what kind of information does this report contain? [multiple answers
possible]
 Follow-up of recommendations
 Follow-up of findings, conclusions
 Evaluation of impact
 Not applicable
Remarks:

38. In case of a global report: how is this report presented? [multiple answers possible]
 Aggregation of follow-up results of individual audits
 Aggregation on level of ministries or departments
 Detailed information of follow-up results of individual audits
 Case studies (narrative techniques)
 Other (explain)
 Not applicable
Remarks:

39. Does the performance monitoring system of your SAI uses the results of follow-ups?
 Yes (explain)  No
Explanation:
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 38

40. Is the follow-up system evaluated (as part of the evaluation of the quality system of the SAI)?
 Often and/or periodically
 Rarely and/or not periodically
 Never
Remarks:

41. In case of evaluation, did it lead to major changes in the system or will major changes be
implemented in the near future?
 Yes (explain)  No  Not applicable

Explanation:

42. Do the results of the follow-up procedures form a part of the risk assessment of the SAI in the
following years?
 Yes  No  Sometimes
Remarks:

43. In case the results are used for risk assessment: do they impact risk assessment at strategic level
(goals and planning at SAI level) and/or operational level (determining audit subjects for specific
matters)?
 Risk assessment at strategic level
 Risk assessment at operational level
 Both
 Not applicable
Remarks:

44. Are there best practices on follow-up on audit reports your SAI would like to share (which could
be useful to other SAI’s)?
Remarks:

45. Are there any other aspects concerning the topic of this survey your SAI would like to share?
Remarks:
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 39

Annex 2: survey answers

Survey Question Answers with number of SAIs

Do your SAI's audit reports contain YES 28

recommendations? SOME REPORTS 5
NO 0

Are the recommendations related to a ALL AUDITS 26

certain kind of audit? PERFORMANCE 2
PERFORMANCE AND COMPLIANCE 2
PERFORMANCE AND FINANCIAL 3

Has your SAI a manual/guideline for YES 26

formulating recommendations? NO 7

Are the recommendations based on a YES 27

dialogue with the auditee? NO 6

Is the auditee asked or expected to accept YES 23

the recommendations? NO 10

Are the recommendations specific or more BOTH 29

general? GENERAL 3
SPECIFIC 1

Are the recommendations directed at the ADMINISTRATION/GOVERNMENT 17

audited administration, the government or ADMINISTRATION/GOVERNMENT/ 3
Parliament? PARLIAMENT
MULTIPLE RECIPIENTS 13

Is there a classification of the YES 6

recommendations? NO 27

Are there legal requirements for YES 8

implementing the recommendations? NO 25

Does your SAI ask the auditee for an action YES 11

plan? SOMETIMES 9
NO 13
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 40

Are there deadlines set for the YES 13

implementation of recommendations? SOMETIMES 6
NO 14

If there are deadlines, they are determined SAI AND AUDITEE 9

by SAI 6
AUDITEE 3
PARLIAMENT (PAC) 1

Does parliament play a role in the YES 13

implementation of recommendations? SOMETIMES 13
NO 7

Does your SAI follow up audit YES 31

recommendations? SOMETIMES 2
NO 0

Does your SAI follow up audit reports YES 10

besides the implementation of its SOMETIMES 10
recommendations?
NO 13

Is there a formal policy or procedure for YES 30

follow-up? NO 3

Which recommendations of an audit are ALL 24

followed up? CASE BY CASE 4
OTHER METHODS 5

What is the information about the NOTICE TAKEN OF RESPONSE BY 0

implementation based upon? AUDITEE
LIMITED REVIEW OF RESPONSE 7
FOLLOW-UP AUDIT 3
LIMITED REVIEW + FOLLOW-UP AUDIT 23

At what time the follow-up of the FIXED TIME SCALE 17

recommendations is carried out? BASED UPON DEADLINES 9
AD HOC 7
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 41

How often is the follow-up of REPEATEDLY 16

recommendations carried out? CASE BY CASE 11
ONCE 6

Does your SAI carry out the follow-up in QUANTITATIVE 1

individual reports in a quantitative or QUALITATIVE 11
qualitative way?
BOTH 21

In case of a quantitative manner, does it NUMBER OF IMPLEMENTED 17

consist of (multiple answers possible) RECOMMENDATIONS
PERCENTAGE OF IMPLEMENTED 17
RECOMMENDATIONS
FINANCIAL IMPACT ESTIMATION 3
OTHER 4

In case of a financial impact estimation, YES 4

does your SAI have a special methodology? NO/N-A 29

In case of a quantitative manner, does it YES 6

distinguish the significant and less NO/N-A 27
significant ones?

In case of a qualitative manner, does it DESCRIPTION OF IMPLEMENTED 27

consist of (multiple answers possible) RECOMMENDATIONS
EVALUATION OF CAUSAL 16
CONNECTION
EVALUATION OF LONG-TERM 11
OUTCOMES
OTHER 3

In case of an evaluation of the causal YES 2

connection or the long-term outcomes, does NO/N-A 31
your SAI have a special methodology?

Does your SAI take action if the SAI assesses YES 17

that there is no adequate implementation of CASE BY CASE 10
the recommendations?
NO 6

Is your SAI legally required to take action in YES 8

case there is no adequate implementation NO 25
of recommendations?
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 42

Are the results of the follow-up of individual YES 26

audits communicated? SOMETIMES 7
NO 0

To whom are the results communicated? INTERNALLY 2

EXTERNALLY 31

Who is responsible for the overall process of SPECIAL DEPARTMENT 10

the follow-up of recommendations? APPOINTED AUDITOR 13
OTHER 10

Does your SAI make a global report on the YES 23

follow-up of recommendations? NO 10

In case of a global report: is this report YES 20

communicated outside the SAI? NO 3
N/A 10

In case of a global report: what kind of FOLLOW-UP OF RECOMMENDATIONS 22

information does this report contain? FOLLOW-UP OF FINDINGS, 10
(multiple answers possible) CONCLUSIONS
EVALUATION OF IMPACT 6
N/A 10

In case of a global report: how is this report AGGREGATION OF FOLLOW-UP 14

presented? (multiple answers possible) RESULTS OF INDIVIDUAL AUDITS
AGGREGATION ON LEVEL OF 8
MINISTRIES OR DEPARTMENTS
DETAILED INFORMATION OF 7
FOLLOW-UP RESULTS OF INDIVIDUAL
AUDITS
CASE STUDIES 3
OTHER 2

Does the performance monitoring system of YES 18

your SAI use the results of follow-up? NO 15

Is the follow-up system evaluated (as part of OFTEN 15

the evaluation of the quality system of the RARELY 14
SAI)?
NEVER 4

In case of evaluation, did it lead to major YES 12

changes in the system or will major changes NO 17
be implemented?
N/A 4
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 43

Do the results of the follow-up procedures YES 21

form a part of the risk assessment of the SOMETIMES 7
SAI?
NO 5

In case the results are used for risk STRATEGIC LEVEL 0

assessment, do they impact risk at strategic OPERATIONAL LEVEL 12
or operational level?
BOTH 16
N/A 5
 FOLLOW-UP OF IMPLEMENTATION OF AUDIT RECOMMENDATIONS / 44

REFERENCES

i Canadian Audit & Accountability Foundation (2019). The impact of performance audits: defining,
measuring and reporting impact, a discussion paper.
ii Van Loocke, E. & Put, V. (2011). The impact of performance audits: a review of the existing evidence.
In J. Lonsdale, P. Wilkins & T. Ling (Red.), Performance Auditing. Contributing to Accountability
in Democratic Government (pp. 175-208). Cheltenham, UK – Northampton, MA, USA: Edward
Elgar.
iii Van Loocke, E. & Put, V. (2011). Put V. (2018). The impact of Performance Auditing – a Practice
Friendly review (http://intosaijournal.org/the-impact-of-performance-auditing-a-practice-
friendly-review/)
iv INTOSAI Capacity Building Committee (CBC) (2010). How to increase the use and impact of audit
reports, A guide for Supreme Audit Institutions.
v Canadian Audit & Accountability Foundation (2019). The impact of performance audits: defining,
measuring and reporting impact, a discussion paper. Van Acker, W. & Bouckaert, G. (2018). The
Impact of Supreme Audit Institutions and Ombudsmen in Belgium and The Netherlands, John Wiley
& Sons Ltd.
vi Put, V. (2018). The impact of Performance Auditing – a Practice Friendly review.
vii Canadian Audit & Accountability Foundation (2019). The impact of performance audits: defining,
measuring and reporting impact, a discussion paper.
viii INTOSAI Capacity Building Committee (CBC) (2010). How to increase the use and impact of audit
reports, A guide for Supreme Audit Institutions.
ix ISSAI 3000 Performance Audit Standards, par. 138.
x ISSAI 3000 Performance Audit Standards, par. 137.
xi INTOSAI Development Initiative (IDI) (2016). Supreme Audit Institutions Performance
Measurement Framework.
xii ISSAI 300, par. 36.
xiii Dimitrova, D. (2019). Not failing to prepare: supporting strategic management in SAI towards
stronger performance. EUROSAI Magazine no. 25, 2019, p 88.
xiv INTOSAI, Paper on a Risk-Assessment Framework for SAIs to Incorporate Relevant SDGs Related
Programs in their Annual Audit Plans
https://www.intosaicommunity.net/document/exposure_draft/11_Paper_on_SDGs_23nov18.pdf
AFROSAI, Risk management guidelines for SAI, https://afrosai-e.org.za/wp-
content/uploads/2019/04/Risk-Management-Guideline-for-SAIs1_00.pdf; OECD, Using Risk
Assessment in Multi-year Performance Audit Planning, https://www.oecd.org/gov/ethics/using-risk-
assessment.pdf .