How to use this tool

Step 0: Data collection and

analysis

Step 1: ORP

Step 2: Complexity evaluation

Step 3: Additional assessment

using COVID-19 Checklist

Step 4: Determine
surveillance frequency using
matrix and modifier Safety data & Risk-b
Safety data & Risk-based Surveillance Concept
ORGANISATION RISK PROFILING (ORP) - AMO
Result column: "1" (Level 1), "2" (Level 2), "3" (Level 3) or "NA" according to POI/PMI assessment

Organisation Name:
RISK LEVEL / PROFILE
Organisation Risk Parameter
Level 3 (High) Level 2 (Average)

A. COMPLIANCE
AMO accepts regulatory access without question
1 and has an open, cooperative and transparent Strongly disagree Agree
attitude

Surveillance results are usually satisfactory and

2 any finding is analyzed and resolved in a timely Strongly disagree Agree
manner

The organization holds a certificate from

Maintenance organization approvals issued by The organization only holds an approval
only one foreign State (with a USOAP EI
3 local and other CAAs in good USOAP standing from local CAAs with a USOAP EI less
> 90% for AIR), besides the local CAA
than 90% for AIR.
approval.

No repeat findings or enforcement actions on the

4 Disagree
AMO have been issued since the last audit.

Not certified to an internationally Internationally Recognized Industry

Internationally Recognized Industry Standards
5 Recognized Industry Standards or Standards Certification for between 2 and
Certification (e.g., ISO Certification, etc)
certified for less than 2 years 5 years

B. ORGANIZATION

Number of years since the AMO was granted

1 Less than 2 years Between 2 and 5 years
approval by the State assessing the risk

Page 3 of 26
 RISK LEVEL / PROFILE
Organisation Risk Parameter
Level 3 (High) Level 2 (Average)

Duration of suspension of AMO approval by the

2 More than 1 year Between 6 months and 1 year
State assessing the risk

Key management personnel have remained stable

since the last audit (on-site and remote). This
includes accountable executive & a person or
3 Strongly disagree Agree
group of persons whose responsibilities include
ensuring that the maintenance organization
remains compliant.

4 Financial health of the organization TBD TBD

5 Addition of new rating since the last audit Aircraft Engine

C. SAFETY MANAGEMENT

Multiple sources of hazards (internal and

external) are considered and reviewed,
as appropriate.
There is a process that defines how
The data analysis process enables
Active Hazard Identification & Risk Assessment hazards are identified though reactive
1 gaining useable safety information.
(HIRA) Process and proactive methods.
Hazards are documented in an easy-to-
understand format.

The triggers and processes for internal

The triggers and the processes for
safety investigations are clearly defined.
2 Internal safety investigations internal safety investigations are not
Internal safety investigations are carried
clearly defined.
out and recorded.

Page 4 of 26
 RISK LEVEL / PROFILE
Organisation Risk Parameter
Level 3 (High) Level 2 (Average)

SPIs linked to the organization's safety

objectives addressing its top risks and
Safety performance indicators (SPIs) are
3 SPIs measuring the effectiveness of safety risk
not fully developed.
controls have been identified. State SPIs
are considered, as applicable.

SPIs are continuously monitored to

The processes of safety performance analyze the trends.
4 Safety performance monitoring and measurement monitoring and measurement are not fully The frequency and responsibility for the
developed. trend monitoring of SPIs are appropriate.
Realistic targets have been set.

There is an SMS training programme

being implemented that includes initial
and recurrent training, covering individual
safety duties (including roles,
The organization's SMS training
Personnel have received SMS training appropriate responsibilities, and accountabilities) and
5 programme has not been established or
to their roles and responsibilities how the organisation’s SMS operates.
has not been fully implemented.
Training material and methodology are
adapted to the audience and include
human factors.
All staff requiring training are identified.

Page 5 of 26
 RISK LEVEL / PROFILE
Organisation Risk Parameter
Level 3 (High) Level 2 (Average)

The organisation has established and is

implementing a change management
process to identify whether changes have
an impact on safety and to manage any
identified hazards and associated risks in
Changes which may impact the safety of products accordance with existing safety risk
and services are managed to ensure outcomes A change management process has not management processes. Triggers for the
6 are achieved with minimum adverse been fully established or the established change management process are
consequences process has not been implemented. defined. The process also considers
business related changes and interfaces
with other organisations/departments.
The process is integrated with the risk
management and safety assurance
processes.Responsibilities and timelines
are defined.

A reporting system, which covers both

mandatory and confidential reports, is
fully established, with process in place to
identify how reports are actioned, and
timescales are specified and addressed.
The reporting system is accessible for
A reporting system, which covers both and being used by all personnel.
7 A healthy reporting culture is in place mandatory and confidential reports, is not Responsibilities, timelines, and format for
fully established. the feedback are meaningful and well
defined.
Data protection and confidentiality is
ensured.
Staff are aware of and fulfil their
responsibilities in respect to the reporting
system.

Page 6 of 26
 RISK LEVEL / PROFILE
Organisation Risk Parameter
Level 3 (High) Level 2 (Average)

Safety promotion activities are fully

developed and means for safety
Safety promotion activities are identified and Safety promotion activities are not fully promotion activities are in place. The
8
resourced developed. organization monitors the effectiveness of
its safety promotion.

A process has been establised to

determine what, when, and how safety
information needs to be communicated.
The process includes contracted
organisations and personnel where
appropriate.The means of communication
An effective communication strategy is in place to The communication strategy to share are adapted to the audience and the
9 share outcomes of safety reports that are outcomes of safety reports is not fully significance of what is being
submitted. developed or implemented. communicated.
Safety critical information is being
identified and communicated throughout
the organisation to all personnel, as
relevant, including contracted
organisations and personnel where
appropriate.

Page 7 of 26
 RISK LEVEL / PROFILE
Organisation Risk Parameter
Level 3 (High) Level 2 (Average)

SUB - TOTAL
LEVEL 3 0
LEVEL 2 0
LEVEL 1 19
NA 0
Total No of Questions 19

ASSESSMENT RESULT
OPERATOR RISK PROFILE
Area Score Range
SCORES
A. COMPLIANCE (Average) 33
B. ORGANIZATION (Average) 33
C. SAFETY MANAGEMENT (Average) 33

ORP Categorization: Overall 33 33-49

Total Score ORP Category
33 – 49 A (Most Desirable)
50 – 63 B
64 – 77 C
78 – 91 D Overall Score Range ORP Category
92 – 100 E (Least Desirable)
33-49 A (Most Desirable)
50-63 B
64-77 C
78-91 D
92-100 E (Least Desirable)

Page 8 of 26
sment

Assessed By / Date:
RISK LEVEL / PROFILE Total Points
Weighting
Result (Manual (Result x
factor
Level 1 (Low) input) Weighting
(Optional)
factor)

Strongly agree 1 100% 1

Strongly agree 1 100% 1

The organization holds a certificate from

more than one foreign State (with a
1 100% 1
USOAP EI > 90% for AIR), besides the
local CAA approval.

Agree 1 100% 1

Internationally Recognized Industry

Standards Certification for more than 5 1 100% 1
years

More than 5 years 1 100% 1

Page 9 of 26
RISK LEVEL / PROFILE Total Points
Weighting
Result (Manual (Result x
factor
Level 1 (Low) input) Weighting
(Optional)
factor)

Less than 6 months 1 100% 1

Strongly agree 1 100% 1

TBD 1 100% 1

Parts 1 100% 1

The organisation has a register of the

hazards that is maintained and reviewed
to ensure it remains up-to-date. It is
continuously and proactively identifying
hazards related to its activities and the
operational environment and involves all
1 100% 1
key personnel and appropriate
stakeholders including external
organisations.
Hazards are continuously assessed in a
systematic and timely manner.

Internal safety investigations are carried

out by appropriately trained personnel to
identify root causes (why it happened, not
1 100% 1
just what happened). The identified
causal/contributing factors are
documented and are acted upon.

Page 10 of 26
RISK LEVEL / PROFILE Total Points
Weighting
Result (Manual (Result x
factor
Level 1 (Low) input) Weighting
(Optional)
factor)

SPIs are demonstrating the safety

performance of the organisation and the
effectiveness of risk controls based on
1 100% 1
reliable data.
SPIs are reviewed and regularly updated
to ensure they remain relevant.

The organization is continuously

monitoring its safety performance using
the established SPIs and SPTs. Where
the SPIs indicate that a risk control is
1 100% 1
ineffective, appropriate action is taken.
There is a proactive change management
process in place.

The SMS training programme is

delivering appropriate training to the
different staff in the organisation and is
being delivered by competent personnel.
SMS training is evaluated for all aspects
(learning objectives, content, teaching 1 100% 1
methods and styles, tests, etc.) and is
linked to the competency assessment.
Training is routinely reviewed to take
feedback from different sources into
consideration.

Page 11 of 26
RISK LEVEL / PROFILE Total Points
Weighting
Result (Manual (Result x
factor
Level 1 (Low) input) Weighting
(Optional)
factor)

The change management process is

used for all changes that may impact
safety, including human factor issues,
and considers the accumulation of
multiple changes. It is initiated in a
planned, timely, and consistent manner
and includes follow up action that ensures
1 100% 1
the change was implemented safely.
The change is communicated to those
affected.
Risk control and mitigation strategies
associated with changes are achieving
the planned planned outcomes with
minimu adverse consequences.

There is a healthy reporting system

based on the volume of reporting and the
quality of reports received.
Safety reports are acted on in a timely
manner.
Personnel express confidence and trust
in the organisations’ reporting policy and
1 100% 1
process.
The reporting system is being used to
make better management decisions and
continuously improve.
The reporting system is available for third
parties to report (partners, suppliers, and
contractors).

Page 12 of 26
RISK LEVEL / PROFILE Total Points
Weighting
Result (Manual (Result x
factor
Level 1 (Low) input) Weighting
(Optional)
factor)

The organisation puts considerable

resources and effort into and monitors the
effectiveness of its safety promotion.
1 100% 1
The organisation provides safety
promotion to its contracted service
providers, as well.

The organization analyses and

communicates safety critical information
effectively through a variety of methods
as appropriate to maximise it being
understood. 1 100% 1
Safety communication is assessed to
determine how it is being used and
understood and to improve it where
appropriate.

Page 13 of 26
RISK LEVEL / PROFILE Total Points
Weighting
Result (Manual (Result x
factor
Level 1 (Low) input) Weighting
(Optional)
factor)

Notes:

1 Risk level criteria descriptions/ figures are illustrative only, subject to

customization and validation of actual figures to be used.

2 Points to be allocated for each parameter assessed - namely 1, 2 or 3 for Level 1,

2 and 3 respectively. Weighting may be considered.

4 This Checklist assessment may be completed by assigned Inspector/ Surveyor

on scheduled basis (such as during organisation audit). He may need to liaise
with the service provider to obtain some of the data required.

5 This ORP assessment process may not be mandatory in view of those

parameters which are outside of normal regulatory purview eg staff turnover rate,
etc. It may be administered as a supplementary/ voluntary participation basis.

6 Total points achieved and its corresponding ORP Category (Cat A to E) to be

annotated. Results should be provided to the organization assessed.

7 Results of this ORP assessment may be correlated with other regulatory

inspection/ audit program findings to identify areas (organisations) with greater
concern or need as per ICAO Annex 19 requirements. Otherwise, notification of
ORP results to each organization alone may suffice as a mechanism to
encourage organizational behavior (safety culture) towards the desirable
category where applicable.

Page 14 of 26
2.1 Table A: Organization Size Categories
Number of Employees Category
1>10 1
Note: CAA can customize this Organization
11>50 2 Size Categories based on number of
51>200 3 locations (for Stations and Base) or
number of subcontractors managed by the
201>500 4
AMO, if applicable.
>500 5

Note: Category of aircraft represents the

aircraft for which the work was performed.
(special services, components, engine,
2.2 Table B: Service Complexity Matrix aircraft).
Scope of work Special
performed Components Engines Aircraft
Services

Category of aircraft a b c d

Takeoff mass less than

1 a1 b1 c1 d1
1200kg and max 4 occupants

Non pressurized and non

2 a2 b2 c2 d2
turbine powered

Pressurized or turbine
3 a3 b3 c3 d3
powered
Takeoff Weight Greater Than
4 a4 b4 c4 d4
5700 kg

2.3 Table C: Service Complexity Matrix Categories

Bands Service Complexity

a1,a2,b1 A
a3,b2,c1 B
a4,b3,b4,c2,c3,d1,d2 C
c4,d3,d4 D

Note: For example if you have an aircraft which is non

turbine and not pressurized but it is big (i.e. over
5700kg at takeoff) the most challenging complexity
level should be adopted.
2.4 Table D: Organization Size-Service Complexity Matrix
Size of Organization Service Complexity
(Number of
Employees) A B C D
1>10 1 A1 - 1/1 B1 - 1/1 C1 - 1/2 D1 - 1/2
11>50 2 A2 - 1/1 B2 - 1/2 C2 - 1/2 D2 - 2/2
51>200 3 A3 - 1/2 B3 - 1/2 C3 - 2/2 D3 - 2/2
201>500 4 A4 - 1/2 B4 - 2/2 C4 - 2/2 D4 - 3/2
>500 5 A5 - 2/2 B5 - 2/2 C5 - 3/2 D5 - 3/3

2.5 Table E: Organization Size-Service Complexity Combined Value Categories

Overall
Bands Value Complexity Score Range
Level

A1,A2,B1 1/1 1 30-60

A3,A4,B2,B3,C1,C2,D1 1/2 2 61-70

A5,B4,B5,C3,C4,D2,D3 2/2 3 71-80

C5,D4 3/2 4 81-90

D5 3/3 5 91-100

2.6 Table F: Overall Organization Complexity (Select the overall complexity level base
Overall Complexity Level 5
ue Categories

omplexity level based on the result from step 2.1 - step 2.5)
 COVID-19 Impact Checklist for AMOs
Notice: This checklist aims to assist operational and maintenance inspectors of the SoR with assessing the risk prof
respond to each statement indicating your level of agreement/disagreement based on information or evidence collec
conduct of on-site inspections (announced and unannounced visits), the review of documents submitted by the serv
and analyses of available safety information, remote and desktop surveillance activities. CAA can customize this che
utilization.
Result column:"3" (Level 3), "2" (Level 2) and "1" (Level 1) according to POI/PMI assessment result for "Strong

5
6
 COVID-19 Impact Checklist for AMOs
Notice: This checklist aims to assist operational and maintenance inspectors of the SoR with assessing the risk profile of an AM
respond to each statement indicating your level of agreement/disagreement based on information or evidence collected by var
conduct of on-site inspections (announced and unannounced visits), the review of documents submitted by the service provide
and analyses of available safety information, remote and desktop surveillance activities. CAA can customize this checklist with
utilization.
Result column:"3" (Level 3), "2" (Level 2) and "1" (Level 1) according to POI/PMI assessment result for "Strongly Disagre

Statement
(Guidance is provied under each statement and in the blue color.)

The AMO's availability of competent personnel and resulting AMO capabilities inaccordance with
the scope of approval have not been affected by the COVID-19 crisis.
· The AMO has implemented programmes to maintain maintenance personnel’s competencies at an
equivalent level to pre-COVID-19, using effective measures and quality assurance.
· The AMO continuously monitors, documents and evaluates its availability of competent personnel
and resulting capacity to provide services on a regular basis.

The AMO has implemented risk mitigations regarding granted exemptions.

· If the AMO has been granted an exemption on a particular subject, its documented mitigations
should be assessed to determine whether they meet SoR requirements.
· The AMO has documented all hazards and their associated risks introduced by existing
exemptions, along with the respective mitigations.
· The AMO has implemented quality assurance mechanism when alternative measures are taken on
recurrent training etc.
· No non-compliance has been observed.
Note: Quick Reference Guides (QRGs) have been developed by ICAO in addressing COVID-19 related
risks to the continuity of business and operations, while ensuring that the safety risks introduced by any
changes from alleviations are also addressed.
Link:https://www.icao.int/safety/COVID-19OPS/Pages/QRGs.aspx

The AMO ensures the compliance with requirements regarding its maintenance responsibilities.
· The AMO fulfills its maintenance responsibilities, with consideration of alternative measures.
· No occurrence related to the AMO’s maintenance responsibilities has been observed.
The AMO's availability of resources (in-house or contracted) to support maintenance has not
been impacted.
· The AMO regularly evaluates and document its availability of resources (in-house or contracted)
and take actions to ensure that maintenance safety is not impacted.
· The AMO can assure the available resources (in-house or contracted) to support maintenance
activities
· Correlation of the allocated resources with the activity needs is constantly ensured.

The AMO has implemented mitigating measures addressing the cumulative risk or compound
effect of the used of multiple exemptions
· The AMO has also identified, developed procedures and documented the cumulative risk or
compound effect of multiple exemptions, along with the respective mitigations.
· The AMO keeps monitoring and documenting the implementation and the effectiveness of each
mitigation.
· No occurrence related to use of exemptions has been observed.
The AMO regularly evaluates and acts to mitigate any impact on its activity due to required
preservation/de-preservation of aircraft, aircraft systems and components as a consequence of
mass parking and storage of aircraft during the COVID-19 crisis as well as their return to service
post the crisis.
· The AMO has assessed the availability of resources to ensure that the high volume and rapid rate of
aircraft entering and then exiting storage would not generate unaddressed technical issues
· The AMO ensures its maintenance responsibilities by adequately maintaining and protecting the
aircraft, safeguard against fuel contamination and wildlife ingress etc.

Total Score
 he risk profile of an AMO affected by COVID-19. Please
ence collected by various surveillance activities such as
by the service providers, meetings with concerned parties
mize this checklist with the subject matter expert before

for "Strongly Disagree", "Agree", and "Strongly Agree".

Strongly
Agree Strongly Agree
Disagree
3 2 1
 ORP Please use the matrix an
Organization Risk Profiling (ORP)
SCORE Table A: ORP-Complexity M
100
90A. COMPLIANCE 33
80
70B. ORGANIZATION 33
60
50C. SAFETY MANAGEMENT 33 Current ORP Result (%)
40 33
30Overall 33 33-49
20
10 50-63
0
A. COMPLIANCE B. ORGANIZATION C. SAFETY MANAGEMENT Overall 64-77
78-91
92-100

Organization Complexity
Level (Code)
Level (Code) Table B: Next Surveillance In
0 Organization
1 Complexity2level 3 4 5 5 ORP-S

Area SCORE RANGE LEVEL CODE

Current ORP Result 33-49 A 1D, 1E, 2C, 2D,
Overall Complexity 91-100 5

Table C: Frequency Modifier

Score Range
Note: Table C and the "COVID-19 Impack Checklist" is a
temporary and optional conponent of this risk profiling 06-09
tool. 10-12
13-15
16-18
ease use the matrix and modifier below to determine surveillance frequency
ble A: ORP-Complexity Matrix
Organization Complexity
30-60 61-70 71-80 81-90 91-100
urrent ORP Result (%) 1 2 3 4 5
A 1A 2A 3A 4A 5A
B 1B 2B 3B 4B 5B
C 1C 2C 3C 4C 5C
D 1D 2D 3D 4D 5D
E 1E 2E 3E 4E 5E

ble B: Next Surveillance Interval Criteria - CAA can customize the frequencies
Next surveillance
ORP-Surveillance Index Bands interval
1A 30 mths
1B, 2A 24 mths
1C, 2B, 3A 18 mths
1D, 1E, 2C, 2D, 2E, 3B, 3C, 3D, 4A, 4B, 4C, 5A, 5B 12 mths
3E, 4D, 4E, 5C, 5D 6 mths
5E 3 mths

ble C: Frequency Modifier with assessment result using COVID-19 Impact Chechlist - CAA can customize this modifier
Adjustment Based on Additional Assessment
Result using COVID-19 Checklist
No adjustment needed
Increase surveillance by 20%
Increase surveillance by 50%
Increase surveillance by 100%
this modifier