[Peetwo]

Q/ Has the Forsyth ASRR panel been deceived?

In recent days in the course of some PAIN research, I have been revisiting large parts of the Forsyth report. While trawling through the ASRR report I discovered a very disturbing discrepancy, one which seems to point towards  the good Rev Forsyth and his esteemed international panel being accidentally misled.

The following is taken from page 13-14 chapter 1 of the ASRR report:

1.3.6 2008 ICAO Universal Safety Oversight Audit Programme

In February 2008, Australia’s aviation safety system was reviewed against ICAO’s global Standards. The audit identified a need to improve Australia’s capacity to retain technical expertise within the safety regulator. The report noted a need for more formal training programs for inspectors.

Figure 3 shows that ICAO’s Universal Safety Oversight Audit Programme (USOAP) audit of Australia’s safety system found the level of effective implementation of ICAO’s Standards and Recommended Practices (SARPs) was above the global average. However, in the areas of legislation, licensing, operations and airworthiness, Australia’s implementation was slightly less than other leading jurisdictions such as Canada, the United States (US), the United Kingdom (UK) and New Zealand. Australia was only benchmarked as having a higher level of effective implementation than its counterparts for accident investigation.

Dear Rev Forsyth et.al - Figure 3 above does not in anyway reflect the true FAA/ICAO USOAP findings in regards to effective implementation of the 8 critical elements of an ICAO USOAP audit. The actual graphic depiction from the 2008 ICAO USOAP audit is correctly represented on page 84 of the audit report:   

The graphic the ASRR report depicts is taken from the adopted (2010) ICAO Continuous Monitoring Approach. The CMA is a fundamental part of the integration of the ICAO Annex 19 and effective implementation of a State SSP:

In 2010, the 37th Session of the Assembly adopted Resolution A37-5, affirming that the evolution of USOAP to the CMA should continue to be a top priority for ICAO to ensure that information on the safety performance of Member States is provided to other Member States and to the travelling public on an ongoing basis. This vital improvement to international aviation safety required the participation and support of all Member States, particularly during the two-year transition period (2011 - 2012) while the tools and guidance required for USOAP CMA were developed.

As of January 2013, USOAP CMA is fully launched and its processes and tools are constantly reviewed to enable a more information-driven, risk-based and result-oriented approach. The objectives of the CMA include: monitoring States' safety oversight systems using a web-based platform — the "Online Framework" (OLF); validating States' progress through various on-site and off-site validation activities; and continuing to assess the effectiveness and sustainability of States' safety oversight systems through audits. With the CMA, the USOAP has gained in cost-effectiveness, dynamicity and flexibility (with the introduction of activities of limited scope).

The USOAP CMA will continue to evolve in the next few years, in order to support State's efforts in implementing a State Safety Programme (SSP). ICAO initially published in 2014 "new Protocol Questions (PQs) on safety management" to be used for audits and other continuous monitoring activities conducted under the Universal Safety Oversight Audit Programme (USOAP) Continuous Monitoring Approach (CMA). These PQs are being used to conduct voluntary and confidential assessments of SSP implementation at the request of ICAO member States. In July 2017, amended SSP-related PQs will be published by ICAO to reflect Amendment 1 of Annex 19, the 4th edition of the SMM and lessons learned to date.  Although Amendment 1 to Annex 19 does not become applicable until November 2019, select States will be approached by ICAO with a view to performing audits including the amended SSP-related PQs in 2018 and 2019 on a voluntary but non confidential basis. As of 2020, ICAO will perform audits using the amended SSP-related PQs on the States which will meet the criteria to be established by ICAO, in line with the GASP.

You can see that the ASRR graphic, for Australia at least, can be exactly replicated in the ICAO safety audit information webpage:

MTF...P2