IS-BAO Audit Procedures Manual 2015, Covering Letter

Size: px

Start display at page:

Download "IS-BAO Audit Procedures Manual 2015, Covering Letter"

Logan Lucas
5 years ago
Views:

1 IS-BAO Audit Procedures Manual 2015 Covering Letter January 1, 2015 This is the January 1, 2015 revision to the IS-BAO Audit Procedures Manual. It is recommended that you review both the IS-BAO Audit Procedures Manual and the IS-BAO. To help with your review of the amendments the IS-BAO and IS-BAO Audit Procedures Manual in Revisions Mode are included in the download. The following documents are available on the IBAC website for download (a username and password issued by IBAC are required): IS-BAO Audit Procedures Manual 2015, IS-BAO Audit Procedures Manual 2015 in Revisions Mode, IS-BAO Audit Forms 2015, IS-BAO 2015, IS-BAO 2015 in Revisions Mode, IS-BAO Internal Audit Manual 2015, SMS Toolkit; Risk Analyses Guidelines. IS-BAO Auditors are granted permission to reproduce the documents from the download for their use in carrying out IS-BAO related activities. Auditors are requested to review their listing on the IBAC web site at to ensure that their contact information and web site/resume links are correct. If there are any corrections please advise the IBAC Administrator at or the Audit Manager at IS-BAO Newsletters are published and posted on the IBAC web site at Whenever this is done an message is sent to each IS-BAO holder. In order to ensure that these messages and other communications are sent to the correct addressee, all IS-BAO holders should complete the IS-BAO Amendment Info Form posted on the IBAC website at During 2015, Fundamentals of IS-BAO Workshops and IS-BAO Auditing Workshops are being held at various locations. They are designed to assist operators and auditors with the implementation and registration process. Check the IBAC web site at IS-BAO Workshop Schedule. A summary of changes to the IS-BAO Audit Procedures Manual 2015 are provided in the following table. 1

2 Because: 1) There are concerns about auditor qualification and competency, 2) auditor performance was identified as the program's biggest risk in a study conducted in 2009, 3) we need to augment the staff to replace retirees, and 4) the audit manager is putting full time effort to process audits, the Program is creating a position to manage auditor accreditation. This person will ensure auditors are properly qualified, trained, competent, and current. Therefore, any reference to the audit manager in the APM regarding the management of auditors should be changed to this new title, Auditor Accreditation Manager. The operator stage should be listed on the certificate. APM 3 APM 3.1 Concept supported by the IBAC Director General. Info: ASQ sets currency requirements for auditors. IS-BAO may benefit from this info. The stage is not indicated on the certificate. Very little, if any, value would be added to the program and industry by adding this information to the certificate. One key risk of adding this information is marketing wars by operators comparing their safety ability versus another operator. This could increase risk to the program. Also, this paragraph states extra certificates are provided at no cost. This practice should be discontinued. Change references as suggested. Modify the last sentence of to read: A second certificate or additional certificates to be carried on board aircraft as proof of registration are available from IBAC. Fees may be applicable. No change regarding the stage on the certificate. However, agree with the modification of to clarify there may be fees for additional true certified copies. 2

3 IBAC will retain audit reports in a secure location and they will be made available only to IBAC staff and the Standards Board Does the Standards Board need to have access to actual audits? The idea of confidentiality would be more sound if only IBAC Staff had access to the audits. I have been an accredited IS- BAO Auditor since 2008 and I have a question for you regarding the current auditor renewal policy. I understand and agree with your reasoning for requiring 24 month auditor renewals, but I don't understand why Auditors aren't allowed to renew upto 3 months early and still retain their original renewal date as is offered to our Operators. This grace period would allow many Auditors the opportunity to select a renewal class location that is closer to their domicile and thus reduce expenses. Please consider this policy at the upcoming IS-BAO Board meeting. APM 3.12 APM 3.5 Operators expect confidentiality. The IBAC staff needs to have access to the audit reports to carry on the mission of IBAC. However, the standards board does not have a need for this access. The IS-BAO program will be modifying the registration process for The auditor will pay for and attend workshops. The fee for the workshop is the same for all attendees. After attending the workshop, the auditor will have up to three months to apply for accreditation or renewal. The registration process will have a fee associated with it that is approved by the IBAC Governing Board. This change will address the concern identified by this suggestion. Delete "and the Standards Board" from Auditor registration process for 2015 will address this concern. No Change Update the APM as needed to reflect the new process for auditor accreditation application. 3

4 The ISBAO certificate of registration does not indicate the type of operation they have been audited against. What if the operation changes significantly after the audit, i.e. from day VFR domestic operations only, to global IFR operations. IBAC should know when an audit is going to take place to ensure the following: 1. the auditors are properly qualified, 2. the auditors are current, 3. the auditors are competent/capabable of handling the scope of the audit, 4. to facilitate a safety culture survey for stage 3 audits of needed. APM 4 APM 4.3 A properly functioning SMS will allow for changes in operations, to include mission and equipment. The IS-BAO management team consensus is to maintain the current process. Some audits have been submitted that reveal the following issues: 1) auditor was not current or qualified, 2) the scope of the audit was not appropriate, 3) the time spent on sight was insufficient, 4) not enough people were interviewed, 5) one person tackled a job that required a team for the time allotted, 6) the operator had not purchased the requisite materials prior to the audit (i.e. IS-BAO manual), 7) other issues. These could have been avoided by being proactive. If IBAC were notified that a audit was being planned and the lead auditor had to fill out a form on the IBAC website that would prevent the issues listed above, it would improve the quality of the audits. No change required. Add APM to state: Once a registration audit is scheduled, the auditor shall send the following information to Audit Manager at auditmanager@ibac.org as soon as practical: a. The planned date of the audit, b. The name of the operator,c. The planned scope, full system or partial and SMS Stage,d. Names of other audit team members,e. How many days onsite planned, andf. The number of interviews planned., except delete "f". 4

5 The members of the NBAA safety committee s positive safety culture sub-committee are currently developing a set of tools to assist corporate flight departments assess, evaluate and design a positive safety culture that supports managing daily risks to a level that is as low as reasonably practicable. A key component as identified by this working team is to partner with the IS-BAO auditing process to suggest incorporating a more defined means to measure a flight departments safety culture especially for a Stage III audit. APM sections and need to indicate that Stage 2 and 3 audits must contain substantive auditor comments regarding the operator's SMS. These comments should provide examples of the operator's methods of conforming to the standards. APM 5 APM Section of the IS-BAO audit procedures manual defines the aim of a stage three audit and specifically requires the confirmation that a positive safety culture is sustained and safety management activities are fully integrated into the operators business. It is the NBAA safety committee s observation that the existing IS-BAO audit guidelines for measuring these important criteria are not adequately defined to provide a valid conclusion that is not open to auditor bias. Many audits are submitted without adequate comments to support the recommendations for stage 2 and stage 3. Modify the Audit Report Form Stage 3 Audit Definition to include "Applicable Performance Indicators are observed as being effective (See APM 5.8.3)" and modify APM to include requirements to align with this. Add Section to state: "Stage 2 and Stage 3 Audits must contain substantive auditor comments to validate SMS performance. These comments should provide examples of qualitative and/or quantitative methods to verity effectiveness". 5

6 APM Figure 6.1 needs to be modified to show Action Required as a NC APM 6.1 "Action Required" is a form of Non-Conformity, although the current diagram does not make that clear. Modify the diagram to indicate: Non-Conformity types: (1) "Action Required" - Low risk to safety. Remedial action plan is NOT required. Documented in the "Analysis of Non-Conformities" block at the end of the applicable protocol chapter. (2) Minor Finding - Low risk to safety, yet urgent attention warranted and a remedial action plan IS required. (3) Major Finding - Significant risk to the safety of the organization. A remedial action plan is required and must be successfully implemented before registration can be awarded. Delete "Action Required" verbiage. Make three levels of assessment: Observation, Minor Finding, Major Finding. Change APM to reflect accordingly. 6

7 APM is a bit ambiguous. Auditors suggest clarification to IS-BAO Policy , which seems to suggest that at least 10% of managed aircraft clients should be interviewed during an audit. Confidentiality requirements in the current business aviation environment make this rather burdensome, and reading the policy may instead be suggesting that personnel at various locations where aircraft may be remotely managed should be interviewed (as was done in this case with remote bases) APM APM Appendix doesn't provide guidance on the latest reasonable time to submit an audit to IBAC. It is also lacking in information regarding the remedial action plan closure requirements. The policy letter and Appendix C make it clear that 10% is referring to the number of clients to be interviewed. Therefore, if the Mgt Co is serving 20 aircraft owners, then the auditors would need to at least visit 2 owners to interview their personnel. Modify the last paragraph to read: The Audit Report, Finding Forms, and Protocols must be filled out completely, incorporating explanatory notes as necessary. The Finding Forms shall include clear remedial action plans. However, remedial action plans related to minor findings do not have to be implemented before the audit is submitted. The auditor shall not delay submitting the audit to IBAC if all the findings are only minor. Signed copies of the original completed forms will be left with the operator. Electronic transmission (.pdf format without document password protection) is the preferred method. Integrate PL into the APM and suspend this policy letter 1 Jan

8 3.2.1d Recommend the following protocols are used to evaluate the ERP: Does the organization have an emergency response/contingency plan appropriate to the size, nature and complexity of the organization? Does the emergency/contingency plan address all possible or likely emergency/crisis scenarios relating to the organization s aviation product or service deliveries? Protocols 3 This input brings to light a bigger issue. The appropriateness and effectiveness questions on the protocols are not reflected the IS-BAO and should be. Change the appropriateness and effectiveness questions on the protocols for ERP coordination to reflect ICAO Doc Also, add appropriateness and effectiveness performance indicators as notes under the applicable standard. and add protocol questions regarding "Appropriateness" and "Effectiveness" to the IS- BAO under the applicable standard as notes. Consideration should be given to changing the title of section 16 from the existing in-flight inspection to In-flight company procedures review. The term inspection can denote from a legal perspective a higher level of required diligence on the auditors versus what the possible real intent is; a review of the company s procedures to verify if they follow the procedure they have documented in internal guidance documents. Protocols 16 Survey was sent to 7 auditors that are confirmed on the AAG asking if the Protocols for Inflight Inspection should be removed and possibly inserted in the IG under Chapter 6 to support an internal standardization program. 5 responded. 4 wanted the Inflight Inspection Removed, 1 recommended they stay. IS- BAO Mgt Team supports this consensus. Recommend Section 16 Protocols be removed and placed in the IG under chapter 6 to provide guidance for an internal standardization program. 8

IS-BAO Audit Procedures Manual 2015

IS-BAO Audit Procedures Manual 2015 Suite 16.33, 999 University Street Montreal, Quebec, H3C 5J9, Canada Tel 1-514-954-6198 Fax: 1-514-954-6161 www.ibac.org IS-BAO Audit Procedures Manual Copyright All