PLANNING A PROPERTY MANAGEMENT SYSTEM AUDIT (PMSA)

Transcription

1 TLO 21 IND LESSON # 21 PLANNING A PROPERTY MANAGEMENT SYSTEM AUDIT (PMSA)

2 LESSON #21 PLANNING A PROPERTY MANAGEMENT SYSTEM AUDIT (PMSA) TLO 21-Given a contractor possessing contract Government property, describe the concepts involved with planning a Property Management System Audit (PMSA). ELOs: 1. Explain terms (property management systems audit (PMSA), standard audit, limited audit, process, defect, population, sample, sample item, confidence levels, choose a random number using a randomizer e-tool. 2. List the ten steps in the commercial audit process, in order. 3. Describe the underlying concept of each step. 4. Cite the reference for the Government authority to audit the contractor. 5. Cite the reference requiring the Property Administrator to perform a Property Management System Audit. 6. Describe the types of Property Management System Audit. 7. Describe when to use the two types of Property Management Systems Audit. 2

3 LESSON #21 PLANNING A PROPERTY MANAGEMENT SYSTEM AUDIT (PMSA) ELOs (cont.): 8. Explain the frequency requirement for performing a Property Management System Audit (PMSA). 9. Explain the FAR requirement for scheduling/planning a PMSA 10. Describe the Property Administrator s advance notification requirement to the contractor. 11. Explain the requirement for a PMSA Entrance Conference/Interview. 12. Identify data collection techniques and methods. 13. Define systemic and non-systemic findings. 14. Describe the two statuses of a PMSA (approved or disapproved). 15. Describe the distribution of a PMSA Summary 3

4 Property Management System (PMS) Audit Definitions: Property Management System Audit - A systematic objective review and evaluation of a Contractor s property management system (PMS) including both the procedures and the application, implementation and compliance with: The Government Property Clause The terms and conditions of the contract, and Any customary commercial practices, voluntary consensus standards, or industry leading practices selected by the Contractor for incorporation and application in the PMS. Standard Audit - A systematic review in accordance with GAGAS of the Contractor s PMS where all applicable Outcomes/Processes, Process Segments, and Criteria are reviewed. This may take place through a plant visit for itinerants or where a PA is resident at the Contractor site.. 4

5 Limited Audit - This is sometimes referred to as a desk audit because it involves a systematic review of the Contractor s PMS generally without the benefit of a plant visit. A limited audit may be done with the assistance of other Government technical representatives, e.g., Quality Assurance Representative, Industrial Specialist, etc. A plant visit may be considered necessary by the PA based upon risk factors and assessment of these factors. Process - An Outcome or segment of a Contractor s Property Management System based upon the life cycle approach set forth in the Government Property Clause at FAR (f). Each process is a unique segment of the PMS established by the FAR and consist of: Acquisition, Receipt (identification), Records, Physical Inventories, Subcontractor Control, Reports, Relief of Stewardship Responsibility, Utilizing Government Property (consume, move, store), Maintenance of Property, and Property Closeout (property disposal). 5

6 Defect - A deficiency such as a lack of contractually required data, erroneous data, inadequate areas or controls, untimely data, etc. Population - An aggregation of documents or records, or even physical assets, or possibly even actions selected for review due to their common characteristics. The term population is also known as a lot. Selecting the correct population is a very important part of the audit because the items select must have common characteristics in order for the results to be valid. Sample - A number of items such as documents, records, articles, or actions selected from a population/lot for a review in order to draw inferences regarding and generalizable to the status of the population. This is one of the more critical aspects of performing the audit because an inappropriate sample can lead to audit bias resulting in the findings being ruled invalid. 6

7 Sample Item - A single document record, article, asset or even an action from the Sample. Confidence Level - The confidence level or reliability is the probability that a sample will accurately represent the population from which it was selected. For example, a sample with a 95% confidence level would be said to accurately reflect the population 95% of the time. That is, if 100 samples were selected from the population, 95 of those samples would accurately reflect the population. (Flesher 1996). Segment - A subdivision of a Process. Since not all property and not all processes surrounding that property are the same, Process Segments were established to assist in the proper framing of a population under a process. These Process segments assist the PA in selecting the proper population for sampling driven by their common characteristics. 7

8 Criteria/Criterion - Is an evaluative item of a PMS Process or Process segment subject to audit and analysis by the appropriate method. Specifically, criteria are the questions that are posed of the Contractor s PMS. Many of these questions involve more complex answers than a simple yes or no. They require an analysis of the data embedded in the question. For example, Property control records conform to the FAR this question requires an analysis of the record card (or electronic records) to assure that the basic information required by the FAR is provided on that record. See FAR (f)(iii) plus any additional information required by customary commercial practices, voluntary consensus standards, or industry leading practices. In general, a yes or no answer is inappropriate for most criteria because it does not provide any audit evidence. 8

9 PROPERTY MANAGEMENT SYSTEM (PMS) The system that identifies a contractor s internal management program encompassing the protection, preservation, accounting for, and control of Government Property from acquisition, disposition through contract closeout. Contractor s Property Management System 9

10 FAR (b) Property Management (1) The Contractor shall have a system of internal controls to manage (control, use, preserve, protect, repair and maintain) Government property in its possession. The system shall be adequate to satisfy the requirements of this clause. In doing so, the Contractor shall initiate and maintain the processes, systems, procedures, records, and methodologies necessary for effective and efficient control of Government property. 10

11 FAR CCP-VCS-ILP The Contractor may employ customary commercial practices, voluntary consensus standards or industryleading practices and standards that provide effective and efficient Government property management that are necessary and appropriate for the performance of this contract (except where inconsistent with law or regulation). 11

12 CCP-VCS-ILP It is a standard practice for contractors to have written procedures available for its employees, its internal corporate auditors, and external auditors. Contractors are allowed to select the customary commercial practices, voluntary consensus standards or industry-leading practices and standards that are best suited to meet the compliance requirements of the contract. 12

13 These written procedures serve as a standard of performance for the Contractor s processes, systems, procedures, records, and methodologies. Contractor employees are expected work to this standard. Internal and External auditors will periodically examine the Contractor s Property Management System to see how well the Contractor complies with their written procedures. With well established standard one can readily determine if the Contractor meets or exceeds it as well as when they are non-compliant. 13

14 What are some of the contract requirements that the Contractor must include in their Property Management System? As long as the Government Property clause is included in the contract the Contractor must include procedures for the outcomes/processes in FAR (f)(i to x). All of them can be audited. What are these outcomes/processes? (see next slides) 14

15 15 Audit Outcomes/Process 1. Property Management 2. Acquisition 3. Receiving 4. Identification 5. Records 6. Movement 7. Storage 8. Physical Inventories 9. Reports 10. Consumption 11. Utilization 12. Maintenance 13. Subcontractor Control 14. Disposition 15. Contract Closeout 15

16 SYSTEMS ANALYSIS (g) Systems analysis (1) The Government shall have access to the contractor's premises and all Government property, at reasonable times, for the purposes of reviewing, inspecting and evaluating the Contractor's property management plan(s), systems, procedures, records, and supporting documentation that pertains to Government property. This access includes all site locations and, with the Contractor s consent, all subcontractor premises. Scheduling and performing an audit is dependent upon the risk the Contractor s PMS poses to the Government. As the risks increase the urgency and frequency of the audit will increase. 16

17 The Audit Process: Scheduling and Planning Notification Entrance Conference Perform the audit Exit Conference Corrective Action Re-audit 17

18 Scheduling At the beginning of each fiscal year, the PA shall prepare a schedule showing the names of the Contractors and the projected dates on which each system AUDIT shall take place. 18

19 Planning The PA shall prepare an AUDIT plan that includes the following: 1. Identification and listing of: Processes Process segments Criterion that will be used to evaluate the Contractor s Property Management System -- USE FAR for Outcome/Process Requirements 19

20 Planning (cont) 2. A listing of the estimated line items by classification: Material, Equipment, Special Tooling, and Special Test Equipment 3. A review of the PMS written procedures to make sure that they are current, accurate, adequate & compliant with contract requirements. 20

21 Planning (cont) 4. Audit files shall contain sufficient narrative and documentation reflecting rationale for deferring accomplishment of the process or process segments 5. System Audit Work Papers* *NOTE: System Audit Work Papers should be prepared before the audit is actually performed. 21

22 Notification The PA shall notify the contractor in writing of the planned dates for the system AUDIT no later than 30 days before the commencement of the review. GAGAS REQUIREMENT 22

23 An entrance conference shall be held with Contractor Managerial Personnel to inform the Contractor of the scheduled system audit, timeframe for performance, processes subject to review, and other pertinent items GAGAS Requirement 23

24 Audit Protocols (10 Steps): 1. State the objectives of the audit 2. Define the Attributes and Transactions 3. Define the Population and/or Sampling Unit 4. Specify the Acceptance and/or Rejection Rate 5. Determine sample size 6. Randomly select sample 7. Perform Audit 8. Analyze Defects and/or Deviations 9. Generalize from Sample to Population 10.Determine status of processes and process Segments 24

25 1. Objectives of an Audit Determined by Outcome/Process subject to review What are you checking? a. Acquisition of Property b. Receiving of Property c. Records of Government Property d. Physical Inventories e. Subcontractor Control f. Reports g. Etc 25

26 2. Define the Attributes and Transactions What Data Items of this Process will you check for compliance to the contract requirements (as well as standards established in the Contractor s Property Management System)? Date, was it timely? Quantity? Record Accuracy? Etc. These items need to be established before the review 26

27 3. Define Population and Sampling Unit Based upon: Process Process Segment Criteria Homogeneity: items in the population have common characteristics Timeframe NOTE: Population Selection is extremely important! 27

28 4. Specify the Acceptance and Rejection Rates The Department of Defense allows the use of variable confidence levels 90%, 95%, or 97% Confidence Level Accept-Reject Rates for 1 item to 100,000+ The Double Sampling Plan allows second chance to accept. 28

29 5. Determine the Sample Size for the desired Confidence Level Sample Size depends upon the lot size as and the Confidence Level!! This is for a 90% Confidence Level 1-18 Select All Select Select Select Select ,000 Select 34 29

30 6. Select Sample Randomly Valid Statistical Procedures require Random Sampling Random Sampling protects against bias Random selection of the sample allows the results to be generalized to the population 30

31 6. Select Sample Randomly (cont.) Random numbers may be selected by using: Random Number Table Computer generated random numbers Random numbers are available on the internet RANDOMIZER.ORG 31

32 7. Perform the Audit Procedure Use: Populations Samples Selected Criteria Work Papers Audit Evidence (Records) 32

33 8. Analyze Defects A defect is a condition in which a process, a sample item, or sample item element of a Contractor s PMS contains one or more deficiencies. 33

34 8. Analyze Defects (cont.) Systemic Defects: These are serious defects that have a negative impact on the Contractor s Property Management System. Non Systemic Defects: These are minor defects and have a minor impact on the Contractor s Property Management System. 34

35 9. Generalize from the Sample to the Population If sample results are good, is the system good? If the sample results are bad, is the system bad? Refer back to #8, Analyze Defects: are they systemic or non-systemic? 35

36 10. Determine the status of the Process, Process Segment, or Criterion Do they comply with the established standards in the Contractor s Property Management System? Is Corrective Action Needed? Correction of System Defects Correction of disclosed items with no systemic correction Non-Compliant Compliant 36

37 Exit Conference with the Contractor Upon completion of the System Audit, the PA shall conduct an exit conference with the Contractor s Managerial Personnel to discuss the overall results of the System Audit GAGAS Requirement 37

38 Exit Conference with the Contractor (cont) This conference must also address any process or process segment in which adequacy of controls, procedures, or the application thereof was found unsatisfactory. 38

39 Exit Conference with the Contractor (cont) The PA shall advise the ACO/CO where corrective action is required. The ACO/CO should request corrective action from the contractor. Agreement should be reached during the exit conference that corrective measures will be taken. 39

40 Corrective Action Plan Contractors are required to correct any deficiencies found during the PM System Audit. A reasonable time shall be allowed for the correction of deficiencies. The corrective action taken by the Contractor will be as specified by the schedule within the corrective action plan. FAR (g) Corrective Action Plan 40

41 FAR (g) Systems Analysis (3) Should it be determined by the Government that the Contractor's (or subcontractor s) property management practices are inadequate or not acceptable for the effective management and control of Government property under this contract, or present an undue risk to the Government, the Contractor shall prepare a corrective action plan when requested by the Property Administrator and take all necessary corrective actions as specified by the schedule within the corrective action plan. Please Note: DCMA requires their Property Administrators to first present the audit evidence for corrective action to the Contracting Officer. The CO (not the PA) will request corrective action from the Contractor. 41

42 Corrective Action Plan (cont.) Contractors shall develop a Corrective Action Plan that shows milestones and timeframes for the correction of the deficiencies 42

43 Two products result from a Property Management System Audit: 1. PMSA Report 2. PMSA Audit Summary 43

44 System Audit Report This is a TECHNICAL DOCUMENT for FILE and is not for distribution! CRUCIAL POINT!!! To support your PMS AUDIT FINDINGS you must include the APPROPRIATE SUPPORTING DOCUMENTATION in the System Audit Report This is: THE HARD AUDIT EVIDENCE supporting audit team findings! FOCUSES ATTENTION on CONTRACTOR PMS DEFICIENCIES to ENABLE CORRECTIVE ACTION SOLUTIONS! 44

45 System Audit Report (cont.) CRUCIAL POINT!!! Supporting your AUDIT FINDINGS must be an ABSOLUTE!! You MUST be able to demonstrate how you defined your Population, Selected your Samples and applied the criteria Your Work Papers MUST permit traceability after-the-fact Your Audit Evidence MUST withstand scrutiny ACID TEST Can you defend your findings in court or before the ASBCA? 45

46 System Audit Summary - At the conclusion of each property system audit, the PA shall prepare a written summary of findings: It is an Executive Level document, Written clearly to convey the results of the audit in general terms, & Supports continued adequacy of the system and/or defects identified and their impact on system status (Approved or Disapproved) The Summary specifically contains and/or addresses: Introduction Overview Methodology General Findings Contractor Actions Taken To-date Distribution (Who gets copies of the System Audit Summary?) 46

47 LESSON #21 PLANNING A PROPERTY MANAGEMENT SYSTEM AUDIT (PMSA) TLO 21-Given a contractor possessing contract Government property, describe the concepts involved with planning a Property Management System Audit (PMSA). ELOs: 1. Explain terms (property management systems audit (PMSA), standard audit, limited audit, process, defect, population, sample, sample item, confidence levels, choose a random number using a randomizer e-tool. 2. List the ten steps in the commercial audit process, in order. 3. Describe the underlying concept of each step. 4. Cite the reference for the Government authority to audit the contractor. 5. Cite the reference requiring the Property Administrator to perform a Property Management System Audit. 6. Describe the types of Property Management System Audit. 7. Describe when to use the two types of Property Management Systems Audit. 47

48 LESSON #21 PLANNING A PROPERTY MANAGEMENT SYSTEM AUDIT (PMSA) ELOs (cont.): 8. Explain the frequency requirement for performing a Property Management System Audit (PMSA). 9. Explain the FAR requirement for scheduling/planning a PMSA 10. Describe the Property Administrator s advance notification requirement to the contractor. 11. Explain the requirement for a PMSA Entrance Conference/Interview. 12. Identify data collection techniques and methods. 13. Define systemic and non-systemic findings. 14. Describe the two statuses of a PMSA (approved or disapproved). 15. Describe the distribution of a PMSA Summary 48