ACCEPTANCE TESTS. design tests qualification tests production acceptance tests product reliability tests - producer - customer

Transcription

1 ACCEPTANCE TESTS Reference: Robert W. Smiley, "Acceptance Testing", in Handbook of Reliability Engineering and Management, W.G. Ireson and C.F. Coombs, Jr., editors, M c Graw-Hill Book Co., Chapter 8, pp , New York, Test Program: feasibility tests design tests qualification tests production acceptance tests product reliability tests - producer - customer Within the context of the third and fourth year design courses, the students are likely to be involved in the first 3 types of tests. With regard to feasibility tests, it may be necessary, for example, to conduct small experiments to better understand the characteristics of an input signal, or it may be desirable to test a small portion of a possible solution to ensure that conceptually it could work. Design tests would involve perhaps computer simulations as part of the synthesis/evaluation loop at either the system design or bottom-block design levels. Design reviews would also have components of evaluation or testing. Qualification tests would be those acceptance tests in the requirement specification that are used to verify that the engineering prototype meets the specifications. This is the connotation with which the term acceptance tests has been used in the course notes for EE 395 and EE 495. In the context of industrial production, obviously the latter two types of test are very important. Production acceptance tests would be used to ensure that the production level device, as opposed to a preproduction prototype, or system still meets the requirement specifications, and the reliability tests are used throughout a production run to ensure that the product continues to meet the specifications and to allow the manufacturing process to be adjusted to maintain compliance with specifications. "The test program must begin with and proceed concurrently with the development of the product..." (p 8.2, Smiley). This ensures that: 1. test points are available, 2. the test program is adequate to verify the requirement specifications, 3. any necessary custom test equipment is designed, built, and ready on schedule, 4. the testing protocol and test equipment are compatible with each other and with tests performed during qualification. 1

2 Kinds of Acceptance Tests The following list is taken from p 8.2 of Smiley: 1. Variables versus attributes testing 2. Destructive versus nondestructive testing 3. Ambient versus environmental testing 4. Level of tests 5. Acceptance testing versus quality control 6. Production assessment testing 7. Screening and burn-in testing 1. Variables versus Attributes Testing: - variables testing measures and records the actual value of a parameter - attributes testing measures a parameter to determine whether or not it is within an acceptable range and then makes a go/no-go decision. Advantages of Variables Testing: 1. allows statistical evaluation of parameters, 2. allows trend analysis to adjust manufacturing process before acceptable limits are reached, 3. extracts maximum amount of information, 4. cost may not be too much greater for automatic test equipment, and 5. smaller sample size can be used for any given confidence level. Advantages of Attributes Testing: 1. excellent for separating acceptable from unacceptable, 2. manual testing is much cheaper, automatic testing somewhat cheaper than variables testing, and 3. manual testing can be done with lower-skilled labor. The selection of which parameters and/or attributes to test is not a simple problem. Various factors are involved: 1. the need to demonstrate functionality under all use conditions, 2. reliability requirements, 2

3 3. the cost of testing including the test equipment, 4. the time required to perform each test, 5. the equipment and personnel available, 6. customer requirements and government regulations, 7. the need to assure spare part interchangeability, 8. the level of desired quality control, and 9. the cost of, and the cost of replacement of, the part tested. 2. Destructive versus Nondestructive Testing: Destructive testing results in a product unfit for delivery to the customer but allows testing of hardware to the limits thus allowing analysis of failure modes and other reliability factors. Economic factors are involved in the trade-off between the two: 1. the number of tests required to attain the same level of confidence, i.e., nondestructive requires more tests, 2. the value of the hardware being tested, 3. the cost of a failure during regular operation, 4. the ability of nondestructive tests to measure the function-related parameters, and 5. the investment and operating costs of the testing equipment. 3. Ambient versus Environmental Testing: The main concern is the cost and complexity of full environmental tests versus the need to ensure compliance with the specifications. Preferable to find a correlation of parameters measured at ambient with parameter values ate the extreme ends of the ranges. Then ambient testing with more stringent limits may be sufficient. 4. Level of Tests: That is, testing on individual pieces versus testing on subassemblies or final product. High-level tests: 1. give the maximum assurance no defects have been introduced, 2. can be used for burn-in to eliminate "infant" mortality, 3

4 3. have costs which are lower because more attributes can be tested simultaneously, 4. spare parts can be tested at their assembly level, and 5. indicate adverse tolerance build-ups. (See Figure 8.7 p 8.14 Smiley for an example of a "funnel of tolerances".) Low-level tests: 1. some important attributes may become inaccessible at higher levels of assembly, 2. eliminates defects before assembly costs have been incurred, 3. subassemblies produced out-of-company should be tested prior to shipment if shipping costs are high, and 4. purchased material should be tested to ensure that requirements are met. 5. Acceptance Testing versus Quality Control Testing: - purpose of acceptance testing: 1. demonstrate compliance with functional requirements and 2. acquire data to allow reliability evaluation of population. - purpose of quality control testing: 1. eliminate defective material prior to use and 2. to generate data to detect undesirable manufacturing variations. Some tests may satisfy both testing strategies, but they may be planned differently. Acceptance testing may be forced by contract with the customer or by government regulations. Quality control testing is decided upon based on the cost/benefit analysis of testing at low levels versus testing of completed assemblies and rejecting defects at the highest level. 6. Production Assessment Testing: This is environmental testing of small samples of a production batch and complements ambient acceptance testing program on the total production population. 1. usually done on periodic samples drawn from a production run. The time between samples is sufficient to allow completion of the environmental test. 2. usually done at higher levels of assembly, 3. are nondestructive so the sample can be delivered to the customer. 4

5 7. Screening and Burn-in Testing: Often done with vibration and/or temperature cycling to accelerate mechanical failures, such as poor solder joints, and to stress electronic components. This is done to accelerate the earlier defects (infant mortality) of a typical "bath tub" type lifetime reliability performance. Testing Facilities In-House versus Contracted Testing: - factors favoring in-house testing: 1. easier co-ordination with manufacturing schedules, 2. capital asset utilization, 3. easier to control the quality of testing, 4. flexibility of test schedules, 5. minimizes hardware transportation and associated damage, 6. hardware is always under company control, and 7. reduces opportunities for industrial espionage. - factors favoring contracting out: 1. technical capability and personnel skills may not be available, 2. total cost may be less, 3. capital investment for in-house testing may be prohibitive, e.g., large size environmental chamber, 4. customer may require third-party testing, 5. may be used for peak-loading situations, and 6. used to maintain outside back-up capability in case of in-house planned or unplanned unavailability, e.g., labor dispute or fire. 5