C.P. 6079, succ. Centre-Ville, Montral, Québec, Canada H3C3A7 c STMicroelectronics, 850 Rue Jean Monnet, Crolles, France

Transcription

1 This article was downloaded by: [Ecole Polytechnique Montreal] On: 19 December 2011, At: 07:15 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK International Journal of Production Research Publication details, including instructions for authors and subscription information: Quality and exposure control in semiconductor manufacturing. Part II: Evaluation Belgacem Bettayeb a, Samuel Bassetto b, Philippe Vialletelle c & Michel Tollenaere a a Grenoble-InP, G-SCOP Laboratory, Grenoble University, 46 Avenue Félix Viallet Grenoble, France b Ecole Polytechnique de Montréal, Department of Mathematics and Industrial Engineering, C.P. 6079, succ. Centre-Ville, Montral, Québec, Canada H3C3A7 c STMicroelectronics, 850 Rue Jean Monnet, Crolles, France Available online: 08 Dec 2011 To cite this article: Belgacem Bettayeb, Samuel Bassetto, Philippe Vialletelle & Michel Tollenaere (2011): Quality and exposure control in semiconductor manufacturing. Part II: Evaluation, International Journal of Production Research, DOI: / To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

2 International Journal of Production Research 2011, 1 18, ifirst Quality and exposure control in semiconductor manufacturing. Part II: Evaluation Belgacem Bettayeb a, Samuel Bassetto b *, Philippe Vialletelle c and Michel Tollenaere a a Grenoble-InP, G-SCOP Laboratory, Grenoble University, 46 Avenue Fe lix Viallet Grenoble, France; b Ecole Polytechnique de Montre al, Department of Mathematics and Industrial Engineering, C.P. 6079, succ. Centre-Ville, Montral, Que bec, Canada H3C3A7; c STMicroelectronics, 850 Rue Jean Monnet, Crolles, France (Received 16 August 2011; final version received 20 September 2011) The purpose of this article is to test the performance of a heuristic algorithm that computes a quality control plan. The objective of the tests reported in this paper is twofold: (1) to compare the proposed heuristic algorithm (HA) to an optimal allocation (OA) method; and (2) to analyse the behaviour and limitations of the proposed HA on a scale-1 test with a before/after test. The method employed to evaluate this algorithm is based on comparisons: (1) The first test illustrates the method and its sensitivity to internal parameters. It is based on a simplified case study of a product from the semiconductor industry. The product is made up of 1000, 800 and 1200 wafers incorporating three different technologies. The production duration is 1 week, and three tools were involved in this test. The behaviour of the proposed algorithm is checked throughout the evolution of the model parameters: risk exposure limit (R L ) and measurement capacity (P). The quality control plan for each tool and product are analysed and compared to those from a one stage allocation process (named C 0 ) that does not take into account risk exposure considerations. A comparison is also performed with OA. (2) The second scale-1 test is based on three scenarios of several months of regular semiconductor production. Data were obtained from 23 etching and 12 photolithographical tools. The outputs provided by the HA are used in the sampling scheduler implemented at this plant. The resulting samples are compared against three indicators. The results of these comparisons show that, for small instances, OA is more relevant than the HA method. The HA provides realistic limits that are suitable for daily operations. Even though the HA may provide far from optimal results, it demonstrates major MAR improvement. In terms of the maximum inhibit limit, the HA achieves better performances than C 0, and they are strongly correlated to R L and to the control capacity. The article concludes that the proposed algorithm can be used to plan controls and to guide their scheduling. It can also improve the insurance design for several levels of acceptance of risk. Keywords: control plan; risk exposure; resource capacity and resource capability 1. Introduction and short review This article constitutes the second part of the article entitled, Quality and exposure control in semiconductor manufacturing. Part I: Modelling. Most shop floor manufacturing systems have a complex physical production flux that turns the joint control of product quality and the production process into a highly intricate task. Moreover, the available measurement capacity, which is limited and often costly, must be judiciously allocated, in order to manage these two operational aspects. In semiconductor manufacturing, this problem is a daily concern in operations management. In order to be able to control processes, best-in-class process control techniques, ranging from equipment to production inspection (Schippers 2001, May and Spanos 2006), are implemented in semiconductor manufacturing (Bean 1997). In order to boost learning from first silicon, measurements are taken after each processing step. *Corresponding author. samuel-jean.bassetto@polymtl.ca ISSN print/issn X online ß 2011 Taylor & Francis

3 2 B. Bettayeb et al. However, sampling techniques must be introduced progressively and cautiously while ramping up production, to enable volume production (Bousetta and Cross 2005, Baud-Lavigne et al. 2010). Even though there is undoubtedly value in metrology operations (Bunday et al. 2007), reducing cycle time is the driver in regular production, as well as for achieving better yields and volumes (Leachman and Ding 2011). Moreover, as a 100% inspection does not result in 100% quality (Pesotchinsky 1987), inspection/control allocation must be optimised in order to minimise one or more of the impact functions related to the failures monitored. In regular production, process control operations are reduced to a minimum. However, if sampling is inadequately set up, a very large number of products could be exposed to unmonitored defects, which would affect the overall yield (Bettayeb et al. 2010). Classical sampling strategies allocate controls according to a given metrology capacity and to the process operations in place. Several industrial developments (Bousetta and Cross 2005, Purdy 2007, Mouli and Scott 2007) have be achieved, aimed at the adaptive control of measurement with respect to yield evolution, measurement capacity and risk. The first of these involves a sampling strategy based on the number of wafers checked on metrology tools, in order to minimise the over-sampling that could be caused by redundant controls. In the second paper, a mechanism is introduced to update the control of process deviations. The third paper presents a generic architecture for updating the strategy for controlling the risk encountered during production. This architecture represents a comprehensive development, which appears to improve quality control performances. Considering the impact on yield of SPC measurement, this is an interesting development of SPC sampling. Artificial intelligence techniques and self-organising maps have been tested, with a view to allocating controls dynamically (Lee 2002, Lee and Lee 2007) and in Wu and Pearn (2008) a control plan is proposed based on a process capability index. These works represent interesting developments that could be combined with factory dynamics (like the delay between processes and measurements) to help in designing intelligent control plans. Similar to the two approaches compared and evaluated in this paper is the work carried out in Villalobos et al. (1993) and Verduzco et al. (2001). In Villalobos et al. (1993) a flexible inspection system for serial and multi-stage production systems is presented and in both a dynamic programming algorithm is provided for optimising a global goal like costs taking into account some local constraints, like inspection tool availability. An interesting example of information-based inspection allocation is presented in Verduzco et al. (2001), where a cost function takes the type I and type II measurement errors into account. Their simulation shows that information-based solutions achieve better performances in terms of classification errors than static inspections. Even with the amount of research carried out on inspection and quality control allocation in the field, the results could still be strongly influenced by factory dynamics. Throughput, or the availability of measurement tools, the travel time between process and measurement (Shanoun et al. 2010) and the currently available capacity often lead to situations of over-control, under-control or lack of control. To minimise the impact of factory dynamics and variability, some key parameters such as delay, process criticality, measurement time and metrology capacity must be taken into account in designing a control plan. Depending on the number of products produced in concurrent runs, the number of process operations to be performed, or the qualifications of the various production tools, the complexity of designing an optimal control plan can increase dramatically. The research considered here is based on a two-stage quality control approach: (1) allocation planning and (2) on-line scheduling. This approach helps in the creation of a plan that is close to demonstrated manufacturing performances, as well as a dynamic adjustment strategy to support that plan. It forms part of the European Project IMPROVE: Implementing Manufacturing science solutions to increase equipment productivity and fab performance. The proposed HA method is evaluated in two tests: (1) The first is a simplified example from a semiconductor manufacturing facility, involving three products and three tools. This portion of the process plan is made up of 17 operations observed over 1 week of production (3000 wafers). The HA performances have been compared to OA performances. Quality control plans based on the HA for each tool and product are analysed and compared to those based on a single stage allocation process (named C 0 ) that does not take into account risk exposure. (2) The second test is a scale-1 test. It is made up of three instances of several months of regular production. Photolithography and etching workshops provided data for 35 tools, and six (technological) products were considered. The algorithm provides the maximum number of wafers that can be run between two inspections for each tool. These values are assessed through a sampling algorithm based on a global sampling indicator (GSI) (Dauzere-Perez et al. 2010) that prioritises lots and wafers in order to measure their added value.

4 International Journal of Production Research 3 Table 1. Case study parameters. Product A B C NOP MixProd Nr. Meas. Tools OBJSAT Meas. Tools Availability OOC% 2 75% 70% 5% Manufacturing tools CMP CVD FUR The paper is structured as follows: the first case study is introduced in Section 2, the scale-1 test from a semiconductor manufacturer is presented in Section 3, an analysis of numerical experiments issued by the sampler is provided in Section 4 and Section 5 concludes the paper with recommendations for further research. 2. Illustrative numerical example 2.1 Case study presentation The example in this section illustrates the two-stage approach described in the previous section, and is based on a typical production plan of a semiconductor manufacturer over one week. A summary of the case is provided in Table 1, and a detailed view is given in Figure 1. The case involves the production of three products, A, B and C, consisting of 1000, 800 and 1200 wafers respectively. The required quantity of each product is noted as X Q j : j2jðiþ For the sake of simplicity, the case contains only three tools: a chemical mechanical polisher (CMP), a chemical vapour deposition tool (CVD) and a furnace (FUR), while in a real plant, there can be over 300 tools distributed in more than 10 workshops. The process plans are also simplified in the example: products A, B and C undergo, 12, 8 and 17 operations respectively: this number is noted as NOP(product). Real-world semiconductor products involve more than 400 operations. Nevertheless, this example will be informative enough to illustrate the first test of the proposed algorithm. The process flow and capability parameters are presented in Figure 1. For every product (PRD column), each operation (OPER) is composed of two steps (STEP). The first step is executed on a toolset (TOOL) characterised by its capability (Cp). The second step is performed on a measurement toolset, also characterised by its capability (Cpm). In this illustration, the measurement tool capacities are also taken into account. The allocation of a fraction P of the total capacity (TOTALCAPA) is simulated, varying the value of TOTALCAPA from 0 to 100%. The notations are the same as those presented in Part I and are complemented by the following: SR min i : Sampling rate minimum for the product i, SR min i ¼ P n1 i j2jðiþ Q j SR max i : Sampling rate maximum for the product i, SR max i ¼ n i P j2jðiþ Q j 2.2 Allocation of control results For every operation, this allocation generates a minimum and a maximum sampling rate (see Figure 2). This makes it possible to compute the same rates by product or by tool (Figure 3). The results obtained in this illustration are the same as those obtained by the approach described in Section 3 for P ¼ 75% and R L ¼ 150.

5 4 B. Bettayeb et al. Figure 1. Process flow and capability parameters. Figure 2. Extract from the results table when P ¼ 75% and R L ¼ 150. Figure 3. Summarised results when P ¼ 75% and R L ¼ 150.

6 International Journal of Production Research 5 Table 2. Comparison of OA and HA performances. OA HA Relative performance (OA-HA)/OA, in percentage Objective function 99, , CMP CVD FUR A B C For example, operation 2300 on product A using the CVD should appear in the result seven times, in order to ensure that the CVD never processes more than 150 successive products in the same operation without a control. The minimum number of controls is computed in stage 1 of the algorithm (Figure 2). The maximum possible number of controls for this operation is 1000, which is the sum of the numbers of controls obtained in stage 1 and stage 2 (Figure 2). Figure 5 summarises the results and gives a general overview of the sampling rates by product and by process tool. In this example, it will be possible to control 96.93%, 100% and 84.74% of the operations of products A, B and C respectively, which corresponds to the control of 81.74%, 95.35% and 93.29% of the operations processed on the CMP, the CVD and the FUR respectively. This case study has also been resolved with the IBM ILOG CPLEX Optimisation Studio using the formulation presented in Section 3.2 of Part I. The optimal solution is obtained in 0.02s and the number of operations is small (17). The OA-generated results are presented in Table 2. The objective function of the proposed heuristic is close to that of OA, the difference being only 1% in favour of OA. The results provided in Table 2 reflect the differences in sampling rates among tools and products for the two methods. Globally, the capacity is sufficient to ensure a high sampling level. For the CMP, the sampling rate in OA is lower by 9.2% than that of the HA method: 81.26% instead of 89.2%. For the FUR, the sampling in OA is higher by 5.59% than that of the HA method: 95.55% instead of 90.3%. The sampling of products is also slightly different. The sampling of product A in OA is higher by 11.03% than that of the HA. The percentages for the product B sampling are unchanged, while the sampling of C in OA is lower by 6.87% than that of the HA. This comparison reveals that, in small instances (fewer than 20 operations), it may be wise to allocate using OA, as the solution time with this method is reasonable. The HA method introduces a step in the allocation process (Step 1) devoted to the allocation of controls to ensure that the threshold R L is not reached. This concept is central to the method proposed in this paper (and in Part I). We now need to study a case where this threshold has been removed. 2.3 Experiments on the HA In this example, an experiment is conducted to evaluate the influence on allocation of management parameters like R L and P. Two approaches to control plan computation are tested and compared. Both are based on allocation of a control capacity equivalent to P.. In the first case, R L is not used, and the capacity corresponding to P is allocated to the operations with respect to the sampling rate X j2jðiþ Q j and the criterion C i ¼ Cpm i Cp i : This case will be considered as a reference and noted C 0.

7 6 B. Bettayeb et al.. In the second case, R L is used to calculate the number of inspections. These controls consume measurement capacity (Stage 1). The remaining capacity is then allocated in Stage 2 following criterion C i ¼ Cpm i Cp i : This second case is noted C RL, and is compared to C 0 for several values of R L. For each scenario, the average sampling rate is calculated. Primary results The influence of R L on sampling rates is presented in Figure 4 for several values of P. When R L tends towards 1, the allocation is driven by C i ¼ Cpm i Cp i and the X Q j criteria: j2jðiþ This observation is valid for every curve presented in Figure 4. As R L is an insurance level, the closer it is to 1, the more influence it has on the control allocation process. If R L ¼ 1, every product manufactured should be controlled, for every tool and in every operation. Low values of P correspond to low control capacity, and this generates increasingly high sensitivity as R L approaches 1. For low values of R L the weight of insurance increases. For example, C R L ¼ 10 generates a need for controls to ensure that the uncertainty will not exceed 10. With a high measurement capacity, the number of controls increases. For the highest values of P, the classical allocation generates a 100% control plan (every operation is controlled, for every product). There is no change in the allocation either with R L or without R L. For P below 60%, there is not enough capacity to add risk management controls. As these controls have priority over the others, capacity must be released. As R L is expressed in the number of products potentially impacted, it has to be compared to the total number of products produced over the period of time considered. R L ¼ 10 means that the insurance level has been set to 0.3% of the total production (3000 products). As we can consider a variation of 10% or less in a phenomenon to be negligible relative to the phenomenon itself, it might be an idea to set R L to 10% of the production horizon plan, which would mean that R L ¼ 300. This does not mean that the loss of 300 wafers every week is acceptable, but that this risk may be encountered every week. Figure 4. Influence of R L on the average sampling rate, comparison between C RL and C 0.

8 International Journal of Production Research 7 With an R L value of 300, we can see (Figure 4) that the average number of controls is not influenced by the value of P. The variation remains within 1% [ 0.5%; þ0.5%]. This is an important simulation result, as it can help in the designing of the R L value and its sensitivity to variations of P. Secondary results This aggregated result has been split into tool and product levels. The tool viewpoint Every tool is impacted differently by R L and P, and that impact depends on the average values of Cpm and Cp. Figure shows the influence of R L and P on the CMP. When R L is greater than 100, there is almost no change. For every value of P, the variation of R L induces at most a 4% variation in the control sampling rate. This means that an insurance policy can be established without any noticeable disruption. When R L drops below 100 (0.3% of the number of operations processed on the CMP), there is a noticeable difference, depending on P. For P 4 80%, there is enough capacity to control every product processed on the CMP. There is then no variation between these two scenarios. For P 5 80% and 10 5 R L 5 100, the average sampling rate for the CMP can be decreased by up to 30%. This means that controls allocated for tool maintenance are requisitioned to reduce the uncertainty on other tools (and in this case the reallocation process favours the CVD, as shown in Figure 6). Where R L 5 10 allocation also controls the CMP. In this case, when R L decreases, the deallocation process is less pronounced for P ¼ 30% ( 30%) than for P ¼ 50% ( 42%). This can be explained by the fact that capacity is added, cut off and/or re-allocated to manage risks. At P ¼ 30%, the remaining capacity employed for control is absolutely mandatory for the CMP. Figure 6 shows the influence of R L and P on the CVD. When R L is greater than 100, there is almost no change. For every value of P, the variation of R L induces at most a 4% variation in sampling rates. This means that a control insurance policy can be established without any noticeable disruption. Sampling rates become more sensitive to R L and P variations when R L is below 100 (1% of production mix). For the CVD engineers, when R L decreases (R L from 100 to 10), there is a continuous increase in controls. A low volume of controls in the reference case explains this. The decrease in R L generates controls dedicated to revealing the level of uncertainty on the CVD. The increase in controls reaches a maximum associated with P saturation. For an R L value below 10, the CVD controls decrease, owing to domination by other tools (see Figures 5 and 7) in the allocation process. Figure 7 shows the influence of R L and P on the FUR. When R L is greater than 100, there is almost no change. For every sampling rate of P, the variation in R L induces at most a 4% variation in control (control removed). This means that an insurance policy can be established without any noticeable disruption. Figure 5. Impact of R L and P on the CMP.

9 8 B. Bettayeb et al. Figure 6. Impact of R L and P on the CVD. Figure 7. Impact of R L and P on the FUR. Sampling rates become more sensitive to R L and P variations when R L is below 100 (0.3% of the production mix). For 8 5 R L and for P below 75%, there is a continuous decrease in the number of controls, owing to the decrease in R L (increasing the pressure for insurance). This variation reaches 30% when P ¼ 30%. This can be explained by a good average value of the ratio C pm /C p and a high volume of operations, i.e. extended use of the tool. As R L decreases, the pressure for insurance controls increases and generates re-allocation of controls for that purpose. After reaching a minimum number of controls (for an R L of around 5), the number of controls increases again and becomes greater than it is for the reference case (R L 5 3). This means that, for a high level of pressure for insurance control, it is more important to control the FUR, owing to the X effect of this tool. Q j j2jðiþ

10 International Journal of Production Research 9 Figure 8. Impact of R L and P variations on the product A control plan. In general, it can be observed that, for R L :3% of X Q j!, there is almost no influence on the average sampling rates. If R L is below this limit, controls must be prioritised over several tools. This mechanism of dominance from one tool to another is not stable, however, and relies heavily on P and the mean values of the Cpm, Cp and P j2jðiþ Q j of the operations processed on each tool. When the re-allocation process takes place, some controls are suppressed for some tools and enhanced for others. A negotiation procedure has to be initiated among tool owners (engineers, technicians and suppliers), in order to sustain this re-allocation process. The product viewpoint For the owners of products, this control policy also has a differentiated impact, which is represented below. Figure 8 shows the impact of R L and P variations on the control plan of product A. For an R L higher than j2jðiþ 100 0:3% of X Q j!, for every value of P, there is almost no variation in the sampling policy as a result of introducing R L (less than 2%). This means that introducing a reasonable insurance policy would not disturb the owner of product A. When R L is below 100 and P ¼ 75%, there is a noticeable increase in the number of controls when R L is between 8 and 100. For P 4 80%, there is enough capacity to control all the operations for each product. For low values of P, there is not enough capacity to control every product and to fulfil the insurance requirements. In this case, step 2 involves choosing the appropriate operations for which some controls will be released in order to remain as close as possible to R L. Figure 9 presents the impact of R L and P variations on the control plans of product B. For an R L higher than 100 (0.3% of the production mix), for every P, there is almost no variation in the sampling policy as a result of the introduction of R L (less than 2%). This means that introducing a reasonable insurance policy would not disturb the owner of product B. For an R L below 100 for product B, there is modification regarding R L variations and P. For P around 75%, an increase in control is noticeable until R L is between 8 and 100. For P above 80%, there is enough allocation capacity j2jðiþ

11 10 B. Bettayeb et al. Figure 9. Impact of R L and P variations on the product B control plan. to monitor every product in every operation. There is, therefore, no increase in the number of controls in either of the two scenarios. For low values of P, there is not enough capacity either to control the product (owing to the Cpm/Cp ratio) or to fulfil the insurance requirements. A re-allocation process is therefore applied to release controls, in order to remain close to R L. This allocation process favours product C, as shown in Figure 10. Figure 10 shows the impact of R L and P variations on the controls of product C. For an R L higher than 100 (0.3% of the production mix), for every P, there is almost no variation in the sampling policy as a result of the introduction of R L (less than þ1% of controls). This means that introducing a reasonable insurance policy would not disturb the owner of product C. For an R L below 100 for product C, depending on P capacity, controls are re-allocated to this product because of its high level of production (1200, compared with 800 and 1000 in the previous scenarios). In this case, the production mix drives the allocation process. Conclusion This example has investigated the complexity of control plan design for producing 3000 wafers incorporating three technologies and manufactured using three tools, based on a process plan of 12, 8 and 17 operations respectively. This simplified example reveals how difficult it is to analyse and manage the behaviour of the allocation process if the exposure limit is too restricted relative to the production plan and the available measurement capacity. In the example, the exposure limit can be introduced without any noticeable disruption to the sampling rates until it represents more than 0.3% of the production plan. In real-world production, this insurance level could be set to 10%. This would mean that a control plan can be implemented which ensures that a threshold level of uncertainty could be respected by taking into account control capacities, tool performances and production volume. We therefore recommend evaluation of the impact of this algorithm at every site prior to its deployment, in order to define an appropriate and manageable value of R L. 3. Scale 1 test 3.1 Case Study The case study takes place in a semiconductor manufacturing facility and focuses on the etch and photolithography operations. Each operation has its own subset of qualified process tools and its own subset of qualified measurement tools. The process flow model is presented graphically in Figure 11. The case is made up of three instances of 1

12 International Journal of Production Research 11 Figure 10. Impact of R L and P variations on the product C control plan. Figure 11. Simplified flow model of the CD measurement case study. month of production. Six technologies were involved, representing several thousand operations per instance at the wafer level:. Instance 1: 26/02/2011 to 26/03/2011, 2598 processed lots, for a total of 25,558 operations.lots, 605,854 operations on wafers and controls of lots. Instance 2: 25/09/2010 to 23/10/2010, 3025 processed lots, for a total of 29,725 operations.lots and 701,601 operations on wafers, and 17,879 controls of lots. Instance 3: 24/04/2010 to 22/05/2010, 2760 processed lots, for a total of 25,619 operations.lots 592,466 operations on wafers, and 14,505 controls of lots Each lot contains feet wafers most of the time. The cost of one wafer varies from $3000 to $15,000. Case study hypothesis: the number of qualified tools in each workshop and the processing times of the process and measurement steps are supposed constant over the allocation period. Delays between processes and measurements have also been taken into account, and are supposed constant for each workshop. The mean delay between etching and measurement is 177 wafers, while the delay between lithography and measurement is 116 wafers. This delay is then subtracted from the R L for each workshop.

13 12 B. Bettayeb et al. Figure 12. Evolution of MAR for EL23M04 in Instance 1. The characteristics of the case study are given below:. NPTE: Number of process tools in the etch workshop (23).. NPTL: Number of process tools in the photolithography workshop (13).. NMT: Number of tools in the measurement workshop (12).. NP: Number of products (technologies or product families) (6).. NOP: Number of operations to be controlled by measurement (313).. NOPp: Number of operations on product p to be controlled by measurement. In order to connect the HA with the on-line scheduler (Dauzere-Peres et al. 2010), the output provided has been computed as M ^ARz using the MAR formulation and n 1 i. For n i ¼ n 1 i þ n 2 i values, the resulting estimated MAR is denoted M ^AR max z. The test is a before/after test. The sampling plan policies have been compared with 3 indicators: (1) mean MAR max z, which computes, for each tool, its maximum MAR and the mean of these values. This indicator can be interpreted as the exposure maximum of the production system. Note that it is not an estimated value, but the real value achieved by the tool. (2) mean z mean Hz ðmar z Þ, which is computed for each tool, its mean (MAR) over its production horizon, and then the mean of these values. It can be understood as the mean exposure of the manufacturing system. (3) Number of lots where the tool was above the limit min M ^ARz, each of which warns the production manager that the manufacturing system is facing a major risk of disruption. Their number is an indicator of the level of exposure to a major event. Figure 12 presents the raw MAR data for the tool EL23M04 collected during Instance 1. The dashed line represents the M ^AREL23M04 computed for the test (results presented in Table 3). In this figure, the number of lots above this limit is equal to 50. This means that 50 times during this period, the backlog of uncertain products was above the manageable limit. During this observation period, the manufacturing system was exposed to a major disruptive event 50 times. This figure also shows the non-regular position of controls along the production line and the influence of scheduling on MAR EL23M04. Table 3 presents the results of min M ^AR z for each tool, as well as the achievable R L for each instance. R L is the lowest value of R L that can be respected by each tool, whatever its production-mix. This value is the insurance level presented so far in this work. Its computation is detailed in the next section. Results are presented in Tables 4, 5 and 6. They are compared based on the differences expressed as percentages.

14 Table 3. minðm ^AR z International Journal of Production Research 13 Þ for the three instances. Instance 1 Instance 2 Instance 3 R L Delays D E ¼ I77 D E ¼ 177 D E ¼ l77 D L ¼ 116 D L ¼ 116 D E ¼ 116 Tool minðm ^ARz Þ minðm ^AR z Þ minðm ^ARz EL23M EL23M EL23M EL23M EL23M EL23M EL23S EL23S EL23S EL23S EL23S EL23X EL23X EL23X EL23X EL23XO EL23X EL23X EL23X EL23X EL23X KL23X EL23X L193C L193C LI93C L193C L248C L248C L248C L248C L248C L248C L248C L248C Table 4. Before/after comparison Instance 1. Þ Instance 1: Before (in wafers) After (in wafers) Variation (in %) #wafers 4 minðm ^AR z Þ meanðmar max z Þ mean z mean Hz ðmar z Þ Equivalent/Achievable R L ¼ þ ( ) In the first instance, the number of wafers above min M ^ARz in the regular production is 692. With the HA, this number decreased by 95.95% to 28. This result shows that the hypothesis of uniform planning is not true, as 28 wafers did not respect the R L constraint. Nevertheless, it is an encouraging result, in that even if not every wafer is under the threshold limit, the quantity is almost respected.

15 14 B. Bettayeb et al. Table 5. Before/after comparison Instance 2. Instance 2 Before After Variation (in %) #wafers 4 minðm ^AR z Þ meanðmar max z Þ mean z mean Hz ðmar z Þ Equivalent/Achievable R L ¼ 450 þ ( ) Table 6. Before/after comparison Instance 3. Instance 3 Before After Variation (in %) #wafers 4 minðm ^AR z Þ meanðmar max z Þ mean z mean Hz ðmar z Þ Equivalent/Achievable *L þ ( ) ¼ The mean MAR max z decreases by 41.57%, from to This means that the mean exposure has been reduced by 41.57% with the proposed heuristic, and mean z mean Hz ðmar z Þ has decreased by 66.38%, from to 46. The fourth line of the table presents the evolution of the optimal R L. R L After is equal to 400. The distance between mean MAR max z and R L (176.03) represents a mean delay between the process and the measurement (this value is influenced by the production mix and the time spent in each workshop). In order to compute R L Before, it has been assumed that this distance remains constant. By the way, R L Before is equal to wafers. From the insurance viewpoint, this first instance helps key managers adapt the insurance policy. First, this study reveals that, before we developed our heuristic algorithm, the manufacturing system was exposed to a mean potential loss of wafers. The tools generated a large part of this loss ( wafers), while production management generated uncertain wafers. This represents a global potential loss of between $1.68 million and $8.39 million. Of course, this is the worst-case scenario, where all the wafers would have been different. With the proposed heuristic, this number can be identified and reduced to 400 wafers, of these being uncertain wafers for tools, with wafers in the measurement buffer. This generates a potential loss of between $1.2 million and $6 million, and adaptation of the insurance coverage by between $0.48 million and $2.39 million. The second instance of this test also yields major improvements. The number of lots above the local threshold limit after our development was applied was reduced by 98.42%, from 2598 wafers to 41. The mean MAR max z decreased from 450 to 25.69, yielding an improvement of 94.29%. The mean z mean Hz ðmar z Þ decreased by 66.58%, from wafers to R L Before reached , while R L After was set at 430. Based on this instance, the insurance can be adapted from [$2.56 million; $12.81 million] to [$1.29 million; $6.45 million]. Changes to the insurance coverage range from $1.27 million to $6.16 million. The improvements to this third instance are also fairly dramatic. The number of wafers above the local threshold limit after our development was applied was reduced by 85.61%, from 7329 wafers to In this case, the number of wafers above the limitis quite high, even with the quality plan in place. The mean MAR max z decreased from to , yielding an improvement of 42.45%. The mean z mean Hz ðmar z Þ decreased by 67.87%, from wafers to R L Before reached , while R L After was set at 300. Based on this instance, the insurance can be adapted from [$1.30 million; $6.53 million] to [$0.9 million; $4.5 million]. Changes to the insurance coverage range between $0.4 million and $2.03 million. This test yields very interesting results and makes it possible to calibrate the gains and the potential of the introduction of this heuristic. The first result points to the lack of MAR follow-up throughout the manufacturing system. This is important because the plant is regularly exposed to huge quantities of uncertain products and unrecoverable losses in the millions of dollars. A synthesis of these results is presented in Table 7.

16 International Journal of Production Research 15 Table 7. Gains. Mean of instances Mean variation #Lots 4 minðm ^ARz Þ 93.32% mean z mean Hz ðmar z Þ 59.43% meanðmar max z Þ 66.94% Mean min insurance gain 0.883MS Mean max insurance gain 3.52MS Figure 13. Results of the With/Without Stage 1 Test, without delay. The proposed algorithm will help key managers to manage this uncertainty and to re-design the insurance policy to take in account this catastrophic event. Nevertheless, it is clear that this level of uncertainty is greatly influenced by production scheduling. 4. Scale 1 tests Conducting this comparison on scale-1 manufacturing data has given us the opportunity to evaluate the internal performances of the proposed HA. The method has been compared to the case where stage-1 was omitted. Comparison of the two sets of results is based on the min M ^ARz obtained. The case where Stage 1 was omitted is named C 0 and is represented in Figure 13 and Figure 14 as R L ¼ 0. Two comparative tests have been performed, one with a mean delay and one without a delay between process and measurement. Results are presented in Figure 13 and Figure 14. The with/without comparison provides insight into the behaviour of the algorithm. The major result is that the risk-based allocation does not outperform a simple partition in every case. For a given control capacity, there is only one interval of R L where the two-stage allocation generates better results (in terms of min M ^AR z ) than the one-stage allocation. Let us note the following: R min &R max =8R L L L R min or 8R L L R max ; min M ^AR RL L z min M ^ARz C 0 We can see that there is a minimum in this interval, named R L 2 R min ; R max L L. Between R min and R, there is not enough control capacity remaining below R L L L for planning. However, Max z2tools min M ^ARz is better in the case of two-stage planning than a partition allocation C 0. We can also see that R L, First R L= Max z2tools min M ^AR z RL. When the delay is null, then this is achieved when the curve Max z2tools min M ^ARz crosses the Identity curve. When the delay is not null, this result is shifted from the delay, as shown in Figure 14.

17 16 B. Bettayeb et al. Figure 14. Results of the With/Without Stage 1 Test, with delay. When R L R L, the need for controls required by step 1 of the HA decreases. The position of Max z2tools min M ^AR z below the Id curves (y ¼ RL) is then not surprising. As there has been enough control capacity since R, there remains sufficient capacity to ensure that R L L is not exceeded. A more surprising observation is that Max z2tools min M ^AR z do not tend to C 0 values when R L increases. For high values of R L, the capacity used to remain below R L can be neglected, relative to the remaining capacity. However, the formulation of M ^AR z is as follows: P P Then, as M ^ARz ¼ X i2f1,...,nopg ToolsðiÞz Q j2jðiþ j i2f1,...,nopg cardðtoolsðiþþ ToolsðiÞz P i2f1,...,nopgn 1 i ToolsðiÞz decreases the need for control decreases and the curves become asymptotic: max ðm ^ARz Þ! R L maxðd z Þ z2tools RL!1 z Based on the sampling rate, the behaviour is different, as X i2f1,...,nopg ToolsðiÞz n 1 i, 8z n 1 i! RL!1 0: From an insurance perspective, the potential losses are controlled when the insurance level is in the interval R min L ; Rmax L. Outside this interval, it is preferable to choose the simple partitioning allocation, C 0. The potential gain of the proposed method is to save R C0 L R L products on a given production horizon. For the example given, with P ¼ 70%, for a wafer cost between $3000 and $15,000, the insurance policy can be decreased by at most R C0 L R L WaferCost ¼ ( ) * ,000 up to $3 million. This value is acceptable if R L varies between 350 and 600, a range that the authors observe is classically applied in semiconductor manufacturing. 5. Conclusion and perspectives This is Part II of the paper entitled, Quality and exposure control in semiconductor manufacturing, and is subtitled Evaluation. Part I of this paper, subtitled Modelling, provides an in-depth evaluation of the performance of the proposed algorithm.

18 International Journal of Production Research 17 That evaluation is mainly based on comparisons. Two are performed: one between the results produced by the proposed heuristic algorithm (HA) and a reference case (C 0 ); and the second between the HA results and those of an optimal allocation (OA) method. The comparison with C 0 is conducted on a fictitious, but informative case study. The other comparisons (between HA and C 0, and between HA and OA) are conducted using real data from a semiconductor manufacturing facility. The comparison process reveals significant differences between the proposed method (HA) and a comparative method from the literature (OA), and also between the HA and a simple allocation process C 0. The authors recommend using the OA formulation for small instances of the problem. For large instances, the HA method provides interesting results as well as cost savings. For the 3-instance problem, the mean gain has been estimated to be between $0.883 million and $3.52 million, with a wafer cost ranging from $3000 to $15,000. The HA method sheds light on controls from the insurance perspective, which is hardly achieved in real operations. The main focus of this second part of the paper is the deployment of the proposed heuristic in a fully operational setting, and comparison of its relative performance on various manufacturing sites. An interesting perspective might be to transfer the concept of material at risk, which is widely employed in semiconductor research, to large-scale products, as in the construction or aeronautics industries, for example. In these environments, the major loss could be expressed in terms of worked hours at risk. Acknowledgements The authors are grateful to STMicroelectronics for providing data and support for their research. This article describes the research in the context of the IMPROVE project, Implementing Manufacturing science solutions to increase equipment productivity and fab performance. IMPROVE is a JTI Project supported by the ENIAC Joint Undertaking, contract no References Baud-Lavigne, B., Bassetto, S., and Penz, B., A broader view of the economic design of the X-bar chart in the semiconductor industry. International Journal of Production Research, 48 (19), Bean, W.J., Variation reduction in a wafer fabrication line through inspection optimisation. Thesis. MIT. Bettayeb, B., et al., Optimised design of control plans based on risk exposure and resources capabilities. ISSM 2010, October, Tokyo, Japan. New York: IEEE. Bousetta, A. and Cross, A.J., Adaptive sampling methodology for in-line defect inspection. Advanced semiconductor manufacturing conference and workshop 2005, April, Munich, Germany. New York: IEEE/SEMI, Bunday, B., et al., Value-added metrology. IEEE Transactions on Semiconductor Manufacturing, 20 (3), Dauzère-Pe res, S., et al., A smart sampling algorithm to minimise risk dynamically. Advanced semiconductor manufacturing conference (ASMC 2010), July, San Francisco, CA. New York: IEEE/SEMI, Leachman, R.C. and Ding, S., Excursion yield loss and cycle time reduction in semiconductor manufacturing. IEEE Transactions on Automation Science and Engineering, 8 (1), Lee, J.H., Artificial intelligence-based sampling planning system for dynamic manufacturing process. Expert Systems with Applications, 22 (2), Lee, J.H. and Lee, K.S., Iterative learning control applied to batch processes: An overview. Control Engineering Practice, 15 (10), May, G.S. and Spanos, C.J., Fundamentals of semiconductor manufacturing and process control. New Jersey: Wiley Interscience. Mouli, C. and Scott, M.J., Adaptive metrology sampling techniques enabling higher precision in variability detection and control. Advanced semiconductor manufacturing conference 2007 (ASMC 2007), June, Stresa, Italy. New York: IEEE/SEMI, Pesotchinsky, L., Problems associated with quality control sampling in modern IC manufacturing. IEEE Transactions on Components, Hybrids, and Manufacturing Technology, 10 (1), Purdy, M., Dynamic, weight-based sampling algorithm. International symposium on semiconductor manufacturing 2007(ISSM 2007), October, Santa Clara, CA. New York: IEEE, 1 4. Schippers, W.A.J., An integrated approach to process control. International Journal of Production Economics, 69 (1), Shanoun, M., et al., Optimisation of the process control in a semiconductor company: model and case study of defectivity sampling. International Journal of Production Research, 49 (13),

19 18 B. Bettayeb et al. Verduzco, A., Villalobos, J.R., and Vega, B., Information-based inspection allocation for real-time inspection systems. Journal of Manufacturing Systems, 20 (1), Villalobos, J.R., Foster, J.W., and Disney, R.L., Flexible inspection system for serial multistage production systems. IIE Transactions, 25 (3), 16. Wu, C.-W. and Pearn, W.L., A variable sampling plan based on Cpmk for product acceptance determination. European Journal of Operational Research, 184 (2),