Pattern Recognition for Manufacturing Process Variation using Ensembled Artificial Neural Network

In quality control, monitoring and diagnosis of multivariate out of control condition is essential in today’s manufacturing industries. The simplest case involves two correlated variables; for instance, monitoring value of temperature and pressure in our environment. Monitoring refers to the identification of process condition either it is running in control or out of control. Diagnosis refers to the identification of source variables (X1 and X2) for out of control. In this study, an ensemble artificial neural network scheme was investigated in quality control of process in Hard Disk Drive manufacturing. This process was selected since it less reported in the literature. In the related point of view, this study should be useful in monitoring and diagnose the bivariate process pattern in Hard Disk Drive manufacturing process. The result of this study, suggested this scheme has a superior performance compared to the traditional artificial intelligence, namely single isolated Artificial Neural Network. In monitoring, ANN expected to be effective in rapid detection of out of control without false alarm. In diagnosis, this scheme was effective to be applied in identifying various types of process variation such as loading error, offsetting tool, and inconsistent pressure in clamping fixture. Whereby, diagnosis cannot be performed by traditional control chart. This study is useful for quality control practitioner, particularly in manufacturing industry


Introduction
In manufacturing, quality is defined as conformance to specification. Poor quality due to process variation is known as a major issue in manufacturing processes. The main concern of process quality control is to achieve and maintain an acceptable level of the desired process quality characteristic consistently. In this connection, accurate monitoring and control of the manufacturing system is very important and consequently, data must be gathered and analyze. In using control charts, samples of the products are drawn during the manufacturing process and sample statistics are then plotted on control charts (Chena & Wang, 2004). Control charts, mostly in the form of X chart, are widely used as aids in maintaining quality and achieving the objective of detecting trends in quality variation before defective parts/products are actually produced. The X chart usually exhibits various types of patterns e.g., normal (NOR), stratification (STA), systematic (SYS), increasing trend (UT), decreasing trend (DT), upward shift (US), downward shift (DS), cyclic (CYC), and mixture (MIX), as shown in Figure 1.1 (Bag, Gauri & Chakraborty, 2011).

Figure 1.1: Nine control chart patterns
Assignable causes are defined as abnormal shocks to the processes, which should be identified and eliminated as soon as possible. When an abnormal variation is signaled by the control chart, quality practitioners or engineers search for the assignable causes and take some necessary corrections and adjustments to bring the out-of-control process back to the in-control state (Yu, Xi & Zhou, 2009). The usual practice has been to maintain a separate (univariate) chart for each characteristic. Unfortunately, this can give some misleading result when the quality characteristic is highly correlated. One of the solutions to overcome this issue is to extend the univariate analysis by plotting a statistic that measures the overall deviations of the multivariate observations from the target (Chena & Wang, 2004). Variation may be defined as any unwanted condition or as the difference between a current and a desired end-state. To control and reduce the variation, the source of the variation must be identified. Statistical process control (SPC) is one of the most effective tools of total quality management, which is used to monitor process variations and improve the quality of production. (Bag, Gauri & Chakraborty, 2012). Advances, variation reduction efforts as such process monitoring and diagnosis should be critically applied towards quality improvements (Masood & Hassan, 2009). For decades, statistical process control (SPC) has played a major role in controlling a product's quality, since Shewhart illustrated the technique of the control chart by applying statistical concepts in the manufacturing process (Chih & Rollier, 1993). The Shewhart control chart certainly is not new, but its use in modern-day business and industry is of tremendous value. The center line represents the place where the characteristic measured should ideally be located and the parallel lines represent the control limits of the characteristic (Esteban et al., 2009). Many manufactures use Univariate Statistical Process Control (USPC) in both their manufacturing and service operations. Automated data collection, low-cost computation, products and processes designed to facilitate measurement, demands for higher quality, lower cost, and increased reliability have accelerated the use of USPC (Rao et al., 2013). However, in many situations the widespread use of USPC has caused a backlash as processes are frequently adjusted or shutdown when nothing is really wrong because estimates of the probability of false positives (Type I error) calculated based on USPC, were unrealistically low. USPC also takes little or no account of the multiple tests that are being performed or the correlation structure that may exist in the data (El-Din, Rashed & El-Khabeery, 2006).

Multivariate Statistical Process Control (MSPC)
Multivariate Statistical Process Control (MSPC) is a methodology, based on quality control charts, that is used to monitor the stability of a multivariate process. Stability is achieved when the means, variances and covariance of the process variables remain stable over rational subgroups of the observation. The conventional MSPC charts mainly include multivariate Shewhart control charts, multivariate CUSUM control charts, multivariate EWMA control charts and chart based on multivariate statistical projection methods. In manufacturing, the MSPC charts serve as well as fundamental tools for multivariate process control application. Consequently, multivariate statistical methods which provide simultaneous scrutiny of several variables are needed for monitoring and diagnosis purposes in modern manufacturing systems. (Chen, Boning and Welsch, 2001). Pattern recognition is one from the need for automated machine recognition of objects, signals or images, or the need for automated decision-making based on a given set of parameters. Pattern recognition is a branch of machine learning that focuses on the recognition of patterns and regularities in data, although is in some cases considered to be nearly synonymous with machine learning A control chart pattern recognizer is to recognize any abnormal function by monitoring the behavior of the system under production. Accurate and fast control chart recognition is essential for efficient system monitoring to maintain the high-quality products. Several approaches have been proposed for CCP recognition, including rule-based, expert system and artificial neural networks (Sagiroglu et al., 2000).

Artificial Neural Network (ANN)
Artificial Neural Network (ANN) or Neural Network (NN) has provide an exciting alternative method for solving a variety of problems in different fields of science and engineering. Artificial Neural Networks are relatively basic electronic models based on the neural structure of the brain. The brain basically learns from experience. The ANN is very useful because it has ability to learn and recognize. ANN can figure out how to perform their function on their own and determine their function based only upon sample inputs. Neural network also has the ability to identify an arbitrary (Kiran et al., 2010). Recently, many studies used ANNs in order to detect patterns more effectively than the conventional approach and their aim is the automatic diagnosis of the patterns. ANN model is a feed forward network which contains an input layer, some hidden layers and an output layer. Each neuron in the network accepts a weighted set of inputs and responds with an output (Shivakumar et al., 2010). Figure 1.21 shows the architecture generalized recognizers; raw data-based and generalized featurebased respectively.

Figure 1.21: Generalized-ANN architectures based on three layer MLP model
Raw data-based input representation yields large dimensional input vectors, computational efforts and time consuming for training ANN recognizer. In addressing this issue, feature based input representation such as summary statistic features, frequency count features, shape features and statistical features have been proposed in developing univariate CCPR schemes. It involves features extraction procedure as shown in Figure 1.22 to extract the properties of the samples (Masood & Hassan, 2010).

Ensemble Artificial Neural Network (ANN)
The existing ANN schemes commonly main generalized based which is only single ANN recognizer was applied. The generalization ability of ANNs can be improved by combining several ANNs in redundant ensembles, where the member networks are redundant in that each of them provides a solution to the same task, or task component, even though this solution might be obtained by different methods (Sharkey, 1999). This approach is now formally known as an artificial neural network ensemble. An ANN ensemble is a finite number of ANNs that are trained for the identical purpose whose predictions are combined to generate a unique output. ANN ensembles offer a number of advantages over a single ANN in that they have the potential for improved generalization and increased stability (Sharkey, 1999). Ensemble methods have been successfully applied in various domains, such as time series prediction, robotics, and medical diagnosis (Shu & Burn, 2004). It has been shown to frequently generalize better than single ANN. However, for ensemble to be effective, the component ANN in the ensemble must be as accurate and diverse as possible. Figure 1.3 illustrates the basic framework of an ANN ensemble.

Figure 1.3: A neural network ensemble
Several approaches have been proposed for CCP recognition, including rule-based, expert system and artificial neural networks (Sagiroglu et al., 2000). ANN was designed to provide the quality control practitioners with the status of the process (in-control or out-of-control), plausible causes for the out-of-control situation, and cost-effective actions against the outof-control situation (Bag, Gauri & Chakraborty, 2011). The advantage with an ANN-based pattern recognizer is that it does not require the provision of explicit rules or templates.

Experimental
The hard disk drive (HDD) is a data storage device and the most important component of a computer. HDD used for storing and retrieving digital information using one or more rigid rapidly rotating disk and coated with magnetic material. Manufacturing of HDD involve many basic mechanical structure such as casing and platters. In this project, an artificial neural network, an artificial intelligence technique, is developed to successfully monitoring and diagnosing variation process variation in manufacturing of HDD. There is a motor in the hard disc drive in order to control the rotation of disc. The mounting of the motor (ID1 and ID2) are defined as critical characteristic that require bivariate quality control (BQC) by using the artificial neural network. However, the existing ANN schemes were mainly utilized single isolated ANN model with raw data input representation. Hence, in this project, a combined ANN model which is called an Ensemble-ANN will be investigated for improving the quality control capability. The proposed model involves utilization of raw data and statistical features input representation technique. Representative pattern recognition scheme (framework and procedures), namely an Ensemble-ANN as a strategy of improvement. The intended scheme should be capable to identify the source variables of bivariate process variation rapidly and correctly with minimum false alarm.

Multivariate (Bivariate) Process Pattern
Scatter diagram can be used to indicate the distribution of samples in bivariate process.
Depending on process situation, the bivariate samples can be in low correlation (ρ = 0.1 ~ 0.3), moderate correlation (ρ = 0.4 ~ 0.6) or high correlation (ρ = 0.7 ~ 0.9). Data correlation (ρ) shows a measure of degree of linear relationship between such two variables. The degree of mean shift can be identified when the center position is shifted away from zero point, whereas the degree of cross correlation can be identified from the shapes of ellipse as shown in Figure 2.1. Low correlated samples yield a circular pattern (circular distributed scatter plot), moderate correlated samples yield a perfect ellipse pattern, whereas high correlated samples yield a slim (small) ellipse pattern. Thus, recognition of these patterns could lead to a better process monitoring and diagnosis.

Input Representation
Input representation is a technique to represent input signal and provides a strong influence on the performance of ANN towards achieving effective recognition. In this study, raw data and statistical features input representations were applied into training of Ensemble-ANN recognizer for improving pattern discrimination capability. These approach objectives to heighten the recognition accuracy on an ANN through smaller network size, lesser computational efforts and shorter training time. Both input representation approaches were applied in this study. The raw data input representations were applied into training of Synergistic-ANN recognizer for improving pattern discrimination capability. Raw data-based input representation contained 24 consecutive standardized samples for variables 1 and variable 2. Therefore, each pattern was represented by 48 input data, i.e., (Z1-P1, Z1-P2, ... Z2-P1, Z24-P2), where variable 1 and variable 2 were denoted as P1 and P2 respectively.  Lucas and Saccucci (1990). Besides resulting longer ARL0, these parameters could influence the performance of EWMA control chart in detecting process mean shifts. Preliminary experiments suggested that the EWMA with small constant parameter ( = 0.05) were more sensitive in identifying small shifts (≤0.75 standard deviations), while the EWMA with large constant parameter ( = 0.40) were more sensitive in identifying large shifts (≥2.00 standard deviations). The MSD and MMSV features were used to magnify the magnitude of mean shifts ( 1, 1):

Feature-Based Input Representation
where ( 1, 2), ( 1, 2)( l 2 , l 2 ) are the means, standard deviations and mean square value, respectively. The mathematical expressions of mean and standard deviation are widely available in textbook on SPC. The mean square value feature can be derived as in Hassan (2003).

Pattern Recognizer
Recognizer is the element that recognized pattern variation from the identifier. In this study the multilayer-perceptron (MLP) architecture were selected since it has been proven effective for MQC (Guh, 2007). MLP model compromises an input layer, one or more hidden layer(s) and an output layer. The number of layers and neurons in each layer could influence an ANN performance. MLP is a supervised training scheme, which requires pre-prepared data with each of the data set need to be classify.

Synergistic-ANN Recognizer
Synergistic-Ann recognizer is a parallel combination between raw data based ANN and featured based ANN recognizers as shown in Figure   Combining the outputs of member ANN is useful only if there is diversity on some inputs. Obviously, combining several identical performances ANN produces no gain. For synergistic-ANN, if the average error rate for an example is less than 50% and the NNs in the ensemble are independent-error, the expected error for that example can be reduced to zero as the number of ANN combined goes to infinity. However, such assumptions rarely hold in practice, because in real applications, the available components ANN are very similar in performance.

Framework of Ensemble ANN Scheme
Framework for the proposed scheme is summarized in Figure 2.6. An ensemble ANN schemes combined the traditional SPC chart(s) with an ANN model. The traditional SPC chart(s) role for monitoring the existence of unnatural variation in bivariate process, whereas an ANN model role for diagnosing the sources of variation. In that case, an ANN model is utilized only when necessary, that is, when an out-of-control signal is triggered. Inversely, the other schemes such as novelty detector ANN consist of fully ANN or fully Machine Learning (ML)-based model for monitoring and diagnosing simultaneously. In this case, an ANN model is continuously utilized for triggering out-of-control signal and then, for identifying the sources of variation. The workflow for the basic model of Integrated SPC-ANN consists of five steps that need to be done. These include: Step 1: Input samples (bivariate). Window size = 24, starting observation samples are: X1i = (X1-1… X1-24) and X2i = (X2-1… X2-24). It is followed by (i th + 1), (i th + 2)… and so on.
Step 3: Second stage monitoring and diagnosis (Synergistic-ANN recognizer): Decision rule: If Maximum output of ANN belongs to N (0, 0): Process is "in-control": Proceed to the next samples else Process is "out-of-control": identify the sources of mean shifts; perform diagnosis, troubleshooting; renew setting and return to the Step 1 (out of scopes this research) end

Recognizer Training and Testing
Partially developed shift patterns and dynamic patterns were applied into the ANN training and testing respectively since these approaches have been proven effective to suit for on-line process situation are summarized in Table 2.71. In order to achieve the best training result for overall pattern categories, the amount of the training patterns was set as follows: (i) bivariate normal patterns = [1500 x (total combination of data correlation)] (ii) bivariate shift patterns = [100 x (total combination of mean shift) x (total combinations of data correlation)]  (1500) and error goal (0.001), whereas the network performance was based on mean square error (MSE). Sigmoid function was used for an output layer meanwhile hyperbolic tangent was used for hidden layer. The training session was stop either when the number of training epochs was met or the required MSE has been reached. Table 2.72 summarizes the target output for an ANN. The maximum values (0, 9) in each row identify the corresponding neuron expected to secure the highest output for patterns correctly classified. Table 2.72: Target output for an ANN

Result and Discussions
The performances of Ensemble-ANN scheme were evaluating based on the recognition accuracy out of control process and average run length (ARL1). The ARL1 results as summarized in Table 3.10 were estimated by the results on three shift pattern categories. The results were simulated based on the correctly classified patterns and its support the conclusion that the mean shift with larger magnitude can be identified more quickly with shorter ARL1 values. This trend supports the conclusion that process mean shift with smaller magnitude would more difficult to detect. Specifically, the Ensemble-ANN scheme indicated rapid detection capability for moderate shift and large shift respectively. In comparison with ARL1 of Raw data based-ANN studied by Masood, I (2012), detection capability as shown by Ensemble-ANN was faster for (shift = 1.0, 1.5, 2.0) as shown in Table 3.11.

Table 3.11: Rapid detection capability in mean shifts by Ensemble-ANN scheme
The pattern recognition scheme using Synergistic-ANN recognizer exhibits a strong capability for online monitoring bivariate process mean shifts. Overall, the Synergistic-ANN can be concluded is more effective than the Raw data based-ANN in detecting mean shifts, particularly for dealing with small and moderate values. Meanwhile, based on range of ARL0 results ( = 0.3, 0.7) there is no false alarm, it can be concluded that the proposed scheme indicated balanced monitoring performance.

Conclusions
The case study involves loading error, off-setting tool and pneumatic problem in machining operations, whereby the proposed scheme has shown an effective monitoring capability in identifying the bivariate process control without any false alarm. In diagnosis aspect, the scheme also effective in correctly identifying the source of mean shift when process becomes out of control. Based on the promising results, the Ensemble-ANN scheme could be a reference in realizing balanced monitoring and accurate diagnosis of bivariate process pattern