Modeling Permeability Prediction Using Extreme Learning Machines

Transcription

1 Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation Modeling Permeability Prediction Using Extreme Learning Machines Sunday Olusanya Olatunji and Ali Selamat Intelligent Software Engineering Laboratory, Faculty of Computer Science & Information Systems, Universiti eknologi Malaysia, 83 UM Skudai, Johor, Malaysia Abdul Azeez Abdul Raheem Centre for Petroleum and Minerals, he Research Institute, King Fahd University of Petroleum and Mineral, Dhahran 36, Kingdom of Saudi Arabia Abstract In this work, an extreme learning machine (ELM) has been used in predicting permeability from well logs data have been investigated and a prediction model has been developed. he prediction model has been constructed using industrial reservoir datasets that are collected from a Middle Eastern petroleum reservoir. Prediction accuracy of the model has been evaluated and compared with commonly used artificial neural network and support vector machines (SVM). We have applied an extreme learning machine (ELM) for single-hidden layer feedforward neural networks (SLFs). As the ELM has the advantage of fast learning speed and good generalization performance. he simulation results have shown a promising prospect for extreme learning machine in the field of reservoir engineering in particular and oil and gas exploration in general, as it outperforms A and SVM. Permeability estimation; well logs; extreme learning machine;, reservoir characterization; support vector machine; artificial neural networks; I. IRODUCIO Permeability is one of the most important reservoir properties, and its prediction has been one of the fundamental challenges to petroleum engineers and researchers []. Accurate knowledge of permeability property is required to determine the amount of oil or gas present in reservoirs, the amount that can be recovered, the flow rate of oil or gas, the forecast of future production, and the design of production facilities. he overall reservoir management and development requires accurate knowledge of permeability [, ]. Permeability or flow capacity is the ability of porous rock to transmit fluid [3]. he fact that a rock is very porous does not necessarily translate to being very permeable. Permeability is the ease with which fluid is transmitted through a rock's pore space. It is a measure of how interconnected the individual pore spaces are in a rock or sediment and it is a key parameter associated with the characterization of any hydrocarbon reservoir [4]. In fact, many petroleum engineering problems cannot be solved accurately without having an accurate permeability value [4]. During the past few decades, numerous efforts have been made to forecast permeability given well log parameters, which include electrical conductivity, density, sonic travel time, neutron porosity, total porosity, bulk density, resistivity and water saturation, through laboratory measurements. In some oilfields, representative values for permeability obtained from different locations are available. Petroleum engineers generally use regression analysis as the main tool to correlate these data with the well log parameters [5-8]. In these works, it was generally assumed that a linear or non-linear relationship exists between permeability and other properties of the rock, like well log parameters mentioned earlier, but unfortunately the method has failed to solve permeability prediction problems [, 9]. he recent success recorded in the application of artificial neural networks (A) to solving various engineering problems has drawn researchers attentions to its potential viability in the petroleum industry. hus, in attempt to resolve problems associated with the parametric approach, the standard As have been used to provide better prediction models [, ]. hese works yielded a significant prediction improvement in the oil and gas industries. See [-4] for further works carried out in this direction. However, the technique still suffers from several drawbacks. hese shortcomings include, among others, the over-fitting problem, low speed operation with attendance high computational time etc. Many works have been carried out by researchers in trying to overcome some of these shortcomings, resulting in several variants of A, among of which is, extreme learning machine that formed the core of this work. Extreme learning machine (ELM) is a recently introduced learning algorithm for single-hidden layer feed-forward neural networks (SLFs) which randomly chooses hidden nodes and analytically determines the output weights of SLFs. In general, the learning rate of feed-forward neural networks (FF) is time-consuming than required and this is becoming bottleneck in their applications / $6. IEEE DOI.9/AMS..9 9

2 According to [5], there are two main reasons behind this behavior, one is slow gradient based learning algorithms used to train neural network () and the other is the iterative tuning of the parameters of the networks by these learning algorithms. o overcome these problems, [5-7] proposed a learning algorithm called extreme learning machine (ELM) for single hidden layer feed-forward neural networks (SLFs). It is stated that In theory, this algorithm tends to provide the best generalization performance at extremely fast learning speed since it is a simple tuning-free algorithm [4]. his unique ability has been demonstrated by the empirical results from this study, which indicate that ELM produce good generalization performance better than the classical feedforward neural network and support vector machines, at extremely high speed. he main objectives of this study are () to investigate the feasibility of ELM in forecasting permeability from well logs; () to develop a new intelligence framework, based on ELM, for predicting permeability from well logs using real industrial reservoir well log data; and (3) to investigate how earlier commonly used standard neural network and support vector machines compare in their performance in predicting permeability of carbonate reservoirs from well logs. he rest of this paper is organized as follows. Section presents the proposed the support of ELM for well logs prediction. Section 3 provides empirical study, implementation process and comparative studies. Results and discussions are also presented in this Section. he conclusion and future work are provided in Section 4. II. HE PROPOSED PERMEABILIY PREDICIO MODEL In this work, extreme learning machine framework has been utilized for predicting permeability from well log parameters, based on distinct real-industrial reservoir data. he introduction of ELM is a good and welcomed development, as in the past, it seems that there exists an unbreakable virtual speed barrier which classic learning algorithms cannot break through and therefore feedforward network implementation then take a very long time to train itself, independent of the application type whether simple or complex. Also ELM tends to reach the minimum training error as well as it considers magnitude of weights which is opposite to the classic gradient-based learning algorithms which only intend to reach minimum training error but do not consider the magnitude of weights. Also unlike the classical gradient-based learning algorithms which only work for differentiable activation functions ELM learning algorithm can be used to train SLFs with non-differentiable activation functions [6]. According to [5], Unlike the traditional classic gradient-based learning algorithms, like back-propagation method, facing several issues like local minimum, improper learning rate and over-fitting, etc, the ELM is a simple tuning-free three-step algorithm that tends to reach the solutions straightforward without such trivial issues. A. he Learning Process for the Proposed Permeability model based on ELM Framework Let us first define the standard SLF (single-hidden layer feed-forward neural networks). If we have samples (x i, t i ), where x i = [x i, x i,, x in ] R n and t i = [t i, t i,, t im ] R m, then the standard SLF with hidden neurons and activation function g(x) is defined as: βigw ( i. xj + bi) = oj, j=,..., (), where w i = [w i, w i,, w in ] is the weight vector that connects the ith hidden neuron and the input neurons, β i = [β i, β i,, β im ] is the weight vector that connects the ith neuron and the output neurons, and b i is the threshold of the ith hidden neuron. he. in w i. x j means the inner product of w i and x j. SLF aims to minimize the difference between o j and t j. his can be expressed mathematically as: β gw (. x+ b) = t, j=,..., () i i j i j Or, more compactly, as: H β = (3) where gw (. x+ b)... gw (. ) x + b.. H( w,..., w, b,..., b, x,..., x )..... =.. g( w. x + b)... g( w. x + b ) β., β =.. β m. and =.. m As proposed by Huang and Babri [8], H is called the neural network output matrix. With the above SLF specification background, thus the training procedures for the proposed ELM based model for permeability estimation can be summarized in the following algorithmic steps. See [5, 7] for further details on the workings of ELM algorithm. 3

3 Input- he inputs to the system, include the well log inputs parameters (input x i R n and target t i R m ), activation function, and the number of hidden neuron,. Output- he outputs of the ELM system are the target permeability values and weights of the layer. Mathematically, given a training set n m = {( xi, ti ) xi R, ti R, i =,..., }, activation function g(x), and the number of hidden neuron =, then, do the following: Step : Initialization. Assign random values to the input weight w j and the bias b j, j =,, Step : Find the hidden layer output matrix H. Step : Find the output weight β as follows: β = H where β, H and are defined in the same way they were defined in the SLF specification above (Equations, and 3). III. EMPIRICAL WORK, DISCUSSIOS AD COMPARAIVE SUDIES In order to carry out empirical study, a real-industrial well logs database from Middle Eastern reservoir well, (code-named Well-) with 477 data points was acquired, having well log inputs parameters that include C (electrical conductivity), DRHO (density), D (sonic travel time), MSFL (Micro spherically Focused Log), PHI (eutron porosity), PHI (total porosity), RHOB (bulk density), R (Resistivity), and SW (water saturation). hat is, we have inputs x ij,,., ; j=,.,m and target t i, where =477 and m=9. A. Implementation Process o evaluate performance of the proposed ELM modeling scheme, each database is divided, using the stratified sampling approach, into 8% training set and % testing set for estimating how the investigated model performed on new unseen data. For testing and evaluation of the newly developed framework, and to carry out effective comparative studies with other earlier methods, most common statistical quality measures that are utilized in both petroleum engineering and data mining journals were employed in this study. hese include correlation coefficient (R ), average absolute percent relative error (E a ) and root mean squared error (RMSE). he training set is used to build the model while the testing set is used to evaluate the predictive capability of the model. We repeat both internal and external validation processes for each of the considered models. he obtained results are presented in tables that follow shortly. he relative error is the absolute error divided by the magnitude of the exact value. he percent error is the relative error expressed in terms of per. And thus, the average absolute percent relative error (E a ) is the average of the absolute percent relative error for all the cases as follows: E a n i i ya yp = i y Where n is the total number of samples, (actual) values known, and a ya is the original y p is the predicted values B. Results and Discussions he results of comparisons, between the proposed ELM based permeability model and the earlier used artificial neural network (A) and support vector machines (SVM) approaches are summarized in ables and. A good forecasting scheme should have the highest correlation coefficient (R ), the lowest average absolute percent relative error (E a ). It could be easily observed that ELM based modeling scheme outperforms others throughout the reported results, most especially in terms of speed, which indicates its potential superiority for use in online permeability estimation system. ABLE : ESIG RESULS FOR WELL- WIH 477 DAASE. R = CORRELAIO COEFFICIE, SD=SADARD DEVIAIO, EA= AVERAGE ABSOLUE PERCE RELAIVE ERROR, RMSE= ROO MEA SQUARED ERROR ime(s) R RMSE E a ELM Model A Model SVM Model ABLE : RAIIG RESULS FOR WELL- WIH 477 DAASE. R = CORRELAIO COEFFICIE, SD=SADARD DEVIAIO, EA= AVERAGE ABSOLUE PERCE RELAIVE ERROR, RMSE= ROO MEA SQUARED ERROR ime(s) R RMSE E a ELM Model A Model SVM Model From ables and, we could observed on the case of testing result, that ELM based model has the highest correlation coefficient, R =.939, which represent.69% improvement over that of artificial neural network (A) and 5.% improvement over that of support vector machines (SVM). In terms of root mean squared error (RMSE), ELM had 3.33% improvement over A and 9% over SVM, while in terms of average absolute 3

4 percent relative error (E a ), ELM had 43.% improvement over A and 6.8% over SVM. he results from the training set also follow similar trends with ELM taking the lead always. Figure shows the cross plots and the fitting chart for the measured (actual) permeability and the predicted permeability using ELM. hose of A and SVM are contained in Figures and 3, respectively. predicted perm. predicted perm. P erm eab ility raining outcome for permeability using Support Vector Machines R = esting outcome for permeability using Support Vector Machines R = Predicted 5 Actual Data Pt. Figure 3: Cross plots and fitting chart for the measured (actual) permeability and the predicted permeability using SVM Figure : Cross plots and fitting chart for the measured (actual) permeability and the predicted permeability using ELM. predicted perm. predicted perm. P e r m e a b i l i ty raining outcome for permeability using eural etworks R = esting outcome for permeability using eural etworks R = Predicted 5 Actual Data Pt. Figure : Cross plots and fitting chart for the measured (actual) permeability and the predicted permeability using A. Another unique attribute of ELM indicated by the result is its extra-ordinary high-speed operations demonstrated in its very low time taken for both training and testing. For instance in the case of training result, ELM had a training time of.396 which implies that the learning speed of ELM is about 9 times that of A and about 33 times that of SVM. As for the testing time, ELM had a speed that is times that of A and 3 times that of SVM. It must however be noted at this point the seeming unusual behavior of support vector machines (SVM) implemented due to the fact that it had slightly better testing performance compared to its training performance. his behavior, in reality, is not unusual as it is rather a rare and unique quality of techniques that are less prone to over fitting problems. Similar results on SVM were earlier reported in the work of [9] where they invested more time and efforts calibrating the SVM approach because SVM was one of their major proposed methods. hey made use of SVM for the offline recognition of Arabic (Indian) numerals, and their presented SVM sometimes achieve higher testing accuracy than its training accuracy. his phenomenon actually demonstrated the reliability of SVM in its ability to do well in the face of unseen new dataset unlike some others like the classical A that may achieve higher training accuracy but perform below expectation when interacting with new unseen datasets in some situations. From the overall reported experimental results, it could be easily noted that ELM performed outstandingly better in all fronts. his is evident as its quality measure values are consistently better than others, with its speed of operations better than others in manifolds. his indicates that ELM is able to deal with the nature of reservoir well log data with good accuracy and at a higher speed. 3

5 IV. COCLUSIO AD RECOMMEDAIOS In this study, middle eastern industrial reservoir data set were used in investigating the feasibility, performance, and accuracy of the proposed ELM based modeling scheme for predicting permeability from well log. he following conclusions and recommendations could be drawn based on previous analysis, discussions, deep investigation, experiments, and comparative studies in this work: A new intelligent modeling scheme, based on the ELM has been investigated, developed and implemented, as predictive solution, that has better generalization ability with higher speed, for predicting permeability from well logs. Validation of the framework is done using real industrial well log data. In-depth comparative studies have been carried out between this new framework and the standard neural networks and support vector. Empirical results from simulations show that the proposed model outperformed all the compared models in all fronts, including its high processing speed, which is an indication of ELM suitability for use in online prediction systems. Finally, a new model based on ELM, has been proposed and implemented, which avoid all the pitfalls of the conventional A training algorithms while producing better generalization within the shortest possible time. As a form of future work recommendations, it is suggested that other models like that of porosity, history matching, lithofacies and other reservoir engineering properties could be built using this ELM based prediction framework. ACKOWLEDGME his acknowledgement goes to Ministry of Higher Education, Malaysia and Universiti eknologi Malaysia for supporting the research fund and conducive environments in conducting the research under the Vot FRGS///SG/UM//(S). Also, the King Fahd University of Petroleum and Minerals (KFUPM) Saudi Arabia is acknowledged for some of the facilities utilized during the course of this research work from KACS project 8-OIL8-4. REFERECES [] B. Balan, S. Mohaghegh, and S. Ameri.: State-of-the-Art in permeability determination from well log data: part I-A comparative study, model development. Proc. SPE Eastern Regional Conference and Exhibition, West Virginia995. [] B. Balan, S. Mohaghegh, and S. Ameri : State-Of-he-Art in Permeability Determination From Well Log Data: Part - Verifiable, Accurate PermeabiIity Predictions, the ouchstone of All Models. Proc. SPE Eastern Regional Conference and Exhibition, West Virginia, USA, September [3] U. Ahmed, S.F. Crary, and G.R Coates.: Permeability Estimation: he Various Sources and heir Interrelationships, Journal of Petroleum echnology, 99 [4] J.S. Lim, and J. Kim : Reservoir porosity and permeability estimation from well logs using fuzzy logic and neural networks. Proc. he Society of Petroleum Engineering (SPE), Asia Pacific Oil and Gas Conference and Exhibition, Perth, Australia, 4. [5] M.P. ixier : Evaluation of Permeability From Electric-Log Resistivity Gradients, Oil & Gas Journal, 949, (June), pp. -3 [6] S.J. Pirson : Handbook of Well Log Analysis (Englewood Cliffs,.J., Prentice-Hall, Inc., ) [7] A. imur: An Investigation of Permeability, Porosity, and Residual Water Saturation Relationship for Sandstone Reservoirs, he Log Analyst, 968, 9, (4), pp. 8 [8] W.A. Wendt, S. Sakurai, and P.H. elson,: Permeability Prediction From Well Logs Using Multiple Regression (Academic Press, ) [9] A.E. El-Sebakhy, A. Abdulraheem, M. Ahmed, A. Al-Majed, P. Raharja, F. Azzedin, and. Sheltami : Functional etwork as a ovel Approach for Prediction of Permeability in a Carbonate Reservoir. Proc. SPE Conference, Bahrain7. [] H.C. Chang, D.C. Kopaska-Merkel, H.C. Chen, and S.R. Durrans,: Lithofacies identification using multiple adaptive resonance theory neural networks and group decision expert system, Computers and Geosciences, Elsevier,, 6, (5), pp [] Mohaghegh, S., Arefi, R., Ameri, S., and utter, R.: Petroleum Reservoir Characterization with the Aid of Artificial eural etworks, Journal of Petroleum Science & Engineering, 996, 6, pp [] Wong, K.W., and Gedeon,.D.: A modular signal processing model for permeability prediction in petroleum reservoir: eural etworks for Signal Processing. Proc. IEEE Signal Processing Society Workshop. [3] El-Sebakhy, E.A., Sheltami,.R., Al-Bukhitan, S.Y., Shabaan, Y.M., P., R.I., and Y., K.: Support vector machines framework for predicting PV properties of crude oil systems. Proc. he 5th Middle East Oil Show, Society of Petroleum Engineering (SPE) Bahrain 7. [4] El-Sebakhy, A.E.: Forecasting PV properties of crude oil systems based on support vector machines modeling scheme, Journal of Petroleum Science and Engineering, 9, 64, (-4), pp [5] Huang, G.B., Zhu, Q.Y., and Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. Proc. Proceedings of International Joint Conference on eural etworks (IJC4), Budapest, Hungary, 5 9 July 4. [6] G.B. Huang, Q.Y. Zhu, K.Z. Mao, C.K. Siew, P.., Saratchandran, and. Sundararajan.: "Can threshold networks be trained directly?, IEEE rans. Circuits Syst. II, 6, 53, (3), pp [7] G.B. Huang, Zhu, Q.Y., and C.K. Siew: Extreme learning machine: heory and applications 7 (6) 489 5, eurocomputing, Elsvier, 6, 7, pp [8] G.B. Huang, and H.A. Babri,: Feedforward neural networks with arbitrary bounded nonlinear activation functions. 9():4 9, IEEE rans eural etwork, 998, 9, (), pp. 49 [9] Mahmoud, S.A., and Olatunji, S.O.: Automatic Recognition of Off-line Handwritten Arabic (Indian) umerals Using Support Vector and Extreme Learning Machines, International Journal of Imaging, 9,, (A9), pp