PFDINN: Comparison between Three Back-propagation Algorithms for Pear Fruit Disease Identification

The diseases presence in various species of fruits are the crucial parameter of economic composition and degradation of the cultivation industry around the world. The proposed pear fruit disease identification neural network (PFDINN) frame-work to identify three types of pear diseases was presented in this work. The major phases of the presented frame-work were as the following: (1) the infected area in the pear fruit was detected by using the algorithm of K-means clustering. (2) hybrid statistical features were computed over the segmented pear image and combined to form one descriptor. (3) Feed forward neural network (FFNN), which depends on three learning algorithms of back propagation (BP) training, namely Scaled conjugate gradient (SCG-BP), Resilient (R-BP) and Bayesian regularization (BR-BP), was used in the identification process. Pear fruit was taken as the experiment case during this work with three classifications of diseases, namely fire blight, pear scab, and sooty blotch, as compared to healthy pears. PFDINN framework was trained and tested using 2D pear fruit images collected from the Fruit Crops Diseases Database (FCDD). The presented framework achieved 94.6%, 97.3%, and 96.3% efficiency for SCG-BP , R-BP , and BR-BP, respectively. An accuracy value of 100% was achieved when the R-BP learning algorithm was trained for identification.


Qassir
Iraqi Journal of Science, 2021, Vol. 62, No. 9, pp: 3128-3137 3130 fire blight pear scab sooty blotch Figure 1-The three types of Pear fruit disease: the first is fire blight, the second is pear scab and lastly is sooty blotch. The first type is caused by common bacterium (E. A.). The second type is caused by the fungus (V. P.) where its shape likes circular spots. The final type is caused by various fungi

K-Means Clustering Algorithm
The disease detection is the extraction of the affected area by disease out of the fruit image. This helps to extract the affected segment out the whole healthy fruit portion . The segmentation divides the image into significant partitions for better analysis and comprehending of the image. Thresholding and growing areas can be described as two fundamental methods of segmentation. It is the most facilitated whereas the gray image can be segmented according to the threshold value [8]. This method does not supply an effective segmentation of the colored image and then limits the categorization of the different items of the image if it includes multiple areas or colors. Furthermore, it decreases the proper detection of the desired area [9]. K-Means clustering is a simple, fast, and iterative method [10]. This method has been utilized due to: firstly, it is almost all the time mathematically faster than the algorithm of hierarchical clustering, if k is kept little. Secondly, tighter clusters than hierarchical clustering [10], [11]. The flowchart for K means clustering is shown in Figure-2. It is worked by using "centroids", K various randomly points as the initial values in the data, and it gives every data-point to the closest centroid. After every point assignment, the centroid is proceeded to the average of all the points which have been given to it [12]. It is described in the equation (1):

∑ ∑‖ ‖
Where j is stands for the objective function; the P is stands for the number of clusters; the Q is the number of cases; A represents the case q and finally the C is stands for the centroid for cluster p.

Statistical Features
In colour and texture feature extraction analysis, the image texture shows information about the locative positioning of a colouring image. Two kinds of statistical features are used in the proposed PFDINN framework for extracted features: Colour Co-occurrence Method (CCM method) and Features Based on First and Second Order Statistics.

Colour Co-occurrence Method (CCM method)
A colour is extensively utilized for visual attribute and also it is mostly utilized to make a comparison between images [13]. This approach extracts the texture and colour out of any image. In CCM; The angular moment (H), as explained in equation (2), it is utilized to detect homogeneity out of the image, where N is the whole number of strength and f(x, y) represents the image. The intensity of covariance (C) matrix is given in equation (3). The entropies in expressions of sum (S) and Difference (D) are given in equations (4 and 5) where (a, b) are the referral pixel at image location and k=I-x or k=I-y where I is strength of the image [14].

Features Based on First and Second Order Statistics
The features are described to quantify properties of an image region by exploiting space relations underlying the gray level distribution of a given image [15]. Figure-3 explains the statistical features classification.

Figure 3-Statistical based features
The first order histogram is used to obtain various statistical properties that depend on individual pixel values to depict the region of interest. In this statistical classification, the intensity mean (IM), standard deviation (Std), which is the number used to tell how measurements of a group are widespread from the average. The skewness (Skn) of an allocation is the number that indicates the degree of asymmetry of the distribution. The kurtosis (kt) of a distribution is the number that indicates the sharpness of a peek distribution in relative to the normal distribution, which can be extracted from the diseased area of pear fruit since these features give good indication for the intensity distribution [16]. Each of these measures is illustrated in equations (6) Where is symbolize the intensities of diseased area and N is the number of pixels in the diseased area. In second order statistics; the spatial grey level dependence matrix algorithm is easy to implement, which provides information on image homogeneity and can be computed in equation (10). Entropy is the measure of the disorder of the spatial gray level dependence matrix it measures the disorder of the spatial gray level dependence matrix. It is computed by equation (11). The correlation is the measure of the level of grey linear reliance between pixels at the determined locations, which can be computed in equation (12). Local homogeneity is computed in equation (13), and inertia is quantifying local variations presented in the image. It is computed in equation (14) [15].

Neural Network Method
For identification process, there are various methodologies used in researches nowadays, like artificial neural networks, support vector machines and fuzzy logic etc. The back propagation training is a familiar method for practicing artificial neural network [17]. The first level is representing network training while the second level represents testing, So that, choosing the input points and weighting operations are major factors. This model of neural network is a multilayer feed forward passes in two kinds of phases, which are known as propagation and weight upgrade. Three algorithms of back propagation training utilized in this work; the first is depends on Scaled conjugate gradient SCG-BP [19], the second is Resilient R-BP [19] and the third is Bayesian regularization BR-BP [20], [21]. In this work; normalization based on min-max method was utilized as explain in equation below: Where U i : The feature input from 1 to 13 value; t, r are the normalization range; X Min X Max are the upper and lower values.

The Presented PFDINN framework
The goal of this work can be described by detecting and identifying three types of pear fruit diseases; which where fire blight, pear scab and sooty blotch. The PFDINN framework pass through three main phases. In the first phase; the K-Means clustering algorithm is used to locate the infected region out of the pear image; hybrid texture features are extracted in second phase. While in third phase the FFNN using three learning algorithms was utilized for disease identification process, as shown in the Figure-

.1 Image Acquisition stage
It is the first step of any identification system, in the presented framework of the pear fruit 2D image was collected from Fruit Crops Diseases Database (FCDD). FCDD is entirely searchable and maintains comprehensive text search. The primary focus of the FCDD is on supplying a possible information of fruit harvest diseases.

Pear Disease Detection Phase
The first phase of proposed PFDINN framework aims to extract the infected area from pear fruit images based on utilizing k-means clustering approach. Figure-5 shows some results of this phase. The value of K that used in this work was 3.

Feature extraction Phase
In presented algorithm thirteen significant texture features are extracted from the infected area of pear fruit image. In given PFDINN framework, two types of statistical features calculations were used: Color Co-occurrence Method (CCM method) based features, hybrid with the features which are based on the First and the second order statistics: four texture features extracted from the first method and nine texture features extracted form the second method, as explained in section (3.2).

Identification Phase
The Identification process uses three network training algorithms: (SCG-BP, BR-BP, and R-BP). To generalize nonlinear relationships of inputs and outputs dataset; SCG-BP, BR-BP, R-BP were used, the neural network uses two layers of feed forward (FFNN), which are the weight and bias values were updated according to each learning algorithm. The measure of error utilized in this work was cross entropy error (CE error). The sigmoid trigger function was utilized for operating nerve cells in the hidden layer. This supervised learning topology of back propagation has a priority because of its implementation simplicity, as well as the availability of adequate "inputtarget" database for training.

Scaling and Training Step
In this work; a min-max scaling was essentially utilized to prevent larger numbers from overriding smaller ones, as indicated in section (3.3). Pear-Inputs matrix and Pear-Targets matrix; represent the (Input-Target) layers for the FFNN, the Pear-Inputs matrix represents the texture features extracted from the infected area of pear fruit images. 178 input image dataset were obtained from FCDD dataset of fresh pear fruit productions which contains the three types of pear diseases. The input matrix size was 13×178; (13) hybrid texture feature values extracted from each pear image of (178) image, the targets matrix size was 3×178 where each column indicates a disease category with a one in either element 1, 2 and 3 or zero for the rest elements, using binary codes [1 0 0]; [0 1 0]; [0 0 1]. The accessible image dataset was arranged as follows: Training image-set: 124 elements pear images; Validation image-set: 27 pear images and Testing image-set: 27 pear images. As the validation error starts increasing; the training process has been ended. After terminating the training procedure, the trained network was formed.

Testing Step
The execution results of R-BP can be shown in Figure-6; and the comparison with other two training algorithms explains in table (1). It is obvious that the minimum squared mistake in case of BR-BP approach is obtained at 23 intervals, its gradient was 9.1143e-08 and in case of SCG-BP approach it has required 18 intervals, its gradient was 0.00062873. In case of R-BP, its gradient was 0.0005197 and achieved at 14 epochs.

Experimental Results
The whole framework was implemented using Matlab software tool box. 178 pear fruit image samples were considered from dataset FCDD. 124 pear images were used for training step; 40 samples of fire blight disease, 40 samples of pear scab disease and 44 samples of sooty blotch disease. For identification of pear image samples using three BP learning algorithms (SCG-BP, R-BP and BR-BP), the network model in each case was instructed with 70% of the images of pear for each disease kind, and the residual 15% pear images can be utilized for each of testing and validation. The accuracy percentage can be defined as the proportion of the correctly recognized pear image samples according to the number of tested samples of pear image. The percentage of accuracy was obtained by the equation (16), and best execution of each neural learning model plot is drawn in Figure-7.
The Neural model was implemented using 13 neurons for input layer, 12 for the hidden layer and 3 for the output layer. Table (1) summarizes the result obtained by each model, so it is obvious that R-BP (resilient back-propagation) needs less epochs than SCG-BP algorithm and BR-BP for the same iteration. Thus; the procedure of training was high performance in case of resilient approach than the other two approaches. The training execution and confirmation schemes showed that the R-BP approaches were faster training methods than (SCG-BP and BR-BP) methods. The transfer function in every network plays a prime role to determine the output values. The best Performance and Error Histogram, the latest shows how accurately the trained model fits the dataset of each neural learning model and is drawn in Figures-(7) and (8) respectively.

Conclusion
In the presented PFDINN framework, it seems very significant to have a demonstrated approach for categorizing the glitches on the pear fruit spontaneously. This work was pass with three main phases: Pear Disease detection, hybrid statistical features extraction and identification process. It was essential utilized for automatically uncovering the pear disease area in addition to identifying these disease types. This work has used K-means clustering algorithm for partitioning of pear fruit image to divide the healthy area, disease region and back-ground region of the inserted pear fruit image according to extracted hybrid statistical features, then fed these features to BP neural model for identification process. The presented framework qualified the identifications of three kinds of pear fruit diseases with 94.6%, 97.3% and 96.3% efficiency for SCG-BP, R-BP, and BR-BP respectively. The robustness in this work was offering an access for effective discrimination to these diseases. This system was utilized to take the place of the instructions pear identification methods, which can be used by cultivation proficient in recognizing the correct pesticide and its amount to get the problem under control in an organized and successful manner. In future work, to increase accuracy rate by using large number of pear fruit samples and the work implemented more diseases kinds by utilizing this approach.