Plants Leaf Diseases Detection Using Deep Learning

Agriculture improvement is a national economic issue that extremely depends on productivity. The explanation of disease detection in plants plays a significant role in the agriculture field. Accurate prediction of the plant disease can help treat the leaf as early as possible, which controls the economic loss. This paper aims to use the Image processing techniques with Convolutional Neural Network (CNN). It is one of the deep learning techniques to classify and detect plant leaf diseases. A publicly available Plant village dataset was used, which consists of 15 classes, including 12 diseases classes and 3 healthy classes. The data augmentation techniques have been used. In addition to dropout and weight regularization, which leads to good classification results by preventing the model from over fitting. The model was optimized with the Adam optimization technique. The obtained results in terms of performance were 98.08% in the testing stage and 99.24% in the training stage. Next, the baseline model was improved using early stopping, and the accuracy increased to 98.34% on the testing set and 99.64% on the training set. The substantial success rate makes it a valuable advisory method to detect and identify


INTRODUCTION
The health of the plants is important for any agricultural society. Plants are essential elements of life that live on earth and play a significant role in our world. People in this world depend mainly on plants for food, either directly or by using it as feed for different animals [1]. It played essential roles in producing various vegetables and fruits, and medicinal plants that contain organic substances for the manufacture of pharmaceutical compounds and forests, which are the primary source of wood [2]. Crops, especially in the tropics, subtropics and temperate regions of the world, are affected by various pests and diseases [3]. Plant diseases require complex interactions between the host plant, the virus, and its vector [4]. The context of this issue is often connected to the consequences and alterations to the environment of climate change. Climate change primarily affects the regional climatic variables such as humidity, temperature, and precipitation, which then serve as the basis for the destruction of a crop by pathogen, viruses, and plagues. Thus, directly impacting the population, including the effect it has on the economy, health and livelihood [5]. The fact that today, illnesses are spread globally more quickly than ever before complicates the situation. New diseases can be discovered in areas where they have not been reported previously, and no local knowledge exists. Inappropriate use of pesticides can cause long-term pathogens to develop resistance that greatly reduces the ability to combat them [6][7][8]. One of the foundations of precision agriculture is the timely and reliable diagnosis of plant diseases [9]. Plant diseases typically consist of fungi, bacteria, viruses, molds, etc. Farmers or experts are generally aware of plants' diseases and diagnoses by naked-eyed [10]. Plant pathologies can be identified in several ways. There can be no visible symptoms in certain cases, or the effect becomes visible too late for intervention, and a sophisticated examination is needed in these situations [11]. Plants must be protected from diseases to ensure its quality and quantity [12]. To choose the right treatment at the right time to prevent its spread, an effective protection strategy should begin with early diagnosis of the disease [13]. The annual losses in crops reach billions of dollars across the world due to plant diseases. Therefore, early detection of diseases in plants has become especially important to reduce economic losses. There are various strategies to detect infections in plants in the initial stages to reduce diseases that occur in plants. A traditional strategy for detecting plant leaves diseases is naked-eye observation techniques, and it is not effective, especially for an enormous harvest. Using the technique of digital image processing and deep learning, disease plant detection becomes more efficient, accurate, and less effort and time consuming [14]. In this context, a range of experiments and studies demonstrated the effectiveness of image processing and analysis through several techniques and the use of deep learning. This supports and facilitates research to enhance image analysis reliability, correctness, and accuracy in the past two decades. Nevertheless, the advancement has to do with the growing precise use of computer vision and the theory of analysis in this area. The precision of classification in many areas like the classification of crops and plant diseases can be improved. The paper is structured as follows: Related works are discussed in section 2. Section 3 introduces a Convolution neural network. Section 4 explores in depth the materials and methodologies. Section 5 addresses the findings and implementation. Section 6 depicts a conclusion and future trend.

RELATED WORK
Several experiments were conducted over the years to improve the diagnosis of plant diseases. The different studies to detect disease in the plant leaf are listed in this section.
Several conventional Machine Learning (ML) models are used to classify plant diseases [15]. Still, recent studies have shown that about traditional ML methods, deep learning methods are important. This article focuses, therefore, on DL-based models for plant disease classification. The author explains in [16] the methodology for detecting early, exact plant diseases using the Artificial Neural Network (ANN) and a range of imaging techniques. Since the method proposed is based on the ANN classifier and the Gabor filter for functional extraction, better results are obtained by a recognition rate of up to 91 %. An ANN classifier classifies various diseases of the plant and uses the blend of textures, colors, and characteristics to classify these conditions. The CNN algorithm, which was used to separate nutritious cucumbers from diseased cucumbers using leaf images, was suggested in 2015 by Kawašaki and others [17]. Two serious CNN infections, "MYSV" (Melon Yellow Spot Virus) and "ZYMV," were diagnosed in this study (Zucchini Yellow Mosaic Virus). The dataset contains 800 pictures of cucumber leaves (200 with ZYMV, 300 with MYSV, or 300 with healthy leaves). The dataset has been expanded by rotating image transformations. Researchers have indicated that the 3-layer CNN structure used was (convolutional layers, pooling layers, and normalization layers). The device activation method is the "Rectified Linear Unit" (ReLU) function, which was accurately measured in the analysis (94.9%). The Malvika Ranjan1 et al. [18] determination process, which is usually noticeable and involves correct expertise and experimental procedures, was introduced in 2015. It catches the picture of the unhealthy leaf. The colors for such images as "HSV" characteristics were obtained as a result of segmentation processes. Later training was given to the ANN to recognize infected and healthy leaves. The performance of the ANN classification amounted to more than 80%, making the classification accurate. In 2019 a method to identify the sugar beet plant's diseases was established using 155 different pictures of diseases of this plant, including Ozguven, Mehmet Metin, and Kemal Adem [19]. A CNN algorithm structure using faster automation parameters with the R-CNN structure was planned. The Rectified Linear Unit (ReLU) function was used in this framework as well. The overall correct rate reached 95.48%, and they assumed this approach would minimize the amount of time and effort involved in identifying sugar beet plant diseases in large farms. In 2019, the CNN model for the classification of the disease on plant village leaves was proposed by Geetharamani G, Arun Pandian J., [20], in contrast to the ML techniques such as the SVM, Decision Tree (DT), and K-Nearest Model Neighbor (KNN). This model was also better than the popular AlexNet, ResNet, VGG-16, and Inception-v3 architectures. Therefore, this article focuses on DL models for classifying plant diseases. The model achieves 96.46 % rating accuracy after an extensive simulation. By conducting a routine survey of Uganda farms taking 10,000 cassava leaf photos in 2020, Sambasivam, G., and Geoffrey Duncan Opiyo, [21] made the dataset. Their analysis was planned to categorize 5 types of cassava. Deep learning was used to build and structure its own classification using the "Convolution Neural network" algorithm, and numerous problems occurred with the data balance of the used data collection. The accuracy of the classification of cassava leaf diseases was higher than 93 %.In 2020 Marwan Adnan Jassim and Jamal Mustafa Al-Tuwaijri [22], presented a system for detection and classification of plant diseases based on the convolution neural network and used the plant village dataset. They used model includes several layers such as convolution layers, non-linear layers, max pooling layers, and fully connected layers. In addition, they utilized regularization such as batch normalization. They achieved excellent accuracy in training and testing, as they obtained (98.29%) accuracy of training, and (98.029%) for testing for all datasets that were used.

CONVOLUTIONAL NEURAL NETWORK
The human brain's learning capacities consisting of neuronal cells interconnected with synapses inspire artificial neural networks. ANNs are not appropriate for photographs as these networks result in image size overfitting. The CNNs are used primarily for image recognition tasks. The CNNs are the most common ones.
The key difference between the Artificial Neural Network (ANN) and CNN is that only the last CNN layer is completely connected. In contrast, every neuron in ANN is connected. Installing neural networks are employed instead of handcrafting to automatically learn a hierarchy of features that can be used for classification. The input picture is successively transformed to a hierarchy of the characteristic maps by using learned filters. The hierarchical approach facilitates more complicated learning. As invariant characteristics in higher levels of translation and distortion. A CNN has four types of layers: Convolution layer, Rectified Linear Layer (Re Lu), Pooling Layer (POOL), Flats layer, and Fully-Connected Layers (FC). Figure 1 shows the standard CNN building block.

Figure 1-Building block of a typical CNN[23]
A.Convolutional Layer Convolution stands for the primary layer for extracting features from an image input. It maintains the connection among pixels using small squares of data based on image capabilities. The layer consists of filters or kernel sizes that are small (for example [3x3] as in Figure 2). These 3x3 filters are slid over the width and height of the image with the image input matrix. Second, the function matrix is pixel-by-pixel multiplied with the chosen image square. Then add the values and then split by the total pixels (in our example, it is 9 due to 3x3 filter size). In a new matrix, the value obtained is added. This method decreases the picture without losing any features. The standard recovery protocol is depicted in Figure 2. The CONV layer accepts the input volume [W1 x H1 x D1], where W1 represents input width, H1 represents input height, and the D1 represents input Depth. The neuron outputs in this type of layers are determined by applying the product between their weights and a local region they are connected to in the input volume. The resulting volume [W2 x H2 x D2] is referred to as the convolution map where the width is W2, the height is H2, and the depth is D2 where D2 filters or convolution kernels have been decided [19]. Convolution maps are provided with equations (1), (2), (3) where W2, H2, D2 are presented as equivalent volumes.
(3) With: F: spatial extend of the filter. K: number of filters. P: zero paddings (hyperparameter controlling the output volume). S: stride (hyperparameter with which we slide the filter).

B. Rectified Linear Unit Layer
The pixels that are not required are deactivated in the Re Lu layer and only the major pixels are retained. We get both positive and negative pixel values from the convolution layer. For additional feature discovery, the positive pixels are significant, and the negative values are less important. The ReLu layer transforms the pixel to either 0 or 1. If the pixel value is negative, it is converted to 0 and maintains the same value greater than 0. The standard activity of ReLU is shown in Figure 3.

C.Pooling Layer
Pooling stands for down sampling an image. To create a single output, it takes a small region of the coevolutionary output to input and sample it. There are diverse forms of pooling, such as maximum pooling, average pooling, and so on. As shown in Figure 3, Max Pooling takes on the biggest pixel value of an area. Pooling reduces the number of parameters to be measured but invariantly converts the network into form, size, and scale, as shown in Figure  4.
(6) D. Flattening layer After several repeats of the convolution layer, non-linear layer, and pooling layer, the data comes to the flattening step, namely, converted into a one-dimensional (1D) array of numbers (or vector) to create a single long feature vector. Moreover, it is connected to the finishing classification model, which is known as fully-connected layer. E.Fully Connected Layer CNN's completely-wired layer has neurons fully connected to all of the previous layer's neurons. The design used is stacked according to several FC layers. Often it is the last layer to predict the output, or the mark of the input class used by CNN. Various activation functions such as SOFTMAX are used to classify multi-class problems. Therefore, it has [1x1xM] output dimensions, where M is the number of groups or labels to be classified.

MATERIALS AND METHODOLOGIES
This section explains further the entire process for creating the deep CNN model of plant disease identification. The method is divided into the needed stages. The system comprises

Al-Akkam and Altaei
Iraqi Journal of Science, 2022, Vol. 63, No. 2, pp: 801-816 807 several steps; beginning with selecting images to be identified using profound neural networks. Figure 5 demonstrates the principal machine structure. Based on Figure 5, the first step in this process is the image acquisition stage. This demonstrates the construction of our proposed system to detect and identify leaf diseases. A.Dataset (Image acquisition) The open-source plant village is a dataset with 34,934 images of leaves of crop categorized in 15 different classes, as shown in the Figure 6. Each of the images downloaded is within the RGB color range and saved in uncompressed JPG format, 256 x 256 in size. The dataset contains photos of all key forms of leaf diseases that could affect the three plant types: tomato, pepper, and potato, which were chosen because they are the most famous types of plant in the world, especially in Iraq. This dataset of three crops contains fifteen categories divided into 12 categories of disease leaves and 3 categories of healthy leaves.

B. Image Pre-processing
Before submitting the image for testing and training purposes, it is processed as a shown in algorithm (1). Several operations are conducted for this purpose as follows:  Dataset balancing: Unbalanced dataset is a common issue in all areas. The Studies show that the unbalanced distribution of data for each class has a significant impact on CNN's performance. To tackle this problem, the dataset must be balanced by taking the same number of samples from each intra-class, so the balanced dataset distributions yield the best performance. After the data has been balanced in the proposed work, it took 1,500 samples from each intra-class.  Images resize: To expedite the training process and obtain the model testing calculation realistically, the dataset images are resized into 150 x 150 dimensions. This process helps speed up the training process while preserving the data's integrity in the image from loss.  Labels conversion: Converting each image label to binary levels.  Algorithm (1) illustrates the steps of preprocessing Input: Images dataset Output: Images that Processed Steps: Step (1): Image Acquisition (load images + read images).
Step (4): converted each image label to binary levels Step (5): Normalize input by dividing each pixel value in images by 255.
Step (6): Use the image augmentation as a kind of preprocessing.
Step (7): Splitting the data into two parts, 70% for training and 30% for the testing.

C. Proposed CNN model
The proposed CNN model for plant leaf diseases detection has an input layer, four convolutional layers, four max-pooling layers, a flatten layer, and three dense fully connected layers. Algorithm (2) explains the main stages of the proposed method. Convolutional and pooling layers are used for feature extraction where the following configuration was used: The first convolution layer uses a 3*3 kernel size and 32 filters (kernel). The second convolution layer uses a 3*3 kernel size and 32 filters. The third convolution layer uses a 3*3 kernel size and 64 filters, and the last convolutional layer has 64 filters with 3*3 kernel size. The convolutional stride is one pixel without space padding. Rectified Linear Unit (ReLU) is used as an activation function for all convolutional layers. Each convolutional layer is connected to a 2*2 Max-Pooling layer, which downsamples the input data into half of its original dimension. After several repeats of the convolution layer, non-linear layer, and pooling layer, the data comes to the flattening step, i.e., transformed into a one-dimensional (1D) array of numbers (or vector) to create a single long feature vector. It is also connected to the final classification model where the fully connected layers are used to classify and predict, which is called a fully-connected layer. The first fully-connected layer uses 512 weights with Rectified Linear Unit (ReLU). Also, to prevent overfitting the training set, the dropout regularization method was used with a probability of 0.5. The second fully-connected layer uses 128 weights with Rectified Linear Unit (ReLU) and L2 regularization rate =0.002 to prevent overfitting. To extract the result from the model, the final fully-connected layer uses 15 weights (nodes) representing the number of leaf categories, which are appended with the softmax activation function. The softmax function computes probabilities. Based on these probabilities, disease is classified as one of 15 plant leaf categories.

Algorithm (2)
illustrates the main stages of the proposed method. Input: Model structure before training. Output: Built CNN converges with optimum weight. Steps: Step 1: Provide input vector to the network.
Step 2: Perform convolution using filter to produce a feature map.
Step 3: Pass the obtained feature map through ReLU to introduce non-linearity.
Step 4: Apply pooling operation on obtained feature map, which introduces translation invariance.
Step 5: Repeat Steps 2 to 4 for repetition of layers.
Step 6: The obtained feature maps are passed to first fully connected layer for classification.
Step 8: Perform second fully connected layer.
Step 9: Pass the output (last fully connected) to a classifier such as softmax.
Step 10: Computer loss at the final layer and calculate gradient w.r.t. all the learnable parameters.
Step 11: Backpropagate the error component and update the parameters.
Step12: Perform the forward pass and repeat Steps 2 to 11 using updated parameters until network Converges with the optimum weight.

D.Training and Testing
Network training is a method of acquiring kernels in convolutional layers and weights in fully connected layers, reducing the gaps between performance predictions and actual ground truth labels on dataset training. During our work, 70 percent of data was used for training to ensure, through the extraction of features from the pictures of plant leaves, the network built learns how to differentiate each image on its base. The framework was trained using the Adaptive Moment Estimation (Adam) as an optimization algorithm with the first 0.001 learning rate. A loss function was used to categorical cross entropic loss. The model has been developed over 50 epochs, and the batch size is 32. To detect validation accuracy, the produced model was tested using test data set. The model's performance was calculated based on training accuracy, training loss, validation loss per epoch. Plant diseases were identified and categorized by three different plant types after previous operations, namely tomatoes, peppers, and potatoes. The following segment will present data, detections, and classifications.

EXPERIMENT RESULTS
In this section, the model results will be discussed and the results obtained from this system through deep learning techniques using the CNN algorithm will be presented. The experiments are performed in the environment with windows 10 pro, Intel(R) core (i7) 2.70GHz, RAM 16.0GB, and 64-bit system type. The code was written using python3.  This model was tested with the accuracy metric. Therefore, for all groups with and without diseases, we have plotted the uncertainty matrix. In the study of misclassification between the groups, the confusion matrix is also used as an assessment metric. Each row of the matrix is an example of the expected class, while each column is an example of the original class. The confusion matrix shows the number of correct and incorrect predictions made by the classification model compared to the actual output. The diagonals show the classes that were classified correctly, while the other values show the classes that were classified incorrectly, as shown in Figure 8. The model achieved accuracy after calculating the confusion matrix was 99.24% on the training set and achieved 98.09% on the testing set. This accuracy was obtained by taking the sum of the diagonals and dividing the result by all the numbers in the matrix.

Al-Akkam and Altaei
Iraqi Journal of Science, 2022, Vol. 63, No. 2, pp:   Various splits were analyzed during the experiment for the training and validation data sets. This means that the model was not over-fitting and that the proposed model was tested appropriately. All data rates for training and testing have been tried and start with 50% only for training data collection and 50% for testing until 90% of the training data versus 10% of the testing data. The highest percentage and the best result obtained were 70% for training and 30% for the set of the testing dataset, especially for the testing process that depends on it in the classification and detection process of plant leaf diseases, are shown in Figure 9, which illustrates these percentages versus total accuracy of training data and test data. Later, the model was improved by the algorithm tuning used by the batch size was reduced and the number of epochs increased because this method is one of the methods of improving performance where the batch size used was 16. The number of epochs used was120. In addition to that, early stopping was implemented. To improve the performance of a model, the early stopping regularization was used. When validation errors continue to rise, the model may be forced to stop learning. This can save a lot of time and may even allow for using more elaborate resampling methods to evaluate the model's performance. The model will be evaluated after improving by using a smaller batch size, increasing the epoch, and early stopping. Figure 10 illustrates accuracy progression and loss retraction of the model's implementation during the training phase.   The accuracy metric evaluated the performance of this improved model. As shown in Figure  11, the model achieved accuracy after calculating the confusion matrix 99.64% on the training set and achieved 98.30% on the testing set. Finally, detection and classification accuracy in the improved baseline model will be extremely high. The errors are almost non-existent or exceedingly rare due to obtaining high and excellent results in the training and testing. As a result of the high accuracy obtained and as a shown in Table 1, this study proved its superiority over previous studies that adopted classification based on the convolutional neural network and used the same dataset.

CONCLUSION AND FUTURE WORK
The research proposes a methodology for the identification and classification of diseases by computer facilities and deep learning techniques, with accurate and quick results because of the importance of agriculture in Iraq and the existence of many plant diseases. This analysis was performed using a dataset that includes fifteen different categories, as well as using the CNN algorithm to obtain the results. High results were obtained that exceeded 98% accuracy. This led to an accurate and rapid detection of the type of disease and the type of plant that carries the disease through the image on the leaves of that plant. The algorithm tuning and using early stopping have proven to be highly effective in improving results. It is concluded that the convolutional neural network is highly effective in feature extraction and classification. In future studies, different learning rates and improvements can be applied, in addition to experimenting with a different dataset. An expert system or mobile application can also be developed to identify and classify plant leaf diseases.