Image Focus Enhancement Using Focusing Filter and DT-CWT Based Image Fusion

Combining multi-model images of the same scene that have different focus distances can produce clearer and sharper images with a larger depth of field. Most available image fusion algorithms are superior in results. However, they did not take into account the focus of the image. In this paper a fusion method is proposed to increase the focus of the fused image and to achieve highest quality image using the suggested focusing filter and Dual Tree-Complex Wavelet Transform. The focusing filter consist of a combination of two filters, which are Wiener filter and a sharpening filter. This filter is used before the fusion operation using Dual Tree-Complex Wavelet Transform. The common fusion rules, which are the average-fusion rule and maximum-fusion rule, were used to obtain the fused image. In the experiment, using the focus operators, the performance of the proposed fusion algorithm was compared with the existing algorithms. The results showed that the proposed method is better than these fusion methods in terms of the focus and quality.


Introduction
Image fusion is a technique for combining complementary information obtained from different sensors to enhance the visual perception of the human eye or to facilitate the image processing and computer vision. Image fusion technology is used in many applications such as medical fields, military, video surveillance, remote sensing, etc. [1]. The merging process is carried out either in the frequency domain or in the spatial domain in three levels: pixel, feature, and decision fusion levels. Many fusion methods and techniques have been implemented to improve and develop the image merging process to reach the best results. Various recent surveys outline these methods [2][3][4][5]. Many transforms are used in the fusion field, like Stationary Wavelet Transform, Discrete Wavelet Transform, Curvelet Transform, etc. Discrete wavelet transform (DWT) is widely used in image fusion applications due to its results that have good localization properties, where the fusion is done in frequency and spatial domains. Despite its benefits, it also has some disadvantages. When the input data are shifted due to the down sampling operation, this causes a difference in the wavelet amplitude. In addition, there is a loss of directional selectivity [6]. Kingsbury proposed Dual Tree-Complex Wavelet Transform (DT-CWT) in 1998 to solve these problems. DT-CWT have good invariance of wavelet shift and directional selectivity. These features of DT-CWT provide good fusion output [7]. Sharpening or focusing of an image is a procedure which is achieved to give it a sharper look. Sharpness is an important part of image processing. Increasing the sharpness by a specific percentage improves the edges of the images, provided that this image is not very clear. By using DWT algorithm in image fusion, the original images are decomposed into low frequency and high frequency coefficients. On these coefficients, the fusion rules are applied. The two most used fusion rules are the averaging and maximizing of the coefficients. The inverse of DWT is achieved to produce the final image [8,9]. The problem of image fusion based on DWT is the absence of shift invariant and the blurring of the edge of the fused image. While DT-CWT is distinguished by certain advantages compared to DWT [10,11]; in two and higher dimensions, DT-CWT is directionally selective and nearly shift invariant. It performs these features with a redundancy detail of only 2-dimention for d-dimensional signals. It also reduces aliasing. Due to these advantages of DT-CWT, it has been utilized in many applications, such as noise suppression [12][13][14][15], face recognition [16][17][18], speech enhancement [19], and image fusion [20,21]. Although DT-CWT based image fusion is distinguished by these properties, it still gives out of focus (blur) results because it uses two DWT filters. In in this paper, the combination of focusing filter and DT-CWT based image fusion is proposed to improve the focus of the resulting image. The rest of this paper is organized as follows. Section 2 presents the methods of the topic. Section 3 explains the proposed image fusion. Assessment measurements are presented in Section 4. In Section 5, experimental results are discussed. Finally, Section 6 provides the conclusions.

DT -CWT
DT-CWT is a form of wavelet transform which was proposed by Kingsbury in 1998. It uses a dual DWT in a parallel way. It creates complex coefficients to produce real and imaginary trees using some low pass and high pass filters. DT-CWT can be expressed as [22,23]: ( ) is the real part of the transformation with filter of even-length, ( ) is the imaginary part of the transformation with filter of odd-length. is a low pass filter and is a high pass filter for the real tree.
is a low pass filter and is a high pass filter for the imaginary tree. The forward transform is performed to obtain the inverse of the DT-CWT, where the tree and the tree are each inverted. The final signal is obtained by averaging the outputs of and . More details can be found in [24].

The Proposed Focusing Filter
Focusing filter is proposed to increase the focus or sharpness of the image containing blur. It consists of two parts: Wiener filter (WF) and Sharpening Filter (SF). WF restores the noisy and blurred image [25]. The statistics of the WF can be defined as: ) is the FF of the degradation function, ( ) is the conjugate complex of ( ) , and SNR is the SNR which is the ratio of the power spectrum of the noise to the power spectrum of the image signal . WF works as a low-pass filter which reduces the noise but blurs the line and edges in the image [26]. Thus, the combination between SF and WF will contribute to solve this problem. The result of this combination is an image with highlighting of lines and edges. The SF works as high-pass filter which preserves the lines and edges in the image. The canter of the SF matrix is a positive value and the surrounding values are negative [27]. The SF matrix can be in this general form: , where , and are any positive real numbers, and usually The Focusing Filter steps can be explained as follows: 1-Reading the blurred image ( ( )).

4-
Converting ( ) into Fourier Transform to obtain ( ). ) to obtain the focused (sharped) image ( ( )) (6) The focusing filter works as a low pass filter and, in parallel, as a high pass filter. This means that it removes noise and blur at the same time from the input image. Figure-2 shows the proposed image fusion process using the focusing filter and DT-CWT filter. Image fusion type here is a pixel-level fusion which is a low level of fusion, where the original mages are fused pixel by pixel. The following is an explanation of the algorithm steps:

The Proposed Image Fusion Method
Step 1: Two processes are taken before the fusion process: first, image registration, which is a process of converting the two images in one coordinate system. In this work, assume that the images are on the same coordinate system. Second, converting the images into 2D images (gray levels).
Step 2: Each one of the registered images ( ) is filtered using the focusing filter to increase the focus of the input images. This process can be defined as follows: ( ) and are the output of the focusing filter, .
Step 3: The DT-CWT filter of 3 level decomposition is applied to each of the focused images obtained from the previous step. The DT-CWT coefficients are obtained from the decomposed focused images (Low frequency and High frequency coefficients). The maximum fusion rule was used, which is a simple fusion rule that selects only the largest coefficients from the focused images. The process of this step can be defined as follows: is low frequency coffecients and is high frequency coefficients for each filtered images. and are the results of the maximum fusion rule.
Step 4: The inverse of the DT-CWT is applied to obtain the fused image. The process of this step can be defined as:

Assessment measurements
The performance of the proposed fusion method based on focusing filter and DT-CWT filter is evaluated using the focus metrics which are: Image contrast (CON), Gaussian derivative (GD), Gradient energy (GE), Gradient energy (GE), and Variance of wavelet coefficients (VoWAV). Table-1 shows a brief description of these metrics. The greater the focus in the image, the greater the metric values. More focus metrics can be found in [28].

Metric name Format
Image contrast (CON) is the firt derivative of the two directions of the image ( ( ) and ( )).

Variance of wavelet coefficients (VoWAV)
The variance of the wavelet coefficients where is the corresponding window of in the DWT subbands , and denote the mean value of the respective DWT subbands within .

Experimental results
Performance evaluation of the proposed fusion method was achieved using some of no reference operators, which are image Contrast (CON), Gaussian derivative, (GD), Gradient energy (GE), and Variance of wavelet coefficients (VoWAV). Experiments were accomplished on three different modalities of images of size pixels, as shown in Figure-3. These datasets were gathered from (https://www.mathworks.com). The dataset of images can be named as pair1 for multi-focus images, pair2 for visible-infrared images, and pair3 for multi-modal medical images, MATLAB b 2017 was used to perform the proposed algorithm in Windows 10. Figure-4 shows the fused images that are obtained using image fusion based on DWT, DT-CWT, and the proposed fusion method with fusion rule based on maximum section. The input and output images generally have some blur. Thus, we wanted to check the amount of blur in these images by using a blur metric before the fusion process. This metric is based on distinguishing between different levels of perceptible blur on the same image. The value of this metric ranges between 0 and 1, from best to worst [29]. Table-2    We notice from the tables (Table-3, Table-4, and Table-5) that the DT-CWT based image fusion gave results of a higher focus value than the DWT based image fusion, because the DT-CWT based image fusion was discovered to solve the problems that the DWT based image fusion suffers from, ensuring invariant in approximate shift and good directional selectivity. This led to high-quality fused images. While the proposed method based on focusing filter and DT-CWT filter gave better results than the results of DT-CWT and DWT based image fusion techniques because adding the focus filter contributed to increase the focus of the image with a high quality of the combined image.

Conclusions
Most of the fusion methods based on the wavelet transform give good results. Despite that, the results of these methods suffer from the blur because these methods used average fusion rule to obtain the Habeeb Iraqi Journal of Science, 2021, Vol. 62, No. 9, pp: 3228-3236 3235 fused image which is suffering from the blurring effects. This reduces the quality of the fused image. To achieve high focus results with high quality, the proposed image fusion using focusing filter and DT-CWT filter is presented in this paper for improving the fusion results. Focusing filter algorithm consists of two filters. One filter is Wiener filter and the other is the sharpening filter. This filter is applied before the fusion process that is performed in DT-CWT domain. The evaluation of the performance of the proposed fusion is achieved on the different multimodal images using different focus metrics. The experiment results showed that the proposed multimodal image fusion gives good results in terms of the focus and quality compared with the traditional DWT and DT-CWT based image fusion techniques.