Automatic Measurement of the Mitral Valve Based on Echocardiography Using Digital Image Processing

Background: The evaluation of mitral valve area through multiplanar reconstruction in 3-dimensional echocardiography is restricted to specific software and to the experience of echocardiographers. They need to manually select the video frame that contains the maximum mitral valve opening area, as this dimension is fundamental to identification of mitral stenosis. Objective: To automate the process of determining the maximum mitral valve opening area, through the application of digital image processing (DIP) in echocardiography tests, developing an open algorithm with video reading in avi format. Method: This cross-sectional observational pilot study was conducted with 25 different echocardiography exams, 15 with normal aperture and 10 with rheumatic mitral stenosis. With the authorization of the Research Ethics Committee, all exams were performed and made available by 2 specialists who used 2 models of echocardiographic devices: Vivid E95 (GE Healthcare) and Epiq 7 (Philips), with multiplanar transesophageal probes. All videos in avi format were submitted to DIP using the image segmentation technique. Results: The measurements obtained manually by experienced echocardiographers and the values calculated by the developed system were compared using a Bland-Altman diagram. There was greater agreement between values in the range from 0.4 to 2.7 cm². Conclusion: It was possible to automatically determine the maximum mitral valve opening area, for cases from both GE and Philips, using only 1 video as input data. The algorithm has been demonstrated to save time on measurements when compared to the usual method.


Introduction
Valve diseases are important determinants of cardiovascular morbidity and mortality, with degenerative diseases of the mitral and aortic valves being the most prevalent, even though rheumatic disease still constitutes an important public health problem in Brazil. Echocardiography plays an important role in the diagnosis and severity assessment of valve diseases. It is able to determine anatomical and pathophysiological details that assist in planning the best intervention for each patient, and it has the advantages of being widely available and low cost, without requiring radiation or nephrotoxic contrast.¹ Among the most common imaging tests, echocardiography is the most frequently used modality in the evaluation of mitral valve diseases. This is possible because it is a more readily available and widely used technique to assess function and structure. Echocardiography makes real-time imaging possible without the presence of ionizing radiation, serving today as a backbone in the field of cardiovascular imaging.² The use of 3-dimensional echocardiography provides numerous advantages, such as the possibility of realtime navigation, image post-processing, and multiplanar reconstruction. It is especially important for the preoperative evaluation of patients who are candidates for valve repair or replacement surgery. It allows intraoperative monitoring of procedures, such as transcatheter aortic valve replacement³ and implantation of clips for percutaneous mitral repair (Mitraclip). It also makes it possible to conduct procedures to treat rheumatic mitral stenosis, such as percutaneous mitral Barros Filho et al. Automatic measurement of the mitral valve based on echocardiography using DIP balloon valvuloplasty, or even percutaneous implantation of a biological mitral valve prosthesis by transcatheter mitral valve replacement, in patients with degenerative mitral disease and calcification of the valve annulus, or percutaneous prosthesis implantation inside a dysfunctional biological prosthesis (mitral valve-in-valve procedure). The procedures described have shown great evolution, with increasingly effective results due to the help of multimodal cardiac imaging in pre-procedure evaluation and during the intervention. 4,5 During interventions, decisions must be be agile, and accurate information must be provided to the interventionist; therefore, the quality of communication between teams is essential. Through visual demonstration of structures, made possible by 3-dimensional reconstruction and rendering, with more accurate measurements that are less dependent on angulations, such as those obtained by multiplanar reconstruction, it is possible to obtain better results, with greater efficiency and lower rate of complications. 5 There is significant variation in the human interpretation of echocardiographic data, in addition to errors in measurements, due to considerable inter-observer variability. The use of automated systems for the interpretation of cardiovascular images can be useful in improving the methods' diagnostic and prognostic performance, and the application of machine learning to medical images, with the help of specialists' interpretation, increases the reliability of the process due to the availability of various data structured by reports. 6 The accurate evaluation of the mitral valve area by means of multiplanar reconstruction on 3-dimensional transesophageal echocardiography is totally dependent on the examiner's experience and expertise in using specific software, which may not be available on some platforms or may not even be very practical to use, thus consuming exam time. 6 Digital image processing (DIP) has been gaining recognition in 2 main areas of application, namely, improving visual information from imaging for human interpretation and processing imaging data for storage, transmission, and representation, considering automatic machine perception. 7 It consists of manipulation of images by a computer, with different applications, such as contrast enhancement, noise reduction, tracking objects and/or people, edge detection, pattern identification, classification and counting of objects and/or people, and many others. 7 Among multiple existing DIP techniques, image segmentation has been applied. This technique produces a binary image, with pixels of value 1, indicating they are part of the object under study, and pixels of value 0, indicating they are not. Thus, it is possible to divide an image into regions, identifying borders and discontinuities, for example. 8 Several studies have contributed to the evolution of echocardiography analysis using computerized tools. For example, Sakamiya et al. created a platform for online monitoring of the contractile behavior of the heart, which are the main functional characteristics of cardiac tissue, simultaneously employing an image processing and piezoelectric detection system. They also evaluated the influence of medications such as isoproterenol and doxorubicin on the contractile behavior of the heart. The medication reactivity results provided by these 2 measurement systems were consistent with the previous reports they had, demonstrating the reliability of the developed platform and its potential for use in medication-related screening applications. 9 Nizar et al. used a convolutional neural network for realtime detection of the aortic valve in echocardiography exams, with the purpose of assisting medical examination, since an automated detection system in an echocardiogram can improve the accuracy of the medical diagnosis and provide additional medical analysis from the resulting detection. Therefore, they concluded that this tool can be of great help for medical purposes. 10 Ostvik et al. verified the possibility of applying artificial neural networks in the classification of transthoracic images obtained by echocardiography. The conclusion was satisfactory for the implementation in 2-dimensional echocardiography, and, for analysis of 3-dimensional echocardiography, studies still need to make gains in performance, seeing that it is conducted based on frames obtained during the procedure. 11 Therefore, the objective of this project is to automate the determination of maximum mitral valve opening area, by applying DIP to echocardiography exams and developing an open algorithm with video reading in avi format.

Methods
Retrospective analyses were conducted in different echocardiography studies by experienced echocardiographers. The exams included patients who had "normal" mitral valves, that is, mitral valve openings in normal conditions, and patients with mitral valves affected by rheumatic stenosis.
For acquisition, the echocardiographers used 2 echocardiography device models, with multiplanar transesophageal probes: the Vivid E95 (GE Healthcare) and the Epiq 7 (Philips). Each has its particularities. The most considerable difference, in relation to this research, is the way in which the image scale is represented. The Philips echocardiograph draws green dots, and the distance between their centroids is indicated as a numerical value belonging to the metric scale. The General Electric (GE) brand echocardiograph represents the scale at the bottom right of the image.
After acquiring the exams saved as videos, they were submitted to DIP using the image segmentation technique, considering that a video is composed of a set of sequential static images. As a result, DIP identifies the maximum valve opening area, highlights its outline, and calculates its value.
An application was developed to facilitate the use of DIP. People unfamiliar with programming are able to use it, as it has an intuitive graphical interface. It was created with python language.
Finally, the results obtained by the application were compared with the results obtained by the echocardiographers, who manually selected the maximum valve opening area using specific software. To verify the correlation between both measurements, the Bland-Altman method was used.

Results
All exams obtained were saved in avi video format. A total of 25 videos were acquired, 22 from the GE echocardiography device and 3 from the Philips device. The number of tests used in the present study was limited by the study time and influenced by the pandemic, which made it difficult to acquire more tests.
T h e s t u d y m o d e l w a s r e t r o s p e c t i v e , c r o s ssectional, and observational, acquired in accordance with the recommendations of the American Society of Echocardiography. 12 Planimetry was performed by multiplanar reconstruction with ventricular and atrial view, using EchoPac v.204 for GE devices and directly on the Epiq 7 device during the echocardiography for the Philips devices. When the application is opened, the home screen is displayed (Figure 1a). It displays 2 options: "Philips" and "General Electric". The user must select the button according to the source of the video to be used for analysis. If the video was saved from echocardiograms performed using Philips devices, the "Philips" button must be selected. If the exams were performed using General Electric devices, the "General Electric" button must be selected.
When selecting the "Philips" option, the screen in Figure 1b will be displayed. The user must click on the "Select Video" button to load the video to be analyzed and, subsequently, inform the numerical scale present in the video. For example, if the exam was performed and saved with a 5 mm scale in the grid, the user must enter a value of 0.5 in the "Scale [cm]" field. Finally, the "Play" button must be selected for the application to perform the entire DIP to identify and calculate the maximum valve opening area. Additionally, the screen displays some instructions to facilitate the use of the application and to optimize the results to be obtained.
If the "General Electric" option is selected on the initial screen (Figure 1a), the screen in Figure 1c will be displayed. This screen allows the user to choose how the video was saved. If it was saved with only the 3-dimensional view, the "Case 1" button must be selected. If it was saved with the 3-dimensional view and the frontal, lateral, and superior views, the "Case 2" button must be selected. This distinction is necessary, because the mode and characteristics of saved videos influence digital processing. The "Case 1" and "Case 2" buttons will display the screens in Figures 1d and 1e, respectively. They function in the same manner as the screen in Figure 1b, previously described.
After selecting the "Play" button in Figure 1b, Figure 1d, or Figure 1e, DIP transforms the video into a set of images (as in the example illustrated in Figure 2), also called frames, to analyze them separately and to identify the image that contains the maximum valve opening area. As all images are isolated, the frame rate (frames per second) of each video did not interfere with the results.
After digital processing, the application displays the result, as illustrated in Figure 3. Figure 3a illustrates the first screen displayed after the application executes the DIP. The maximum valve opening area is identified and drawn (white outline). The value of the maximum area is also displayed. In the case shown in Figure  3a, the area is 4.45 cm ².
In order to make the results more accurate, the application allows the user to check the preceding and subsequent images, with the buttons "Previous image" and "Next image", respectively. They make it possible to verify the outlines that best correspond to reality.
In the result shown in Figure 3, although the outline in Figure 3a shows a satisfactory result, the outline in Figure 3b is more accurate and more consistent with the shape of the maximum valve opening. In fact, the maximum valve opening area became 4.5 cm².
The "Home screen" button returns to the initial screen of the application (Figure 1a) and the "Select image" button saves the results obtained.  Analysis of Figures 4 and 5 reveals that the results were very close. For the "normal" case, the application provided the value of 4.59 cm² (Figure 4b), and the echocardiographer's measurement provided the value of 4.7 cm² (Figure 4c). In the case with stenosis, the application provided the value of 0.51 cm² (Figure 5b), and the echocardiographer's measurement provided the value of 0.6 cm² (Figure 5c). Furthermore, it was observed that the adoption of some practices optimizes the results of the application, for example, saving videos highlighting the mitral valve region, applying zoom, and saving videos with few elements, preferably only the region of interest and the scale used. However, further tests would be needed to validate this finding.
With the identification of the maximum valve opening and the calculation of its value, the developed application is also able to detect whether or not the patient in the referred exam has stenosis. Furthermore, the processing time of the application is relatively low, remaining under 3 minutes for the majority of analyses.
In order to verify the concordance of the results obtained by the application and by the echocardiographers' measurements, the Bland-Altman method was applied, as shown in Figure 6.
Using the Bland-Altman method, evaluating the cases with and without stenosis separately, agreement was found between the DIP and echocardiographers' measurements in obtaining the maximum valve opening area (Figure 6a   and Figure 6b). Combining all cases, only 2 were outside the acceptable range (Figure 6c). These results contribute to the validation of both the DIP technique employed and the developed application.
For cases without stenosis (Figure 6a), the greatest difference between measurements was 0.7 cm², while most remained within the range of −0.5 cm² to 0.5 cm².
For cases with stenosis (Figure 6b), the greatest difference between measurements was 0.2 cm², and all remained within the range of −0.2 cm² to 0.2 cm². Analyzing all cases (Figure 6c), those with stenosis (points further to the left in Figure 6c) had smaller differences in measurements compared to those without stenoses (points further to the right in Figure 6c).

Discussion
This study stands out because it employs DIP, based on the image segmentation technique, to detect the maximum mitral valve opening area automatically, based on echocardiogram videos in avi format. Therefore, it contributes to the detection of mitral stenosis. A major differential is the application developed with python language, which allows easy access and easy handling of the DIP tools. It needs only be installed on a computer. The application's screens were developed to improve users' experience, and they do not require prior knowledge of programming.
The developed application is capable of analyzing more complex cases, such as patients with arrhythmias, because the automatic selection makes it possible to check each image in the video, highlighting the image with the maximum valve opening. Moreover, the more measurements that are performed with the developed application and compared with medical measurements, especially with different and atypical cases, the more it will be tested, allowing better assessment for its consolidation.
This automatic measurement has some advantages when compared with manual selection performed by echocardiographers, because the result can be heavily influenced by the moment when they pause the video to perform the analysis. This manual selection requires a great deal of the specialist's time, considering that they must be very careful to carry out a selection with as little error as possible and with maximum attention. Considering a full day of care, the accuracy of the selected regions will be greater for the first patients of the day, and they will decrease according to the echocardiographer's physical and mental fatigue, in addition to other external factors that may influence the measurement.
When conducting a survey of the literature, nothing similar was found. However, one of the great efforts of researchers is to ensure the automation of measurements in operator-dependent examinations for the investigation of various pathologies, with the aim of reducing or eliminating manual selection errors. Thus, Saine et al. presented a technique for automatic boundary detection of the left atrium and ventricle based on 2-dimensional echocardiograms, with the objective of determining the dilation area of these chambers in patients with mitral valve insufficiency. 13 Melo et al. developed a semiautomated method that determines variation in ventricular area on dynamic 2-dimensional echocardiograms using DIP techniques such as time averaging, wavelet-based denoising, edge enhancement filtering, morphological operations, homotopy modification, and watershed segmentation. This method has become useful for analyzing global ventricular function by measurements of area. 14 Mahadi et al. verified the use of DIP in 2-dimensional echocardiograms to transform scan slices into a set of pixels and thus determine the distance of points that could measure the diameter of a mitral valve in a given view. 15 Aquila et al. presented the use of a tool called Siemens eSie Valves which made it possible to automatically determine a series of parameters of the mitral valve annulus such as area, anteroposterior and posterolateral diameters, and intertrigonal distance. Therefore, this study compares existing anatomical alterations between functional and organic mitral regurgitation processes based on the automatic acquisition of valve measurements. 16 Accordingly, this study offers another tool to combat possible human errors in measuring valve areas and to contribute to automation.

Conclusions
It was possible to automatically determine the maximum mitral valve opening area, for cases from both GE and Philips echocardiography devices, using only 1 video as input data.
As the application chooses the appropriate frame and calculates the area almost instantly, it can be intuitively concluded that it is faster than manual measurement.
This study shows interesting potential because it does not depend on specific software, and it has a good outlook for obtaining measurements remotely.

Potential Conflict of Interest
No potential conflict of interest relevant to this article was reported.

Sources of Funding
This study was partially funded by Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).

Study Association
This article is part of the thesis of Doctoral submitted by Genilton de França Barros Filho, from Universidade Federal da Paraíba, Programa de Pós-Graduação em Engenharia Mecânica.