Fully Convolutional Deep Neural Networks with Optimized Hyperparameters for Detection of Shockable and Non-Shockable Rhythms

2.1. Public Holter Databases

The full-length ECG recordings of three publicly available databases are considered:

All ECG signals were stored at sampling frequency fs = 250 Hz. No additional filtering was applied, although ECGs might be prefiltered during their hardware acquisition in Holters. Each ECG recording was split in nonoverlapping 10 s strips, which are annotated following the rhythm annotation scheme (see Section 2.3 ). Data augmentation by overlap of extracted strips in long-term Holter databases has not been applied because extra replication of identical rhythms from the same patient produces artificially big datasets without introducing statistically valuable information for extraction of new diagnostic features. Indeed the training process on such databases could be substantially slowed down with no effect on improving the machine learning accuracy.

2.2. OHCA Databases

The OHCA databases were collected during 959 interventions with a commercial AED (Fred Easy, Schiller Médical, Wissembourg, France) in the region of Paris and outlying areas in two nonoverlapping periods:

One-channel ECG signals were acquired via the defibrillation pads in the anterolateral position, filtered in a bandwidth (1–30 Hz) by the AED input hardware circuits for baseline drift and high-frequency noise suppression, and sampled at fs = 250 Hz. Ten-second ECG strips during AED analysis were extracted and annotated following the rhythm annotation scheme (see Section 2.3 ).

2.3. Rhythm Annotation

The rhythm of each 10 s ECG strip was observed and independently annotated by three cardiologists. Majority voting was applied in cases of annotation disagreement. The annotations follow the AHA rhythm classification scheme [ 4 ], where performance goals are defined only in the absence of artifacts. Five basic Sh and NSh rhythm categories are defined (illustrated in Figure 1 ):

2.4. Training/Validation Subsets

The annotated ECG strips were partitioned in two independent datasets, composed from different databases, as follows:

The sample size of the training and validation datasets is presented in Table 1 .

3.1. DNN Architecture

The architecture of the end-to-end fully CNN investigated in this study is presented in Figure 2 . The input feature space is a 1D data vector read from one-lead raw ECG signal with size (1 × L ₁ ). It is next processed by a sequence of N convolutional blocks, each one including the three common layers of the CNNs architecture: 1D convolution (Conv1D), max-pooling and dropout [ 38 ]. The Conv1D layer of block number ( i) consists of Fi filters with specific 1D convolution kernels of size (1 × Ki ), thus providing Fi feature map representations of the ECG signal that maintain its temporal order [ 60 ]. The output of the fth filter ( f = 1, 2,… Fi ) is computed as:

i = [1, 2, … N ] identifies the sequential number of the convolutional layer.

S _i is the input vector of the i ^th convolutional layer, with size (1 × Li ).

j = [0, 1, … Li – Ki + 1] indexes the output feature vector, applying convolutional operation with a valid padding [ 37 ].

$w_{ki}$ are the weights and $b_{ki}$ are the biases of the convolution kernel;

A is the applied nonlinear activation function ReLU (rectified linear unit);

Next, the Conv1D layer output ( $con{v}_i^f$ ) with size (1 × ( Li − Ki + 1) × Fi ) is downsampled by a max-pooling layer (pool size = (1 × MP )). It applies maximum operation over nonoverlapping segments of the feature vector $con{v}_i^f$ , thus generating a new feature vector $poo{l}_i^f$ with MP times smaller width (1 × $\frac{\mathrm{Li}-\mathrm{Ki}+1}{\mathrm{MP}}$ $\mathrm{x}\mathrm{Fi}$ ). To avoid overfitting and improve the generalization, a dropout regularization layer is applied during training with a dropout rate α ∈ [0; 1], thus generating an output vector $dro{p}_i^f$ (1 × $\frac{\mathrm{Li}-\mathrm{Ki}+1}{\mathrm{MP}}$ $\times \mathrm{Fi}$ ) with portion of ‘0′ nodes equal to α. In the test process $dro{p}_i^f=poo{l}_i^f$ . The input signal for the next convolutional layer is S _i+1 = $dro{p}_i^f$ .

The sequence of N convolutional blocks with the structure described above is followed by a global max-pooling (GMP) layer, which downsamples $dro{p}_N^f$ of each filter ( f = 1, 2, … F _N ) to a single value equal to its maximal value. Thus the GMP layer has an output feature size (1 × 1 × F _N ), which is fed into a binary classifier implemented as a dense layer with a sigmoid activation function. The output layer provides the diagnostic probability for Sh/NSh rhythm detection p ∈ [0: Sh,1: NSh].

The trainable parameters in the proposed DNN architecture correspond to the weights and biases of the N -blocks Conv1D layers and the final dense layer, therefore, this model can be considered as a fully CNN. The number of trainable parameters can be calculated with the following equation:

3.2. Hyperparameters Optimization

Although the structure of the designed DNN architecture implements a sequence of standard CNN layers ( Figure 2 ), their basic hyperparameter (HP) settings are a priori unknown. These include:

We hypothesize that $HPs={\left\{N,F_i,K_i\right\}}_{i=1}^N,$ determining the number of trainable parameters (Equation (2)) might have significant influence on the Sh/NSh rhythm detection performance. Therefore, we performed a process of search, analysis and optimization of the top-ranked HP settings with the final aim to justify the choice of the best CNN model. The whole process is schematically summarized in Figure 3 and will be further described in detail.

3.3. Training of DNN Models

All DNN models were programmed in the TensorFlow framework using Keras built-in APIs for model design, training, and evaluation. After the training was completed, the parameters of all networks were stored in HDF5 files. All experiments were conducted on a workstation with Intel(R)Xeon(R) CPU E5-2630 0 @ 2.30 GHz (2 processors), 32G B RAM, NVIDIA Quadro K4000-3Gb GPU.

The following concepts were applied for management of the input data for model fitting:

The following settings were applied for the model design and fitting:

4.1. Random HPs Search

The random search training of CNN models was automatically performed within the defined HP grid of values $HPs={\left\{N,F_i,K_i\right\}}_{i=1}^N$ ( Section 3.2.1 ), using a fixed setting of the network depth ${\left\{N=const\right\}}_{N=1}^7$ in one training session. A relatively similar duration of the training sessions has been respected, however, owing to some specifics in model training (mentioned below), the number of trained CNN models with distinct depths was globally different:

We illustrate the performance of all 4216 models generated by random HPs search in Figure 4 —first shown as individual hits in BAC scatterplot ( Figure 4 a) and second as a statistical interpretation of those BAC hits in respect to the network depth N ( Figure 4 b). The dense overlapping scatterplots are seen as very similar BAC quartiles (fixed roughly in a 1% span within 98%–99%) and BAC min-max ranges (fixed roughly in a 5.5% span from about 95% to 99.5%) for all models with N ≥ 3. The shallowest models have a downshift performance, estimated within a BAC quartile range (97.6%–98.4%, N = 2) and (92.7%–95.7%, N = 1). Following the strategy for statistically justified HPs optimization, our focus was only on the top ranked performance models, which are zoomed in Figure 4 a and pointed out (arrows) in the rightmost part of the BAC histograms in Figure 4 c. According to the observations, we define four BAC thresholds for selection of the top-ranked models:

4.2. HP Statistical Analysis

A simple example, which can visually justify the principle of our statistics-based HP optimization is illustrated in Figure 5 a. It represents a 2D colormap $BAC=f\left\{K_1,F_1,\right\}$ for all models with one CNN layer ( N = 1), which are trained for the full search grid in the 2D space $HPs\left\{K_1,F_1\right\}\in \left[5;200\right]$ . The colormap gradient clearly identifies that BAC is nonlinearly dependent on $HPs\left\{K_1,F_1\right\},$ with a maximal performance red area in a relatively narrow mid part of $K_1=\left[30;85\right]$ and a wider range of high $F_{1}40$ . However, this optimal area has no clear peak or contours because it is representative for the performance variance that each model can have in different training attempts. As soon as all models in the random search are trained only once, the performance of all models in the grid is randomly distributed within this variance. Therefore, all $HPs\left\{K_1,F_1\right\}$ were adjusted to BAC, as shown in Figure 5 b, where the common values in 50% of observations (quartile range and median value) were used to derive conclusions about the HPs distributions, which are associated with specific BAC range. These statistical distributions show clear nonlinear trends for $K_1=f\left(BAC\right)$ and $F_1=f\left(BAC\right)$ , which have important implications on the analysis of the feature importance score, as further communicated. Our current focus is on the 50% of the top ranked performance models, which manifest very similar $HPrank$ values, allocated in narrow quartiles—the red highlighted range in Figure 5 b, translated to the white rectangle in Figure 5 a, i.e., $HPrank{\{K}_1=\left[50;65\right],F_1=\left[100;125\right]\}.$ Our optimization criterion based on $median\left(HPrank\right)$ (Equation (6)) predicts a new statistically justified model: $HPopt\{K_1=50,F_1=113\}$ , depicted with the white square in Figure 5 a. We observe that its coordinates fall in an interpolated red zone of high performance (between four points of our search grid with BAC = {96.50, 96.52, 96.65, 96.74}) and it is a subject of our further optimization results to prove that the real $HPopt$ performance corresponds equally well to the predicted in this graph performance.

The above statistically justified principles were applied for the performance analysis in the multivariate search grid $HPs={\left\{K_i,F_i,\right\}}_{i=1}^N$ of deeper CNNs (N = 2, 3… 7) so that HP median values and quartile ranges were drawn in function of BAC ( Figure 6 ). Our focus on the top-ranked models identified the quartile ranges of $HPrank$ (highlighted red zone), and most importantly the median points, which were used to derive our optimal CNN configurations: $HPopt=median\left(HPrank\right).$ We further considered the number of trainable parameters since they might have an important influence on the model training process and accuracy. They were subjected to the above statistical approach, adjusting their distributions in function of BAC ( Figure 7 ). The graphs were used only for observational purposes and were not part of the model optimization settings $HPopt=median\left(HPrank\right)$ as soon as Params were a derivation of the other HPs in the random search (Equation (2)): $Params=f{\left\{N,F_i,K_i\right\}}_{i=1}^N$ . The observations show that the top-ranked models (highlighted red zone in Figure 7 ) have limited trainable parameters, with quartile ranges of about 5000 to 45,000 parameters.

4.3. Rank of HPs Importance

Another result is the computed importance score of $HPs={\left\{Params,F_i,K_i\right\}}_{i=1}^N$ as individual predictors of CNN performance, presented in Table 2 . This importance score can be used for identifying the significant nonlinear trends of $HPs=f\left(BAC\right)$ , depicted in Figure 5 b and Figure 6 , Figure 7 . The red highlighted cells in Table 2 (IS > 0.5) clearly indicate that the kernel sizes ( N = 1, … 7) and the number of filters ( N = 7) in the first convolutional layers, and consequently the number of CNN parameters ( N = 1, …, 7) have significant importance to the BAC.

4.4. Optimal HP Models

Table 3 presents the predicted optimal model settings $HPopt=median(HPrank{\left\{N,F_i,K_i\right\}}_{i=1,}^N$ , N = 1, … 7, which were derived from the highlighted top-rank ranges in Figure 5 b and Figure 6 . The performance of these optimal models is presented in Figure 8 for different learning rates. The min–max and quartile range distributions in Figure 8 show the natural BAC variance, which was observed between different training runs of a model with the same $HPopt$ settings. We note that BAC variance is dependent on the LR and is minimal for LR = 0.0001–0.001 (<0.5%). Besides, we found that LR = 0.0001–0.001 is the optimal setting for training of the top-ranked $HPopt$ models (BAC > 99% for N ≥ 3). The best performances of all $HPopt$ models are reported in Table 3 . Among them, our best model (BAC = 99.5) was found to be:

5. Discussion

6. Conclusions

This study presents the optimal HPs of deep CNNs with two to seven convolutional layers, derived by statistical justification of the common HPs of the top-ranked models. We observe a certain limit in the depth and width of CNNs design so that inferior performance is common for all shallow models (less than three convolutional layers), as well as deep (three to six layers) but very wide or very narrow models (highlighted importance for the kernel size of the first two layers). The deepest models ( N = 7) might lose performance due to maximal model shrink, where the kernel size is fixed to minimum and the number of filters in deep convolutional layers become strong predictors of the CNN performance. The presented optimal HPs have a generally high importance to the ability for good model training and feature extraction, and might be efficient in other ECG diagnostic applications.

The general conclusion after evaluation in Holter and OHCA databases would be that our best CNN model provides very high Se (>95.2%) and Sp (>98.6%), even for very short analysis durations (2 s), and could be considered compliant with the AHA performance goals [ 4 ]. While the optimized network, based on fully convolutional hidden layers with pretrained weights can be run alone in real time without the need of preprocessing, waveform measurements, transformations, or other machine learning algorithms, it has perspectives for certain applications in unsupervised database annotation and diagnosis platforms, as well as in reliable real-time AED shock-advisory systems in OHCA.

7. Limitations

The size of the training/validation datasets is not divided according to the common scheme of 80%/20% as it isn’t a manually controllable factor but depends only on the number of cases found in independent data sources, predefined before the study. Thus, the cases are distributed uncommonly with majority of data for validation. The scheme with independent data sources is, however, respected to avoid the common cross-validation scenario without a patientwise control, where rhythms from the same patient are randomly shuffled in the training and validation datasets, and thus an overtraining on the specific patient rhythm is highly probable in deep learners.

From the methodological point of view, the HP optimization is done on the validation set, which is also used for evaluation. However, as soon as the study sets out to investigate the HP optimization itself, it is not an issue for the main conclusions, regarding the importance of HPs and comparison to other published DNN models, all trained and evaluated under the same conditions and datasets.

Although the presented study supports the derivation of valuable conclusions about the optimal depth and width of CNN models for Sh/NSh rhythm discrimination, we should note the inevitable influence of the size and content of the training and validation databases. Considering, however, the application of both public Holter and OHCA electrocardiograms, which cover the wide variety of Sh and NSh rhythms appearing during cardiac arrest and other critical situations and present as much as 3890 cases for training and 6660 cases for validation, we do not expect substantial differences in the context of Sh/NSh rhythm detection.

Author Contributions

Conceptualization, V.K., I.J., S.M., and J.-P.D.; methodology, V.K. and I.J.; software, V.K. and I.J.; validation, V.K., I.J., S.M., and J.-P.D.; formal analysis, V.K. and I.J.; investigation, V.K. and I.J.; resources, S.M, and J.-P.D.; data curation, S.M, and J.-P.D.; writing—original draft preparation, V.K. and I.J.; writing—review and editing, V.K., I.J., S.M., and J.-P.D.; visualization, V.K.; supervision, S.M., J.-P.D.; project administration, V.K. and I.J.; funding acquisition, V.K. and I.J. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.