gms | German Medical Science

Introduction: Developing machine learning-based AI methods usually requires access to large amounts of data. However, personal health data is particularly sensitive and not easily accessible to data scientists or software developers. To improve this situation, classical anonymization methods based on aggregation and generalization, as well as modern data synthetization methods can be used.

Methods: We chose results from our previous HiGHmed Use Case Cardiology study by Sommer et al. [1] to study the usefulness of protected data generated through traditional anonymization and data synthetization. To this, we compared the Barcelona BioHF version 1 and MAGGIC heart failure risk scores for 1-year and 3-year mortality calculated on the original dataset with those derived from anonymized and synthesized data.

In the anonymization process, we first removed all variables that are not needed for the use case studied i.e., the calculation of MAGGIC or BioHF heart failure scores. We then applied different transformation methods to the 17 remaining variables. This was performed as an automated process using the optimization algorithms provided by ARX [2], [3]. In terms of protection levels, the dataset was transformed to fulfill the k-anonymity property with k=2, which means that the resulting dataset did not contain any unique records regarding all variables.

For the data synthesis process, we used the open-source software suite SDV [4], [5], which provides four distinct tabular data synthesizers (GaussianCopula, TVAE, CTGAN, and CopulaGAN). We trained each of the synthesizers with the original data, and subsequently generated sufficient datasets for HF score calculations. The best performing model (out of 4) was selected post hoc by comparing the synthesized and original input variable distributions using appropriate statistical tests.

Results: In the original data set, we computed 890 MAGGIC scores from 2441 partially incomplete patient records [1]. The number of computable MAGGIC scores was lower for the anonymized dataset (n=750) and synthesized data set (n=416). In the latter, we synthesized 2441 records for comparison. The median 1yr mortality risk was 0.093 for anonymization, 0.1065 for synthetization and 0.111 in the original data set. We also compared the score distributions using a two-sided Kolmogorov-Smirnov test (p=0.02362 for anonymized data and p=0.1078 for synthesized data). Similar results were obtained for the BioHF scores.

Figure 1 [Fig. 1]

Discussion and conclusion: While both methods yield results in good agreement with those of the original (non-de-identified) data they both have their distinct advantages and draw-backs. Synthetic data models may be trained automatically and could produce any necessary amount of data. However, they may suffer from the inability to capture complex statistical properties on smaller data sets. Traditional anonymization processes, on the other hand, reduce the amount of available data as compared to the original dataset, and may better preserve the data structure.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.