Skip to content
Research Article Open access

Novel Data Mining Techniques for Incomplete Clinical Data in Diabetes Management

Herbert F. Jelinek, Andrew Yatsko, Andrew Stranieri, Sitalakshmi Venkatraman

Current Journal of Applied Science and Technology · pp. 4591–4606 · Published 16 Sep 2014

10.9734/BJAST/2014/11744

Abstract

An important part of health care involves upkeep and interpretation of medical databases containing patient records for clinical decision making, diagnosis and follow-up treatment. Missing clinical entries make it difficult to apply data mining algorithms for clinical decision support. This study demonstrates that higher predictive accuracy is possible using conventional data mining algorithms if missing values are dealt with appropriately. We propose a novel algorithm using a convolution of sub-problems to stage a super problem, where classes are defined by Cartesian Product of class values of the underlying problems, and Incomplete Information Dismissal and Data Completion techniques are applied for reducing features and imputing missing values. Predictive accuracies using Decision Branch, Nearest Neighborhood and Naïve Bayesian classifiers were compared to predict diabetes, cardiovascular disease and hypertension. Data is derived from Diabetes Screening Complications Research Initiative (DiScRi) conducted at a regional Australian university involving more than 2400 patient records with more than one hundred clinical risk factors (attributes). The results show substantial improvements in the accuracy achieved with each classifier for an effective diagnosis of diabetes, cardiovascular disease and hypertension as compared to those achieved without substituting missing values. The gain in improvement is 7% for diabetes, 21% for cardiovascular disease and 24% for hypertension, and our integrated novel approach has resulted in more than 90% accuracy for the diagnosis of any of the three conditions. This work advances data mining research towards achieving an integrated and holistic management of diabetes.

Data mining missing value imputation diabetes management classifiers diagnosis accuracy.

References (52)

  1. 1 Statistical Analysis With Missing Data [DOI]
  2. 2 Induction of Decision Trees [DOI]
  3. 3 Induction of decision trees [DOI]
  4. 4 Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls [DOI]
  5. 5 Standards of Medical Care in Diabetes—2014 [DOI]
  6. 6 Standards of Medical Care in Diabetes—2013 [DOI]
  7. 7 Multiple imputation of discrete and continuous data by fully conditional specification [DOI]
  8. 8 A fuzzy K-nearest neighbor algorithm [DOI]
  9. 9 Statistical Analysis with Missing Data [DOI]
  10. 10 Poor Control of Risk Factors for Vascular Disease Among Adults With Previously Diagnosed Diabetes [DOI]
  11. 11 Multiple Imputation and its Application [DOI]
  12. 12 Analysis of Incomplete Climate Data: Estimation of Mean Values and Covariance Matrices and Imputation of Missing Values [DOI]
  13. 13 A Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms [DOI]
  14. 14 Feature Extraction Construction and Selection: A Data Mining Perspective [DOI]
  15. 15 Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier.
  16. 16 Methods for imputation of missing values in air quality data sets [DOI]
  17. 17 Missing covariate data in medical research: To impute is better than to ignore [DOI]
  18. 18 Type 1 diabetes: recent developments [DOI]
  19. 19 Impact of imputation of missing values on classification error for discrete data [DOI]
  20. 20 Comparative Effectiveness of Sulfonylurea and Metformin Monotherapy on Cardiovascular Events in Type 2 Diabetes Mellitus [DOI]
  21. 21 Comparison of three data mining models for predicting diabetes or prediabetes by risk factors [DOI]
  22. 22 Discretization for naive-Bayes learning: managing discretization bias and variance [DOI]
  23. 23 Incidence of type 1 and type 2 diabetes in adults and children in Kronoberg, Sweden [DOI]
  24. 24 Effect of pre-diabetes on future risk of stroke: meta-analysis [DOI]
  25. 25 Dietary acid load and risk of type 2 diabetes: the E3N-EPIC cohort study [DOI]
  26. 26 Issues in multiple imputation of missing data for large general practice clinical databases [DOI]
  27. 27 A decision tree-based missing value imputation technique for data pre-processing [DOI]
  28. 28 Missing Value Imputation Based on Data Clustering [DOI]
  29. 29 Empirical Likelihood‐based Inference in Linear Models with Missing Data [DOI]
  30. 30 Missing values in longitudinal dietary data: A multiple imputation approach based on a fully conditional specification [DOI]
  31. 31 Poor control of risk factors for vascular disease among adults with previously diagnosed diabetes [DOI]
  32. 32 Data-Mining Technologies for Diabetes: A Systematic Review [DOI]
  33. 33 Knowledge Discovery from Legal Databases [DOI]
  34. 34 Marked temporal increase in the incidence of type 1 and type 2 diabetes among young adults in Finland [DOI]
  35. 35 An innovative Multi-disciplinary Diabetes Complications Screening Program in a Rural Community: A Description and Preliminary Results of the Screening [DOI]
  36. 36 Examining the mechanisms of glucose regulation.
  37. 37 A pre-processing method to deal with missing values by integrating clustering and regression techniques [DOI]
  38. 38 Predicting cardiac autonomic neuropathy category for diabetic data with missing values [DOI]
  39. 39 Constructive Induction of Cartesian Product Attributes [DOI]
  40. 40 Advances in Computer Science and Engineering [DOI]
  41. 41 Data Decomposition and Decision Rule Joining for Classification of Data with Missing Values [DOI]
  42. 42 Measurement of insulin action: a tribute to Sir Harold Himsworth [DOI]
  43. 43 Empirical Investigation of Decision Tree Ensembles for Monitoring Cardiac Complications of Diabetes [DOI]
  44. 44 Generalization Performance Of Backpropagation Learning On A Syllabification Task
  45. 45 Feature selection using misclassification counts
  46. 46 Wrapper subset evaluation facilitates the automated detection of diabetes from heart rate variability measures
  47. 47 A Comparison of Machine Learning Algorithms for Multilabel Classification of CAN
  48. 48 AWSum - applying data mining in a health care scenario [DOI]
  49. 49 International Textbook of Diabetes Mellitus [DOI]
  50. 50 The Kaohsiung Journal of Medical Sciences [DOI]
  51. 51 A Review and Empirical Evaluation of Feature Weighting Methods for a Class of Lazy Learning Algorithms [DOI]
  52. 52 Data Decomposition and Decision Rule Joining for Classification of Data with Missing Values [DOI]

Cited by 16

Data analytics identify glycated haemoglobin co-markers for type 2 diabetes mellitus diagnosis

Herbert F. Jelinek, Andrew Stranieri, Andrew Yatsko · Computers in Biology and Medicine · 2016

Detecting depression severity using weighted random forest and oxidative stress biomarkers

Mariam Bader, Moustafa Abdelwanis, Maher Maalouf · Scientific Reports · 2024

Research and Citation Analysis of Data Mining Technology Based on Bayes Algorithm

Mingyang Liu, Ming Qu, Bin Zhao · Mobile Networks and Applications · 2016

Identifying the Relation between Fasting Blood Glucose and Glycosylated Haemoglobin Levels in Greek Diabetic Patients

M Stamouli, A Pouliakis, A Mourtzikou · Annals of Cytology and Pathology · 2016

Diagnostic with incomplete nominal/discrete data

Herbert F. Jelinek, Andrew Yatsko, Andrew Stranieri · Artificial Intelligence Research · 2015

Showing 6 of 16 known citations — external sources report more than can currently be individually listed.

Article metrics

Real usage data collected on this platform.

41

Page views

0

PDF downloads

4

Outbound clicks

16

Citations

Views over time

Views by country

Approximate, from request IP at view time — not citizenship or institution. Countries with fewer than 5 views are grouped as "Other".

Traffic sources

Referring site, by host.

Views and downloads exclude known bots/crawlers. Citations combines this platform's own DOI-resolved index with each external source's own reported total — see Cited by above for individually listed citing works. Last refreshed 0 seconds ago.