Skip to content
Research Article Open access CC BY 4.0

iCleaner: A Data Cleansing Tool for Outlier Detection in a Data Warehousing Environment

Kofi Sarpong Adu-Manu, John Kingsley Arthur, Joseph Kobina Panford, Joseph George Davis

Journal of Advances in Mathematics and Computer Science · pp. 1–17 · Published 4 Nov 2016

10.9734/BJMCS/2016/28861

Abstract

The implementation of Data Cleansing (DC) in Data Warehousing (DW) is essential in recent years. Organizations around the world generate huge amount of data from their day-to-day activities for their operations. These organizations will not survive if the data they generate remains dirty or erroneous. There are errors or outliers that make the data become dirty such as data entry errors, outdated data in the database, data migrated from old databases, and changes made at the source repository. The changing needs by customers to update their records (for example customer attributes such as marital status, phone number or address changes with time) cause records to become obsolete and reducing the quality of data. In order to obtain high quality data over time, the data requires cleansing. The proposed Integrated Cleaning (iCleaner) tool is developed to facilitate the data cleaning process and to address the problems associated with duplicated records. In addition, the proposed cleaning tool is able to detect and update missing data by merging key columns within the records. The system is flexible to use and comes with a convenient user-friendly interface designed for the data cleansing process. We provide an efficient, but simple algorithms designed to perform these functionalities and provide the running time for the system performance.

Data cleansing data quality database system management data warehousing automatic periodic scheduler duplicates missing data data integration data quality tools.

References (41)

  1. 1 Data Cleaning: Problems and Current Approaches.
  2. 2 Springer Science and Business Media
  3. 3 Data quality in context [DOI]
  4. 4 Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem [DOI]
  5. 5 A framework for analysis of data quality research [DOI]
  6. 6 Data Mining and Knowledge Discovery [DOI]
  7. 7 Potter's Wheel: An Interactive Data Cleaning System
  8. 8 Frameworks for entity matching: A comparison [DOI]
  9. 9 United Nations Economic Commission for Europe (UNECE) [DOI]
  10. 10 Quality-Driven Query Answering for Integrated Information Systems [DOI]
  11. 11 Data integration under integrity constraints [DOI]
  12. 12 Quantitative Data Cleaning for Large Databases
  13. 13 Decision Support and Data Warehouse Systems
  14. 14 BigDansing [DOI]
  15. 15 Exploiting missing clinical data in Bayesian network modeling for predicting medical problems [DOI]
  16. 16 Data quality: A statistical perspective [DOI]
  17. 17 Arktos: towards the modeling, design, control and execution of ETL processes [DOI]
  18. 18 Towards reliable interactive data cleaning [DOI]
  19. 19 Seminal Contributions to Information Systems Engineering [DOI]
  20. 20 SampleClean: Fast and Reliable Analytics on Dirty Data.
  21. 21 Data Integration under Integrity Constraints [DOI]
  22. 22 Data Quality Management with Semantic Technologies [DOI]
  23. 23 IntelliClean [DOI]
  24. 24 Biological data cleaning: a case study [DOI]
  25. 25 An Enhanced Technique to Clean Data in the Data Warehouse [DOI]
  26. 26 Trends and Applications in Knowledge Discovery and Data Mining [DOI]
  27. 27 Analysis of Data Cleansing Approaches regarding Dirty Data A Comparative Study [DOI]
  28. 28 Open Source Data Quality Tools: Revisited [DOI]
  29. 29 Dealing with dirty data
  30. 30 A Framework for Data Quality in Data Warehousing [DOI]
  31. 31 OPEN USER INVOLVEMENT IN DATA CLEANING FOR DATA WAREHOUSE QUALITY
  32. 32 A Review of Data Cleansing Concepts Achievable Goals and Limitations [DOI]
  33. 33 The Challenges of Data Cleansing with Data Warehouses [DOI]
  34. 34 Three Tier level Data Warehouse Architecture For Ghanaian Petroleum Industry [DOI]
  35. 35 DataBryte: A Proposed Data Warehouse Cleansing Framework.
  36. 36 A Conceptual Framework for Data Cleansing A Novel Approach to Support the Cleansing Process [DOI]
  37. 37 Leveraging the Common Cause of Errors for Constraint-Based Data Cleansing [DOI]
  38. 38 INTERNATIONAL JOURNAL OF COMPUTER APPLICATIONS [DOI]
  39. 39 IEEE Transactions on Knowledge and Data Engineering [DOI]
  40. 40 2011 Developments in E-systems Engineering [DOI]
  41. 41 ActiveClean: Interactive Data Cleaning While Learning Convex Loss Models [DOI]

Cited by 0

No indexed citations yet.

Related research

Article metrics

Real usage data collected on this platform.

66

Page views

0

PDF downloads

17

Outbound clicks

0

Citations

Views over time

Views by country

Approximate, from request IP at view time — not citizenship or institution. Countries with fewer than 5 views are grouped as "Other".

Traffic sources

Referring site, by host.

Views and downloads exclude known bots/crawlers. Citations combines this platform's own DOI-resolved index with each external source's own reported total — see Cited by above for individually listed citing works. Last refreshed 0 seconds ago.