Data Quality Assistance - The Use of Data Mining Algorithms to Enhance Data Quality

Abstract

Large and over the years grown databases are a persistent concern in the field of data quality. Data sets grow over time from multiple sources and various users. Data Quality is one of the key issues that needs to be considered. This paper introduces a further development of an interactive data mining assistance system for ensuring data with high quality. What exactly is data quality? Data Quality in our approach is that the data that need to fulfill special requirements. Therefore, in a first instance, data mining algorithms are used to find outliers and duplicates. In the next step, the data mining assistance system generates rules that describe the whole data set. Furthermore, a rule administration is part of the concept. Interesting rules that have been found within the data set through the application of various data mining techniques are supposed to be added at this stage. The system serves, therefore to store and review rules that can be applied to the decision support system. For generating rules, various algorithms from the field of data mining are used. These rules have to be evaluated by experts to see if they can be applied as a type of suggestion rule to the decision support system.