DataCleaner

DataCleaner is a strong data profiling engine for discovering and analyzing the quality of user’s data. DataCleaner offers features such as support adding HDFS datastores, support for simple fixed-width mainframe/EBCDIC files, extended the monitor REST job trigger, news channel into DataCleaner.

Monitor: Improve scheduling page load times, Monitor: Old repository backed up when new repository is loaded, Monitor: Reference data can now be configured in UI, IBM's ICU library used for better Unicode standard compatibility, Hashing component added, Fixed Pentaho plugin, Extended dictionary matcher (added possibility to ignore diacritic signs), Added scroll bar for value distribution (result) page and made connection lines lighter in desktop graphical job representation.

DataCleaner is built to handle data both big and small from CSV files, Excel spreadsheets to Relational Databases (RDBMs) and NoSQL databases. DataCleaner allows users to build their own cleansing rules and compose them into several use scenarios or target databases whether it is simple search/replace rules, regular expressions, and pattern matching or completely custom transformations.

DataCleaner’s Monitoring establishes the starting point and goals, and to ensure a process of following up on data quality issues. The monitoring server of DataCleaner enables users to not just make point-in-time profiles of users’ data, but to schedule periodic data quality checks and receive notifications if quality KPIs get out of control.

DataCleaner’s Data Quality Eco-System delivers not only out-of-the-box functionality, but also hosts an eco-system of community driven application extensions, integrations and shared content. Datacleaner’s duplicate detection feature, builds on Machine Learning principles for ease of configuration and improved inferential matching.