Smart data quality tool

For food industry players, precisely managing a product reference system can be a sizeable challenge. Today, thanks to machine learning, it is possible to optimise the quality of product information.

That was precisely the aim of our customer, who entrusted our teams of data scientists with the task of setting up an automated system to detect and analyse errors in its product reference system.

Background

When a company has a product catalogue comprising more than 6,000 products with over 250 variables, regularly updated and constantly changing, the quantity of product-related data just keeps increasing too…

And so does the number of errors! This is a critical situation that can throw the entire production line and distribution network supply chain into chaos.

Objectives

There were too many errors in the reference system, and business function validation rule systems were not comprehensive or responsive enough…

Our customer wanted to make its catalogue more reliable and thus reduce costs stemming from errors. Our aim was therefore to offer a solution that would detect PIM (Product Information Management) errors and suggest relevant corrections without any explicit written rules.

Methodology

Understanding the existing situation

Support with technical choices (databases, analysis tools, etc.)
Establishment of an efficient, responsive architecture

Expert support

On algorithmics and Machine Learning solutions;
On Python programming to develop models and adapt the results to make them easy to integrate in the tool;
On interactive and flexible visualisation design (Power BI).

Deliverable(s)

Development of algorithms to automatically detect errors in the product reference system using algorithms based on statistics and machine learning
Construction of an interactive analysis tool (Power BI) making it possible to understand the errors detected by the algorithms: interactive interface usable by the business functions (summary dashboard, correction prioritisation dashboard, dashboard giving the details of each error by product, correlation analysis).
Processing of false positives
Adoption of the tool by the business functions