Automating and Scaling Fraud Detection with AI

Moving Away from Manual Towards Automated Fraud Detection with AI/ML

Appen was looking to make their partially automated but mostly manual fraud detection system more efficient, in order to:

• Scale the number of distributed workers it can monitor per day to raise the bar for detecting and preventing malicious activity on the platform
• Build on their existing ability to train ML/DL models for their platform through automated data labeling, annotation, and categorization
• Reduce the amount of manual work done by distributed workers, to increase the speed and efficiency of data processing and to eliminate human error

Appen used crowd workers to label data sets to train ML and DL models. Their fraud detection system — the one used to ensure data quality by eliminating low-quality contributions — was a manually activated solution triggered by SQL and Python scripts.

Though partially automated, such a solution did not allow to increase the efficiency of crowd workers and, most importantly, to scale fraud detection. The latter was the major requirement for the company to grow and attract new enterprise clients.

For Appen, it proved rather challenging to efficiently monitor more than 50 jobs per day in a mostly manual manner. The team considered hiring 20+ data analysts to keep up with the platform’s growth. The alternative option was an intelligent fraud detection solution powered by AI and machine learning.

Appen decided to join forces with Provectus to design and build an automated ML-powered fraud detection platform, with a scalable SaaS architecture and a GUI for optimal user experience.

Building an Automated, End-to-End Fraud Detection Platform with Human-in-the-Loop

To deliver an automated, end-to-end fraud detection platform with human-in-the-loop, Provectus did the following:

• Designed and built data pipelines to make it easier for the Appen team to label, annotate, categorize, and moderate data
• Designed, trained, and tuned highly accurate ML/DL models, which constiture the AI core of the fraud detection solution
• Developed a user-friendly Web application for the Appen team, to enable more efficient handling and management of data and alerts
• Ensured that all of the components of the fraud detection solution are automated and properly integrated for maximum efficiency and ease of use

Before moving forward with the development, the Provectus team did due diligence to research the latest academic papers in the crowdsourcing and fraud detection spaces, to ensure the envisioned solution would fit the bill on all accounts.

The machine learning and deep learning components — the AI core of the solution — were developed using TensorFlow. The DL models were deployed, served, and monitored with Hydrosphere.io installed on top of Amazon ECS.

Java microservices, SQS events, and React.js UI application were utilized to automate data pipelines and ensure an end-to-end user workflow.

Continuous monitoring was enabled by using Prometheus and Grafana. The services allowed the Provectus team to provide complete visibility into the solutions’s performance to Appen’s engineers, fraud analysts, and business stakeholders.

Scaling Fraud Detection Operations on a New, AI/ML-powered Solution

Appen’s new ML-powered fraud detection platform enabled the team to process and handle 20x more jobs per day, with almost 97% of all jobs handled automatically.  Thanks to more efficient deep learning algorithms at its core, the platform allowed Appen to reduce scammer activity by no less than 25%, which resulted in a 5x drop in churned judgements.

The fraud detection platform brought about greater productivity and efficiency of staff, too. Appen managed to forego hiring 20+ data analysts, to considerably reduce operational costs in the long run.

Most importantly, the new fraud detection platform helped Appen meet the customers’ requirements in regard to data quality and service efficiency. The changes made allowed the company to better satisfy existing and attract new enterprise clients, to accelerate and make more sustainable their global expansion.

Moving Forward

1. Learn more about the Provectus AI Solutions, ML Infrastructure, and Data QA
2. Watch the webinar on MLOps and Data Quality: Deploying Reliable ML Models in Production
3. Apply for Machine Learning Infrastructure Acceleration Program to get started