Automating and Scaling Fraud Detection with AI

Appen overhauls its fraud detection operations, efficiently detecting and preventing scam with a scalable, ML-powered fraud detection platform designed and built by Provectus

Home » Case Study » Automating and Scaling Fraud Detection with AI

Appen is the leading provider of high-quality training data for organizations that build effective AI systems at scale. It fully integrated Figure Eight, a human-in-the-loop AI/ML-powered platform for data transformation, in their solution offering in April 2020, to help drive their effort of transforming text, image, audio, and video data into customized high-quality training data more efficiently.


Appen needed to automate its manual fraud detection mechanism to more efficiently detect and prevent malicious activity on their platform


Provectus designed and built an automated, end-to-end fraud detection platform with human-in-the-loop using TensorFlow and AWS products


Appen managed to scale trust and security of the platform, which allowed them to manage 20x more jobs per day and detect 25% more scammers

Satisfaction, safety, and trust of major enterprise customers

25% reduction in scammers on the platform

20x more jobs monitored per day; 97% automatically

5x fewer churned judgments about data


Moving Away from Manual Towards Automated Fraud Detection with AI/ML

Appen was looking to make their partially automated but mostly manual fraud detection system more efficient, in order to:

  • Scale the number of distributed workers it can monitor per day to raise the bar for detecting and preventing malicious activity on the platform
  • Build on their existing ability to train ML/DL models for their platform through automated data labeling, annotation, and categorization
  • Reduce the amount of manual work done by distributed workers, to increase the speed and efficiency of data processing and to eliminate human error

Appen used crowd workers to label data sets to train ML and DL models. Their fraud detection system — the one used to ensure data quality by eliminating low-quality contributions — was a manually activated solution triggered by SQL and Python scripts.

Though partially automated, such a solution did not allow to increase the efficiency of crowd workers and, most importantly, to scale fraud detection. The latter was the major requirement for the company to grow and attract new enterprise clients.

For Appen, it proved rather challenging to efficiently monitor more than 50 jobs per day in a mostly manual manner. The team considered hiring 20+ data analysts to keep up with the platform’s growth. The alternative option was an intelligent fraud detection solution powered by AI and machine learning.

Appen decided to join forces with Provectus to design and build an automated ML-powered fraud detection platform, with a scalable SaaS architecture and a GUI for optimal user experience.


Building an Automated, End-to-End Fraud Detection Platform with Human-in-the-Loop

To deliver an automated, end-to-end fraud detection platform with human-in-the-loop, Provectus did the following:

  • Designed and built data pipelines to make it easier for the Appen team to label, annotate, categorize, and moderate data
  • Designed, trained, and tuned highly accurate ML/DL models, which constiture the AI core of the fraud detection solution
  • Developed a user-friendly Web application for the Appen team, to enable more efficient handling and management of data and alerts
  • Ensured that all of the components of the fraud detection solution are automated and properly integrated for maximum efficiency and ease of use

Before moving forward with the development, the Provectus team did due diligence to research the latest academic papers in the crowdsourcing and fraud detection spaces, to ensure the envisioned solution would fit the bill on all accounts.

The machine learning and deep learning components — the AI core of the solution — were developed using TensorFlow. The DL models were deployed, served, and monitored with installed on top of Amazon ECS.

Java microservices, SQS events, and React.js UI application were utilized to automate data pipelines and ensure an end-to-end user workflow.

Continuous monitoring was enabled by using Prometheus and Grafana. The services allowed the Provectus team to provide complete visibility into the solutions’s performance to Appen’s engineers, fraud analysts, and business stakeholders.


Scaling Fraud Detection Operations on a New, AI/ML-powered Solution

Appen’s new ML-powered fraud detection platform enabled the team to process and handle 20x more jobs per day, with almost 97% of all jobs handled automatically.  Thanks to more efficient deep learning algorithms at its core, the platform allowed Appen to reduce scammer activity by no less than 25%, which resulted in a 5x drop in churned judgements.

The fraud detection platform brought about greater productivity and efficiency of staff, too. Appen managed to forego hiring 20+ data analysts, to considerably reduce operational costs in the long run.

Most importantly, the new fraud detection platform helped Appen meet the customers’ requirements in regard to data quality and service efficiency. The changes made allowed the company to better satisfy existing and attract new enterprise clients, to accelerate and make more sustainable their global expansion.

Moving Forward

  1. Learn more about the Provectus AI Solutions, ML Infrastructure, and Data QA
  2. Watch the webinar on MLOps and Data Quality: Deploying Reliable ML Models in Production
  3. Apply for Machine Learning Infrastructure Acceleration Program to get started

Contact Us!

Looking to explore the solution?

  • Hidden
  • Hidden
  • This field is for validation purposes and should be left unchanged.

See the Provectus privacy policy for details on how we collect, use, and share information about you.

See the Provectus privacy policy for details on how we collect, use, and share information about you.