Augmenting Sales and Marketing Operations in Financial Advisory Services with ML Lead Scoring

In today’s highly competitive market, financial advisors should be able to provide exceptional service to their clients, while also ensuring sustainable business growth. To do so, they look to innovative strategies and new technologies like artificial intelligence (AI) and machine learning (ML).

Carson Group Holdings LLC, a leading provider of financial advisory services, was looking for new and innovative ways to help their investment advisor clients acquire new customers more effectively. They decided to pursue the AI/ML adoption path, starting with a machine learning model for scoring leads received from Salesforce. By implementing ML lead scoring, Carson wanted to gain the ability to narrow down their leads, to steer their marketing team toward customers with the maximum likelihood of investing. The leaders of Carson hoped to reduce time spent filtering leads that were unlikely to convert, to significantly optimize costs associated with sales and marketing operations, while delivering actual business results and driving growth for their clients.

Lacking expertise in AI adoption and ML development, Carson joined forces with Provectus, an AWS Premier Tier Consulting Partner with competencies in Generative AI, Machine Learning, and Data & Analytics. Together, they designed and built a highly precise and accurate machine learning model for lead scoring, including EDA and modeling, development of an end-to-end model training pipeline, and implementation of an inference pipeline, designed to process new data and generate predictions from Salesforce.

This blog post delves into the details of the ML lead scoring project, including its challenges, solutions, and outcomes. It highlights the significant benefits that AI/ML adoption can bring to organizations providing services in the financial advisory sector.

01 The Challenge

Carson Group Holdings LLC is an entire ecosystem for financial advisors, designed to help them better serve their clients and unleash their firms’ full potential to drive business growth. Carson Group offers a wide range of services, including:

• Advisor coaching programs
• Process optimization development and investment strategies
• Discovery of lead and customer acquisition opportunities

With Carson, advisors receive industry-leading support for marketing, technology, compliance, investments, succession planning, and mergers and acquisition, enabling their firms to secure better positions on the market.

Carson wanted to enhance the potential of their sales and marketing services with more accurate lead scoring powered by AI & ML. Having the right data for training ML models, Carson had the potential to streamline the process of evaluating and scoring leads by their sales and marketing teams. Instead of relying on the complex rules and heuristics that enabled their existing predictive system, they hoped to leverage a self-training ML solution, to enjoy the highest level of accuracy and efficiency in lead scoring.

For Carson, it was clear that even a single ML model for scoring leads from Salesforce could augment their sales and marketing operations, and enable their teams to acquire new customers for their clients in the most effective and cost-efficient manner.

The ML lead scoring solution would empower them to identify leads with the greatest likelihood of customer conversion and investment, reducing the time spent filtering — and engaging with — leads that are less likely to convert.

02 The Solution

Provectus began the project by gaining a comprehensive understanding of Carson Group’s long-term objectives, as well as the challenges and obstacles that exist within their industry. The desired path forward required implementing AI in a specific area, starting with the ML lead scoring use case, to eventually generate at-scale business value with AI/ML across the entire organization.

Preparation Phase

During the preparation phase, a thorough assessment of Carson’s data was conducted. That included the analysis of such valuable data points as:

• Labeled lead records
• Marketing reports on impressions
• Marketing reports on clicks
• Lead conversion costs
• Marketing spend

In addition, a review of the available components for data ingestion and report generation was conducted. The rules and heuristics that were integrated into Carson’s current lead scoring solution were also assessed.

EDA and Modeling

The EDA and Modeling phase encompassed defining data schemas, comprehending the data’s structure and properties, cleaning the data and identifying data gaps, conducting feature engineering, training the baseline model, and developing a robust model evaluation framework.

The business process was thoroughly examined, and following a customer funnel analysis, the criteria for successful lead conversion were established. However, a challenge arose in determining the precise definition of a “failed” offer to a customer.

At first, all unsuccessful contacts were labeled as failed, which proved to be inaccurate because clients were only asked about their assets, age, location, and other relevant parameters after they answered the phone. If the interviewed customer met the specific profile deemed profitable for the company, they were then moved to the qualification stage. From that point forward, the customer’s intent became crucial. Thus, the task of the team was to determine the actual number of failed contacts within the qualification cases.

In addition, the dataset comprised various heterogeneous channels, each supplying distinct feature sets. To attain the maximum data volume required for effective model training, all cases were consolidated into a unified dataset.

During the training process, the model underwent continuous refinement and fine-tuning, to achieve an 88% recall rate and a 67% precision rate for predicting the probability of lead conversion.

Input Data

The lead scoring model was based on data gathered from Salesforce and lead activities data. Lead descriptive attributes such as the lead creation time (time window, day of the week), UTM source, amount of AUM (assets under management), lead account type, duration, and lead comments (text data) were used to work on the model.

However, because every lead in Carson’s pipeline came with multiple attributes, a problem emerged: What features should be incorporated into the model to ensure its best performance? Based on inputs from domain experts, it was proven that incorporating numerous attributes did not noticeably enhance the model.

The target variable of the model is lead status. For that reason, the model was trained to predict whether the final status of a lead is “converted” or not.

Development of the Training and Batch Inference Pipeline

The initial model was first implemented using “pure” notebooks. Then, the machine learning workflow was restructured and migrated to Amazon SageMaker Pipelines. SageMaker Pipelines allows for the automation and management of almost the entire ML lifecycle by effectively integrating CI/CD practices into machine learning.

The development of the Training pipeline involved data ingestion and preprocessing (training mode), tuning of model hyperparameters, model training and evaluation, and model release automation.

The deployment and implementation of a batch-mode Inference pipeline entailed data ingestion from Salesforce (inference mode), data preprocessing (inference mode), batch transformation, and exporting prediction results in user-friendly formats.

As depicted in the architecture diagram, a diverse suite of AWS services was utilized to facilitate the seamless ingestion, processing, and export of data from Salesforce, thereby enabling the generation of valuable lead insights in an optimal and efficient manner.

The final Training Pipeline includes the following components:

• The Data ingestion step (Amazon SageMaker Processing job) ingests the source data and applies basic data cleaning/filtering.
• The Data Preprocessing steps (Amazon SageMaker Processing job) preprocess the data, generates the features, split the images into full/train/test/val sets, and prepare the data to be consumed by Amazon SageMaker Training jobs.
• The Hyperparameter Tuning step (Amazon SageMaker HP Tuning job) takes as input a subset of the training and validation set and executes a series of small training jobs under the hood. As a result, the Tuning step determines the best parameters for the full training job.
• The Full Training step (Amazon SageMaker Training job) launches the training job on the entire data, given the best parameters from the Hyperparameter Tuning step.
• The Model Evaluation step (Amazon SageMaker Processing job) is executed once the final model has been trained. This step produces a report containing the model’s metrics on the test set.
• The Amazon SageMaker ModelCreate step wraps the model into the Amazon SageMaker Model package and pushes it to the Amazon SageMaker Model Registry. Model Registry allows to catalog and manage model versions, store model metadata (metrics, etc.), checks model status, and automates model deployment.

The Inference Pipeline includes the following components:

• The Data Ingestion and Preprocessing steps (Amazon SageMaker Processing Job) are reused from the Training pipeline. They perform the same set of actions with some minor differences.
• The Model Apply step (Amazon SageMaker Batch Transform job) takes the specified model from SageMaker Model Registry and applies this model to the preprocessed input data.
• Export Predictions step (Amazon SageMaker Processing Job) exports the predictions.

All steps are executed in an automated manner after the pipeline has been executed.

03 The Outcome

The Provectus Data and ML Engineering teams successfully delivered the lead scoring model in just five weeks. Carson promptly put the solution to the test in real-world scenarios, reaping the benefits of the ML engineering work within a matter of days.

In practice, the machine learning model for lead scoring exhibited an F1 score of over 80% for each evaluated lead, substantially enhancing the capability of Carson Group’s sales and marketing to focus on leads with a high potential for conversion.

The ML lead scoring solution utilized data from Salesforce, analyzing and processing it before delivering precise predictions to sales and marketing professionals. This seamless process allowed Carson Group to more efficiently leverage lead scores in their work, with the ability to quickly identify promising clients through the “potential to convert” score assigned to each lead.

As a result, the conversion process was significantly streamlined, leading to greater efficiency, improved customer satisfaction, substantial cost reduction, and enhanced flexibility and scalability for the Carson Group’s overall business operations.

Carson Group expressed satisfaction with the outcomes of Provectus’ efforts. Leanne Ball, Data Architect at Carson Group, stated:

“Carson Group had a great experience partnering with Provectus. The team was highly engaged, asked insightful questions, and showed great expertise in data science toolkits and implementation. Combined with our in-house subject matter and data expertise, we were able to create a valuable model that was ready for prime-time usage in a short amount of time. Thanks to Provectus, we were able to speed up the process of incorporating machine learning into our firm.”

Building on this success, Provectus eagerly anticipates continuing to provide Data and ML Engineering Services to Carson Group. This ongoing partnership aims to support their growth, foster innovation, and enhance resiliency within their specific market.

04 Conclusion

All of the Provectus deliverables outlined here have enabled Carson Group to narrow down their leads, to help their sales and marketing team focus on customers with the maximum likelihood of investing. By eliminating the need to spend valuable time filtering out less promising leads, marketing professionals were able to significantly optimize operational costs, while simultaneously driving growth for Carson’s investment advisor clients.

The rapid and seamless development and delivery of the Carson Group ML lead scoring solution was made possible by Provectus’ exceptional expertise in Data Science, AI Adoption, ML Development, and DevOps. Our unparalleled ability to identify the most suitable AI/ML use case for implementation played a critical role in the success of the Carson Group solution.

To learn more about the Provectus ML Use Case Discovery Program, visit the webpage and watch the webinar for more practical advice.

If you are interested in building a robust machine learning infrastructure for your AI/ML use cases, apply for the ML Acceleration Program to get started.