The Law Society of BC: Optimizing Audits Through Risk Ranking and Anomaly Detection

We recently sat down with Thomas Kampioni, Director of IT at The Law Society of British Columbia to understand their key data and analytics use cases, the role Dataiku plays in helping them solve critical business problems, and the tangible value they have achieved since they started using Dataiku.

About Law Society of British Columbia

• The Law Society of British Columbia is a non-for-profit organization that regulates lawyers in British Columbia with the mandate to protect the public interest in the administration of law and to ensure independence and competence of lawyers.
• They also bring a voice to issues affecting the justice system and the delivery of legal services.
• The Law Society of BC regulates 13,000 practicing lawyers and 3,800 law firms.
• They cover the end to end law lifecycle, from regulation initiatives to access to justice.

Background

The senior management team at The Law Society of BC firmly believes that AI and machine learning will play an important role in their responsibilities in the near future. They knew it was time to take advantage of their collected data and leverage technology to identify patterns and behaviors and increase effectiveness and efficiencies within Law Society programs.

The first step in any AI or machine learning project is to define the business problem that needs to be solved. In the case of the Law Society, it was to increase the efficacy of the trust assurance audit program. The organization regulates 3,800 law firms and audits approximately 550 firms per year, which means that each firm is audited at least every four to six years. Predictive analytics and a risk-driven audit schedule allowed the team to focus their audit efforts on those firms that present higher risks in relation to the volume and type of trust transactions. 

The Law Society has three decades of historical data, which enables them to categorize law firms according to their risk level: low, neutral, or high risk. The organization made the decision to focus on risk factors and, from there, work to adjust the audit schedule based on the risk category of each firm. To approach this business problem, the team knew they had to:

• Bring in a data science and machine learning platform to help analyze the troves of data they have (which will continue to grow as their digital footprint does) and reduce the risk of human error.
• Modernize their data and analytics program to easily obtain insights into their own data, uncover trends and patterns, and solve business problems.
• Spend time properly defining the business objective in a SMART way (Specific, Measurable, Achievable, Relevant, and Time-Bound). This way, everyone on the team knows what they are trying to accomplish and, in the end, can assess if the project is successful.
• Form a group of motivated, relevant stakeholders, which included subject matter experts, data scientists, and a project manager.
• Iterate with the group to determine the factors for high-risk firms such as time in business, how much money they handle, and so on.

Primary Use Cases

The Law Society of BC evaluated a myriad of data science and machine learning platforms and ultimately chose Dataiku. The Law Society uses Dataiku to support proactive regulation of law firms. The trust audit program has the overall goal of being an effective and efficient program that helps ensure lawyers handle trust funds appropriately, and Dataiku helps identify risk factors for law firms. 

The Law Society uses predictive analytics in Dataiku to understand which factors contribute to a firm being a risk, such as years in business, the lawyers’ average years of practice, the number of complaints and hearings, the last trust audit score, and so on. 

Using Dataiku, they predict the probability of a firm’s risk from a compliance perspective, identifying the firms with high, low, and neutral risk factors according to their background and history. Upon identifying firms that fall under the high risk category, the Law Society then asks the firm for books and records to detect and analyze anomalies further using audit procedures and algorithms. 

A Collaborative Approach: The Law Society of BC and Dataiku

For The Law Society of BC, who currently use Dataiku’s free edition, the most appealing thing about Dataiku is the end-to-end aspect, enabling them to complete an evaluation, run a project, and share insights with executives via data visualizations within the platform. This came in handy during their proof of concept with Dataiku, demonstrating the value of the solution to a broad range of users. Further, they like how easy the tool is to use — you don’t need experience with traditional data science programming languages like Python or R to jumpstart data efforts, there’s a strong selection of on-the-shelf algorithms and visualization tools, and it’s easy to connect internal data sources. 

Thanks to Dataiku (and a robust analytics strategy), The Law Society of BC was able to:

• Improve the efficiency of their trust audit program, as analyzing large datasets can now be done in seconds to highlight the abnormalities that require auditor attention
• Reduce the number of auditors required per one large audit by 14% and the time needed to complete a large audit by 29%. While these reductions are credit to the Law Society’s own algorithm, Dataiku helped them understand the possibilities and what kinds of data analyses are possible
• Leverage predictive analytics and a risk-driven audit schedule to help prioritize audits 
• Enable data-driven decisions, helping them be more agile and efficient with the massive amounts of data they have
• Kickstart their machine learning efforts, as they found it easy to refine and train their models with Dataiku

Predictive Analytics With Dataiku

Customers building predictive analytics solutions with Dataiku benefit from:

• The ability to centrally and seamlessly connect to data, wherever it’s stored.
• A simple and fast interface for ETL, including interactive data cleaning and integrated advanced processors.
• AutoML features, including the ability to compare dozens of algorithms directly from the Dataiku interface (both for supervised and unsupervised tasks).
• One-click model deployment on the cloud with Kubernetes.
• Robust model monitoring features to prevent model drift.