Auckland Transport: Smarter Solutions for Better Service With Dataiku

Auckland Transport (AT) operates one of New Zealand’s largest and most complex transportation networks, serving millions of customers across New Zealand’s largest city. To improve customer experiences, streamline operations, and boost employee engagement, AT needed to eliminate inefficiencies and scale its data strategy.

AT adopted Dataiku as its centralized analytics and AI platform and integrated Microsoft Azure for cloud services. With Azure OpenAI, AT enriched unstructured text data and developed custom retrieval-augmented generation (RAG) applications to enhance their generative AI work.

Before we had Dataiku, our data science solutions were scattered — stored on personal drives, GitHub, or other places. Everyone was essentially working in silos. When we implemented Dataiku, part of the project was to migrate everything we had built onto the platform. This ensured that everything was in one place, where it could be properly managed and governed. Instead of having multiple pipelines, we started building a consolidated pipeline that could be used for a wide range of purposes. Tony Aitken GM AI & Advanced Analytics at Auckland Transport

Why Auckland Transport Chose Dataiku

AT chose Dataiku because it addressed three key priorities: usability, readiness, and seamless integration. Dataiku’s intuitive interface allowed the team to quickly adopt the platform and start seeing value immediately. Unlike other solutions requiring extensive customization, Dataiku offered an out-of-the-box experience that fit seamlessly into their existing workflows. Another standout feature was Dataiku’s seamless integration with AT’s existing data platforms, (such as Vertica, their database management software). By enabling the team to process data directly within their current infrastructure, Dataiku eliminated the need for large-scale data migration, saving time and optimizing resources.

We installed Dataiku, got up and running, and moved everything over within three months. The team absolutely loved it. Tony Aitken GM AI and Advanced Analytics at Auckland Transport

AT experienced these key benefits with Dataiku:

• Streamlined MLOps: Simplified updating models and managing them efficiently to optimize workflows.
• Robust Deployment and Monitoring: Enhanced the reliability of models with effective deployment and monitoring processes.
• Governance and Control: Enabled secure access to data and models while providing guardrails to ensure compliance.
• Transparency and Collaboration: Improved team collaboration by making workflows and data processes more accessible and clear.

Enhancing Customer Experience With Semantic Search

AT’s CRM system processes vast amounts of customer feedback from channels like social media, emails, online forms, and phone calls. Previously, the system relied on lexical or exact text search, which struggled to interpret context, synonyms, and nuances in feedback, making it difficult to identify and prioritize cases efficiently.

To resolve this, AT’s data science team developed a semantic search solution using techniques such as text pre-processing, cleaning, and Named Entity Recognition (NER). The solution integrated seamlessly with the CRM system and leveraged Power BI, enabling customer service teams to quickly access actionable insights. With semantic search, the customer experience team reduced case identification time from 30 minutes to just seconds. This improvement enhanced accuracy, enabling faster resolutions and building greater customer satisfaction and trust in AT’s services.

Another selling point was the fact that Dataiku was really good at the data scientist and citizen data scientist end of the spectrum. It’s low-code, easy to use, and easier for people. Tony Aitken GM AI and Advanced Analytics at Auckland Transport

Streamlining Operations With Data Pipeline Automation

Following the success of customer feedback improvements, AT focused on optimizing internal processes. Before Dataiku, the Performance and Analytics Team relied on manual data transformations with local Python scripts and shared CSV files, which often caused errors and inconsistencies.

With Dataiku, the team automated their entire data pipeline by integrating Python and SQL workflows into a unified platform. This transformation streamlined the process of turning raw data into performance metrics aggregated by local boards and network types, ensuring consistent and reliable outputs.

By eliminating manual tasks, the team saved 30 hours of work each month while also reducing the risk of errors. This increased efficiency delivered a $500K productivity gain and an additional $350K in use case value gain, both driven by time savings and the ability to repurpose resources toward higher-value activities and tasks. Furthermore, improved data consistency and enhanced monitoring capabilities allowed the team to troubleshoot issues faster and focus on delivering more accurate insights, improving overall efficiency.

Empowering Employee Engagement With Survey Analysis

AT also used Dataiku to transform how it analyzes employee feedback. The Capability & Culture team conducts quarterly employee check-in surveys via survey provider Qualtrics, which include open-ended responses. However, manually analyzing this data was slow and lacked depth, making it difficult for the team to extract actionable insights in a timely manner.

With Dataiku, the data science team streamlined the analysis process by leveraging advanced NLP techniques such as sentiment analysis, emotion detection, and key phrase extraction. These capabilities allowed the team to uncover deeper patterns in employee sentiment and gain actionable insights for more targeted actions.

By automating the process, the team dramatically reduced the time needed for analysis. This efficiency enabled them to address employee concerns faster, leading to increased engagement and satisfaction. As a result, AT saw a higher employee Net Promoter Score (eNPS), underscoring the value of a data-driven, employee-focused approach.

Dataiku allows Auckland Transport to take action on employee feedback quicker, lifting eNPS from negative to positive. Shane Ramsay Head of Capability and Culture at Auckland Transport

Looking Forward: Expanding AI and Generative AI

AT’s use of AI extends far beyond the three core use cases highlighted in this story, with 18 projects currently in production with Dataiku. Some standout initiatives include a parking prediction model that forecasts street parking availability to help drivers plan more efficiently and an NLP-backed tool that clusters website feedback and provides actionable insights to improve user experience. Another notable use case analyzes CRM data to enhance services for customers with accessibility needs, fostering a more inclusive transportation system.

It was this shift — having everything on one platform — that allowed us to consolidate and bring solutions together. Tony Aitken GM AI and Advanced Analytics at Auckland Transport

Building on these successes, AT is actively exploring Generative AI to further enhance its operations and services. For example, the team has implemented a Retrieval-Augmented Generation (RAG) solution to streamline responses to certain Official Information Act (OIA) requests and, ultimately, deliver faster and more consistent answers. This same technology has been adapted internally to help employees quickly access HR policies and is being leveraged to improve customer service, enabling the team to handle over 1.4 million queries annually with greater accuracy and efficiency.

With a diverse portfolio of AI-driven solutions and continued exploration of innovative technologies, AT is well-positioned to set the standard for leveraging AI to address current challenges, elevate customer experiences, and drive the future of transportation in New Zealand.