DNV GL Uses Databricks to Process Massive Volumes of Sensor Data

DNV GL, a provider of classification and technical assurance along with software and independent expert advisory services to the maritime, oil & gas and energy industries, has selected Databricks for large-scale energy data analytics. With Databricks, DNV GL utilizes innovative machine learning techniques to quantify the relationship between smart meter and weather pattern data. Through predictive models, it is able to accurately predict energy demand based on massive amounts of historical weather data, a process sometimes referred to simply as "weather normalization." This new analytics capability enables millions of households to quantify their energy savings.

DNV GL delivers testing and expertise to companies throughout the energy value chain. Its legacy data management solution was slow, cumbersome and unable to scale alongside the company's fast business growth, delaying time-to-insight and project completion. DNV GL selected Databricks to overcome these scalability and performance challenges in applying advanced analytics to transform the energy industry.

After implementation, DNV GL has seen the following benefits. The ability to provision and scale Spark clusters within minutes, compared to its legacy cloud analytics solution that took several hours. Increased speed of data processing by nearly 100 times without incurring additional operational costs compared to its legacy system. Eliminated time spent on DevOps tasks, allowing its data scientists and engineers to focus on solving data problems.

Additionally, Databricks' highly secure approach to providing access to multiple users paired with AWS S3's industry-leading security and encryption satisfied DNV GL's strict security requirements. Databricks provides DNV GL with a secure, managed data platform that allows it to focus on advanced modeling techniques and machine learning algorithms with far less computational burden.

"Our team members are first and foremost consultants, statisticians and data scientists. Spending inordinate amounts of time on DevOps delayed time-to-production and often required large investment in training or bringing on technical staff to help with infrastructure," said Jonathan Farland, senior data scientist at DNV GL. "Databricks allows us to address these completely revolutionary industry aspects without reinventing the wheel."

"DNV GL's ability to leverage advanced analytics at scale using Databricks has enabled it to deliver insights to its clients faster and at a lower overall cost," said Kavitha Mariappan, vice president of marketing at Databricks. "DNV GL is using the power and speed of the Databricks platform to make the next-generation smart grid a reality, contributing to more efficient energy .