ATTOM Data Solutions Story

4
Whether you’re an individual homeowner or a major corporation, buying residential property could be either one of the best or worst investments you’ll ever make. While luck plays a certain role, most buyers would prefer to count on something else: reliable data. However, gathering and analyzing all of the relevant information—including local market data, risk factors, and sale history—can be challenging. Fortunately, ATTOM Data Solutions makes getting an informed decision that much easier.Localized news alerts in the “land of fire and floods” “For our customers, SQL Server 2016 in a hybrid-cloud environment with Azure really cuts through all the logistics of managing large data sets. Rather than requiring them to manage file-based ETL processes, we can provide data as a service, enabling our customers to focus on their business instead of information management.” Richard Sawicky, Chief Data Officer, ATTOM Data Solutions Transforming real estate data services with hybrid- cloud technologies

Transcript of ATTOM Data Solutions Story

Whether yoursquore an individual homeowner or a major corporation buying residential property could be either one of the best or worst investments yoursquoll ever make While luck plays a certain role most buyers would prefer to count on something else reliable data However gathering and analyzing all of the relevant informationmdashincluding local market data risk factors and sale historymdashcan be challenging Fortunately ATTOM Data Solutions makes getting an informed decision that much easierLocalized news alerts in the ldquoland of fire and floodsrdquo

ldquoFor our customers SQL Server 2016 in a hybrid-cloud environment with Azure really cuts through all the logistics of managing large data sets Rather than requiring them to manage file-based ETL processes we can provide data as a service enabling our customers to focus on their business instead of information managementrdquo

Richard Sawicky Chief Data Officer ATTOM Data Solutions

Transforming real estate data services with hybrid-cloud technologies

Customer Name ATTOM Data Solutions Industry Professional servicesmdashReal estateCountry or Region United StatesCustomer Website wwwattomdatacomEmployee Size 80

Customer ProfileATTOM Data Solutions is the curator of the ATTOM Data Warehouse a multi-sourced national property database that aggregates property tax deed mortgage foreclosure environmental risk natural hazard neighborhood characteristics and other property characteristic data for more than 150 million US properties The ATTOM Data Warehouse delivers actionable data to businesses consumers government agencies universities policymakers and the media in multiple ways including bulk file licenses APIs and customized reports

Facing data challenges ATTOM Data Solutions provides detailed insight into property parcels across the United States for industries such as real estate government education and media Its national data sets include information on more than 150 million residential and commercial properties The company delivers its customized data sets in multiple ways including data file licensing an application programming interface (API) custom reports and neighborhood and property due-diligence sites

RealtyTrac now part of parent company ATTOM Data Solutions was created in 1996 Since then data has grown rapidly in both volume and complexity Today people increasingly expect more on-demand services Staying on top of those changes required a different approach Keeping the data warehouse up-to-date required increasingly time-consuming extract transform and load (ETL) processes and the company needed more flexibility to handle new types of data such as geospatial information ldquoWe were hitting some thresholds that were really uncomfortablerdquo says Richard Sawicky Chief Data Officer at ATTOM Data Solutions ldquoAfter receiving data from our collection team carving the information into individual data sets and sharing them in a 24-hour period could be challengingrdquo

ATTOM Data Solutions needed a more elastic data platform to not only handle more business but also offer innovative

new services ldquoWe want to be able to grow with our customersrdquo says Brittney Wooten Director of Marketing at ATTOM Data Solutions ldquoAs our customers continue to evolve and grow their business we want to ensure that we have the capabilities to serve them and their expanding business needs Simultaneously as our data warehouse and product offerings grow we need to know that we have the infrastructure and ability to provide optimal data solutions for companies that have larger more intricate data needsrdquo

ATTOM believed that the real estate industry as a whole was also due for a change ldquoFlat file delivery is still the standard today for data providers requiring customers to build their own ETL to ingest datardquo explains Sawicky ldquoHow many ways should you have to construct a real estate database schema if what you really want is just access to data to get insight into your business and productsrdquo

Creating a hybrid-cloud roadmapTo take on new challenges ATTOM decided to become an early adopter of Microsoft SQL Server 2016 and ultimately extend to a hybrid-cloud infrastructure by connecting to the Microsoft Azure platform ldquoATTOM Data Solutions has always been primarily a Microsoft shop from the website down to the database layerrdquo says Sawicky ldquoAnd as far as the data warehouse environment goes the

ldquoProcesses that ran 24 hours on our old platform came back in less than three hours The ability to generate the files much faster and apply updates more quickly has definitely increased our ability to serve more complicated data setsrdquo

Richard Sawicky Chief Data Officer ATTOM Data Solutions

selection was SQL Server for two reasons First the price is much more competitive compared to Oracle and some of the other choices out there Also we have more developers and other skilled resources available for SQL Serverrdquo

The company was attracted to several enhancements including improvements in columnstore indexing as well as new features such as temporal tables Also called system-versioned tables temporal tables store a history of data changes that could be used to support ATTOM Data Solutionsrsquo auditing needs And the numerous performance enhancementsmdashincluding more efficient cube processing and data compressionmdashwould also speed reporting

ATTOM Data Solutions installed SQL Server 2016 in two clusters taking advantage of the SQL Server AlwaysOn Availability Groups feature to enable disaster recovery and ensure high-levels of performance as the company expands While currently still on-premises the company looks forward to extending to Azure for burst-processing and enhanced disaster recovery capabilities Sawicky says ldquoWith our new architecture we could spin up an additional availability group whether on-premises or in the cloud and keep scaling with the business as neededrdquo

ETL processes are handled by the primary server which replicates data to a secondary server that takes care of cube updates analytics processes and bulk file delivery As a result the company has achieved dramatic performance gains in a major service that provides large data sets of real estate and risk information ldquoIn our present methodology we write massive flat files And depending on the

customerrsquos needs those can be quite complicated data setsrdquo says Sawicky ldquoWe saw up to 10 times speed improvement on some of the extraction processes Processes that ran 24 hours on our old platform came back in less than three hours The ability to generate the files much faster and apply updates more quickly has definitely increased our ability to serve more complicated data setsrdquo

In addition to handling more complicated data sets with ease ATTOM Data Solutions has the scalability it needs to meet SLAs now and in the future ldquoWith the SQL Server 2016 platform we have room to breatherdquo says Sawicky ldquoGaining the ability to run complex data setsmdashespecially requests for customized flat filesmdashup to 10 times faster has put us well ahead of our service level Irsquove been very happy about thatrdquo

Powering business growth gaining scalability and cutting costsThe ability to provide new and more services means that ATTOM Data Solutions can step up business expansion In the past the company had wanted a more scalable technology platform before accelerating growth Now ATTOM Data Solutions has ample room to increase both its portfolio of offerings and its customer base Sawicky says ldquoWersquore looking at growth in our ability to not only deliver more bulk data sets but also expand the data marts that feed our APIrdquo

Data compression also gives the company more headroom ldquoAfter we loaded new data sets in our legacy data warehouse

ldquoWith the SQL Server 2016 platform we have room to breathe Gaining the ability to run complex data setsmdashespecially requests for customized flat filesmdashup to 10 times faster has put us well ahead of our service level Irsquove been very happy about thatrdquo

Richard Sawicky Chief Data Officer ATTOM Data Solutions

the volume was about 125 terabytesrdquo says Sawicky ldquoBy using a custom columnstore architecture in SQL Server 2016 we consolidated to about 45 terabytes And when yoursquore talking about high-end SSD storage the savings isnrsquot trivialrdquo

Transforming the real estate industryConnecting to the cloud will give ATTOM Data Solutions even more room to grow with the ability to spin up more servers as needed whether on-premises or on Azure The company will also be able to take advantage of a host of new capabilities including machine learning and predictive analytics For even more flexibility ATTOM Data Solutions is currently testing SQL Server R Services a new feature in SQL Server 2016 that enables Sawickyrsquos team to work with data in the open-source R language

Sawicky sees its roadmap as an opportunity to ultimately modernize the real estate industry ldquoFor our customers SQL Server 2016 in a hybrid-cloud environment with Azure really cuts through all the logistics of managing large data setsrdquo he says ldquoRather than requiring them to manage file based ETL processes we can provide data as a service enabling our customers to focus on their business instead of information managementrdquo

This case study is for informational purposes only MICROSOFT MAKES NO WARRANTIES EXPRESS OR IMPLIED INTHIS SUMMARY Document published July 2016

Softwarebull Microsoft SQL Server 2016

Customer Name ATTOM Data Solutions Industry Professional servicesmdashReal estateCountry or Region United StatesCustomer Website wwwattomdatacomEmployee Size 80

Customer ProfileATTOM Data Solutions is the curator of the ATTOM Data Warehouse a multi-sourced national property database that aggregates property tax deed mortgage foreclosure environmental risk natural hazard neighborhood characteristics and other property characteristic data for more than 150 million US properties The ATTOM Data Warehouse delivers actionable data to businesses consumers government agencies universities policymakers and the media in multiple ways including bulk file licenses APIs and customized reports

Facing data challenges ATTOM Data Solutions provides detailed insight into property parcels across the United States for industries such as real estate government education and media Its national data sets include information on more than 150 million residential and commercial properties The company delivers its customized data sets in multiple ways including data file licensing an application programming interface (API) custom reports and neighborhood and property due-diligence sites

RealtyTrac now part of parent company ATTOM Data Solutions was created in 1996 Since then data has grown rapidly in both volume and complexity Today people increasingly expect more on-demand services Staying on top of those changes required a different approach Keeping the data warehouse up-to-date required increasingly time-consuming extract transform and load (ETL) processes and the company needed more flexibility to handle new types of data such as geospatial information ldquoWe were hitting some thresholds that were really uncomfortablerdquo says Richard Sawicky Chief Data Officer at ATTOM Data Solutions ldquoAfter receiving data from our collection team carving the information into individual data sets and sharing them in a 24-hour period could be challengingrdquo

ATTOM Data Solutions needed a more elastic data platform to not only handle more business but also offer innovative

new services ldquoWe want to be able to grow with our customersrdquo says Brittney Wooten Director of Marketing at ATTOM Data Solutions ldquoAs our customers continue to evolve and grow their business we want to ensure that we have the capabilities to serve them and their expanding business needs Simultaneously as our data warehouse and product offerings grow we need to know that we have the infrastructure and ability to provide optimal data solutions for companies that have larger more intricate data needsrdquo

ATTOM believed that the real estate industry as a whole was also due for a change ldquoFlat file delivery is still the standard today for data providers requiring customers to build their own ETL to ingest datardquo explains Sawicky ldquoHow many ways should you have to construct a real estate database schema if what you really want is just access to data to get insight into your business and productsrdquo

Creating a hybrid-cloud roadmapTo take on new challenges ATTOM decided to become an early adopter of Microsoft SQL Server 2016 and ultimately extend to a hybrid-cloud infrastructure by connecting to the Microsoft Azure platform ldquoATTOM Data Solutions has always been primarily a Microsoft shop from the website down to the database layerrdquo says Sawicky ldquoAnd as far as the data warehouse environment goes the

ldquoProcesses that ran 24 hours on our old platform came back in less than three hours The ability to generate the files much faster and apply updates more quickly has definitely increased our ability to serve more complicated data setsrdquo

Richard Sawicky Chief Data Officer ATTOM Data Solutions

selection was SQL Server for two reasons First the price is much more competitive compared to Oracle and some of the other choices out there Also we have more developers and other skilled resources available for SQL Serverrdquo

The company was attracted to several enhancements including improvements in columnstore indexing as well as new features such as temporal tables Also called system-versioned tables temporal tables store a history of data changes that could be used to support ATTOM Data Solutionsrsquo auditing needs And the numerous performance enhancementsmdashincluding more efficient cube processing and data compressionmdashwould also speed reporting

ATTOM Data Solutions installed SQL Server 2016 in two clusters taking advantage of the SQL Server AlwaysOn Availability Groups feature to enable disaster recovery and ensure high-levels of performance as the company expands While currently still on-premises the company looks forward to extending to Azure for burst-processing and enhanced disaster recovery capabilities Sawicky says ldquoWith our new architecture we could spin up an additional availability group whether on-premises or in the cloud and keep scaling with the business as neededrdquo

ETL processes are handled by the primary server which replicates data to a secondary server that takes care of cube updates analytics processes and bulk file delivery As a result the company has achieved dramatic performance gains in a major service that provides large data sets of real estate and risk information ldquoIn our present methodology we write massive flat files And depending on the

customerrsquos needs those can be quite complicated data setsrdquo says Sawicky ldquoWe saw up to 10 times speed improvement on some of the extraction processes Processes that ran 24 hours on our old platform came back in less than three hours The ability to generate the files much faster and apply updates more quickly has definitely increased our ability to serve more complicated data setsrdquo

In addition to handling more complicated data sets with ease ATTOM Data Solutions has the scalability it needs to meet SLAs now and in the future ldquoWith the SQL Server 2016 platform we have room to breatherdquo says Sawicky ldquoGaining the ability to run complex data setsmdashespecially requests for customized flat filesmdashup to 10 times faster has put us well ahead of our service level Irsquove been very happy about thatrdquo

Powering business growth gaining scalability and cutting costsThe ability to provide new and more services means that ATTOM Data Solutions can step up business expansion In the past the company had wanted a more scalable technology platform before accelerating growth Now ATTOM Data Solutions has ample room to increase both its portfolio of offerings and its customer base Sawicky says ldquoWersquore looking at growth in our ability to not only deliver more bulk data sets but also expand the data marts that feed our APIrdquo

Data compression also gives the company more headroom ldquoAfter we loaded new data sets in our legacy data warehouse

ldquoWith the SQL Server 2016 platform we have room to breathe Gaining the ability to run complex data setsmdashespecially requests for customized flat filesmdashup to 10 times faster has put us well ahead of our service level Irsquove been very happy about thatrdquo

Richard Sawicky Chief Data Officer ATTOM Data Solutions

the volume was about 125 terabytesrdquo says Sawicky ldquoBy using a custom columnstore architecture in SQL Server 2016 we consolidated to about 45 terabytes And when yoursquore talking about high-end SSD storage the savings isnrsquot trivialrdquo

Transforming the real estate industryConnecting to the cloud will give ATTOM Data Solutions even more room to grow with the ability to spin up more servers as needed whether on-premises or on Azure The company will also be able to take advantage of a host of new capabilities including machine learning and predictive analytics For even more flexibility ATTOM Data Solutions is currently testing SQL Server R Services a new feature in SQL Server 2016 that enables Sawickyrsquos team to work with data in the open-source R language

Sawicky sees its roadmap as an opportunity to ultimately modernize the real estate industry ldquoFor our customers SQL Server 2016 in a hybrid-cloud environment with Azure really cuts through all the logistics of managing large data setsrdquo he says ldquoRather than requiring them to manage file based ETL processes we can provide data as a service enabling our customers to focus on their business instead of information managementrdquo

This case study is for informational purposes only MICROSOFT MAKES NO WARRANTIES EXPRESS OR IMPLIED INTHIS SUMMARY Document published July 2016

Softwarebull Microsoft SQL Server 2016

selection was SQL Server for two reasons First the price is much more competitive compared to Oracle and some of the other choices out there Also we have more developers and other skilled resources available for SQL Serverrdquo

The company was attracted to several enhancements including improvements in columnstore indexing as well as new features such as temporal tables Also called system-versioned tables temporal tables store a history of data changes that could be used to support ATTOM Data Solutionsrsquo auditing needs And the numerous performance enhancementsmdashincluding more efficient cube processing and data compressionmdashwould also speed reporting

ATTOM Data Solutions installed SQL Server 2016 in two clusters taking advantage of the SQL Server AlwaysOn Availability Groups feature to enable disaster recovery and ensure high-levels of performance as the company expands While currently still on-premises the company looks forward to extending to Azure for burst-processing and enhanced disaster recovery capabilities Sawicky says ldquoWith our new architecture we could spin up an additional availability group whether on-premises or in the cloud and keep scaling with the business as neededrdquo

ETL processes are handled by the primary server which replicates data to a secondary server that takes care of cube updates analytics processes and bulk file delivery As a result the company has achieved dramatic performance gains in a major service that provides large data sets of real estate and risk information ldquoIn our present methodology we write massive flat files And depending on the

customerrsquos needs those can be quite complicated data setsrdquo says Sawicky ldquoWe saw up to 10 times speed improvement on some of the extraction processes Processes that ran 24 hours on our old platform came back in less than three hours The ability to generate the files much faster and apply updates more quickly has definitely increased our ability to serve more complicated data setsrdquo

In addition to handling more complicated data sets with ease ATTOM Data Solutions has the scalability it needs to meet SLAs now and in the future ldquoWith the SQL Server 2016 platform we have room to breatherdquo says Sawicky ldquoGaining the ability to run complex data setsmdashespecially requests for customized flat filesmdashup to 10 times faster has put us well ahead of our service level Irsquove been very happy about thatrdquo

Powering business growth gaining scalability and cutting costsThe ability to provide new and more services means that ATTOM Data Solutions can step up business expansion In the past the company had wanted a more scalable technology platform before accelerating growth Now ATTOM Data Solutions has ample room to increase both its portfolio of offerings and its customer base Sawicky says ldquoWersquore looking at growth in our ability to not only deliver more bulk data sets but also expand the data marts that feed our APIrdquo

Data compression also gives the company more headroom ldquoAfter we loaded new data sets in our legacy data warehouse

ldquoWith the SQL Server 2016 platform we have room to breathe Gaining the ability to run complex data setsmdashespecially requests for customized flat filesmdashup to 10 times faster has put us well ahead of our service level Irsquove been very happy about thatrdquo

Richard Sawicky Chief Data Officer ATTOM Data Solutions

the volume was about 125 terabytesrdquo says Sawicky ldquoBy using a custom columnstore architecture in SQL Server 2016 we consolidated to about 45 terabytes And when yoursquore talking about high-end SSD storage the savings isnrsquot trivialrdquo

Transforming the real estate industryConnecting to the cloud will give ATTOM Data Solutions even more room to grow with the ability to spin up more servers as needed whether on-premises or on Azure The company will also be able to take advantage of a host of new capabilities including machine learning and predictive analytics For even more flexibility ATTOM Data Solutions is currently testing SQL Server R Services a new feature in SQL Server 2016 that enables Sawickyrsquos team to work with data in the open-source R language

Sawicky sees its roadmap as an opportunity to ultimately modernize the real estate industry ldquoFor our customers SQL Server 2016 in a hybrid-cloud environment with Azure really cuts through all the logistics of managing large data setsrdquo he says ldquoRather than requiring them to manage file based ETL processes we can provide data as a service enabling our customers to focus on their business instead of information managementrdquo

This case study is for informational purposes only MICROSOFT MAKES NO WARRANTIES EXPRESS OR IMPLIED INTHIS SUMMARY Document published July 2016

Softwarebull Microsoft SQL Server 2016

the volume was about 125 terabytesrdquo says Sawicky ldquoBy using a custom columnstore architecture in SQL Server 2016 we consolidated to about 45 terabytes And when yoursquore talking about high-end SSD storage the savings isnrsquot trivialrdquo

Transforming the real estate industryConnecting to the cloud will give ATTOM Data Solutions even more room to grow with the ability to spin up more servers as needed whether on-premises or on Azure The company will also be able to take advantage of a host of new capabilities including machine learning and predictive analytics For even more flexibility ATTOM Data Solutions is currently testing SQL Server R Services a new feature in SQL Server 2016 that enables Sawickyrsquos team to work with data in the open-source R language

Sawicky sees its roadmap as an opportunity to ultimately modernize the real estate industry ldquoFor our customers SQL Server 2016 in a hybrid-cloud environment with Azure really cuts through all the logistics of managing large data setsrdquo he says ldquoRather than requiring them to manage file based ETL processes we can provide data as a service enabling our customers to focus on their business instead of information managementrdquo

This case study is for informational purposes only MICROSOFT MAKES NO WARRANTIES EXPRESS OR IMPLIED INTHIS SUMMARY Document published July 2016

Softwarebull Microsoft SQL Server 2016