Smart Meter Data Analytic Using Hadoop

Omkar Nibandhe and Abhishek Korpestudents

SMART METER DATA ANALYTICS

(SMDA)USING HADOOP

By :

Omkar Nibandhe ( Student )

Abhishek Korpe ( Student )

03-0

4-20

14SM

DA u

sing

HAD

OO

P

2

Why Smart Meter Data Analytics ?

03-0

4-20

14SM

DA u

sing

HAD

OO

P

3

What are SMART METERS ?

MIF

• Track and store the amount of energy used.• Send the collected data to the Energy

Distribution company server at regular time intervals.

House/Industry

Smart Meter Server

03-0

4-20

14SM

DA u

sing

HAD

OO

P

4

Advantages

• Service Provider• Demand-Response• Time of use tariff• Load Profile Analysis • Theft Detection• Billing Accuracy

• Customers• Usage Pattern• Billing Accuracy• Convenience in change

of service provider

03-0

4-20

14SM

DA u

sing

HAD

OO

P

5

Confronting the data deluge

• Rate Generation – 15 minutes.• For single meter – 3000 readings/month (approx ).

• For 1 million meters – 36 Billion readings/year (approx).

• Annual Growth – 13% ( 2010 – 2015 ).

• Total Shipment – 460.9 Million Smart Meters.

Source: Build smart metering solutions with IBM Informix TimeSeries

03-0

4-20

14SM

DA u

sing

HAD

OO

P

6

Capitalizing on the unique value of Hadoop

Solution ?

• Reducing data load times.• Improving query performance.• Massive Scalability.

03-0

4-20

14SM

DA u

sing

HAD

OO

P

7

Demand - Response

03-0

4-20

14SM

DA u

sing

HAD

OO

P

8

Time of use Tariff

03-0

4-20

14SM

DA u

sing

HAD

OO

P

9

Load Profile Analysis

Using hadoopUsing hadoop

load

time

03-0

4-20

14SM

DA u

sing

HAD

OO

P

10

Data Flow Diagram

Predictive Analysis

( FLUME )

Load Profile Analysis

03-0

4-20

14SM

DA u

sing

HAD

OO

P

11

Hadoop Cluster in LAB 412

Masters

Slaves

Slave1

Slave7 Slave8 Slave9 Slave11

Slave6Slave5Slave4Slave3Slave2

Slave10

Slave19Slave18Slave17Slave16Slave15Slave14Slave13

Slave12

Slave20 Slave21

03-0

4-20

14SM

DA u

sing

HAD

OO

P

12

LAB 412 (MESCOE Pune, India)

03-0

4-20

14SM

DA u

sing

HAD

OO

P

13

SMDA - NameNode

03-0

4-20

14SM

DA u

sing

HAD

OO

P

14

SMDA - SecondaryNameNode

03-0

4-20

14SM

DA u

sing

HAD

OO

P

15

SMDA - JobTracker

03-0

4-20

14SM

DA u

sing

HAD

OO

P

16

SMDA – Input ( .MIF )

03-0

4-20

14SM

DA u

sing

HAD

OO

P

17

1392 19501 0.157

SMDA - Output

03-0

4-20

14SM

DA u

sing

HAD

OO

P

18

Test Job 1

03-0

4-20

14SM

DA u

sing

HAD

OO

P

19

Test Job 2 ( Combiner )

03-0

4-20

14SM

DA u

sing

HAD

OO

P

20

Future Scope• Analysis of -• Customer segmentation.• Customer behavior.• Meter ping commands. • Outage management.• Power quality.• Extending data point(s) : weather, geographical location, family

consumption, etc.

03-0

4-20

14SM

DA u

sing

HAD

OO

P

21

Thanks to ….• Irish Social Science Data Archive ( ISSDA ).

• Rahul Khinvasara – Director, zCon Solutions Pvt. Ltd.

• Modern Education Society’s College of Engineering (MESCOE).• Prof. Balaji Bodkhe – Guide, MESCOE.• Prof. N. Shaikh – Head of Computer Department, MESCOE.• Prof. A. Hake – Vice Principal, MESCOE.• Prof. P. Raut – Administrative Head, MESCOE.

03-0

4-20

14SM

DA u

sing

HAD

OO

P

22

Any Suggestions ??

03-0

4-20

14SM

DA u

sing

HAD

OO

P

23

Thank You.

03-0

4-20

14SM

DA u

sing

HAD

OO

P

24

Any Questions?

03-0

4-20

14SM

DA u

sing

HAD

OO

P

25