Planning Your Hadoop NoSQL Projects for 2011

8/7/2019 Planning Your Hadoop NoSQL Projects for 2011

http://slidepdf.com/reader/full/planning-your-hadoop-nosql-projects-for-2011 1/19

Planning your Hadoop/NOSQL Projects

for 2011

Recorded version available at

http://www.impetus.com/webinar_registration?event=archived&eid=37

Impetus proprietary



Agenda

Why Hadoop/NoSQL ± a primer!

Challenges of moving to big data solutions- Hadoop, MPP

and/or NoSQL

Requirement assessment and planning for the right solution

An extension to existing IT strategy or a complete migration?


http://www.impetus.com/webinar_registration?event=archived&eid=37Impetus proprietary



The Bottlenecks

Analyze Hadoop/NoSQL fitment

Ease its adoption for business benefits

Minimize risk

Make the transformation seamless and disruption free

Alignment with IT strategy

Impetus proprietary





Hadoop/ NoSQLFitment analysis

Best Fit

solution

Create POC

Operational CostAnalysis

Deployment Cost

Analysis

Team Training Cost

SolutionImplementation

Production

Readiness

Deployment

The Failure-proof Way

Costing ImplementationCriteria

Impetus proprietary





Hadoop/NoSQL Fitment Analysis

Challenges/Factors

Can my business problem be solved using

Hadoop/NoSQL?

R&D is costly and resources are limited

What is the best solution among so many options?

Can I utilize µClouds¶ for operational costs?

Approach

Identify core pain areas, diagnose and choose suitableBig Data solution

Leverage knowledge of experienced Hadoop/NoSQL

professionals for compute & data intensive solutions

Impetus proprietary





Challenges/Factors

Hadoop/NoSQL have numerous solutions- which is the

right one for me?

Would the solution be cost effective?

Approach

Provide early associated costing and deployment

strategies

Use technical expertise on Hadoop/NoSQL technologies

Compute intensive and/or Data intensive?

Maximize reuse of existing resources

Best Fit Solution Identification

Impetus proprietary





Challenges/Factors

How to make sure the solution can be implemented?

How to verify that the solution is cost effective?

Batch processing v/s real time solution?

Approach

Provide a working prototype addressing key issues

Does the solution work?

Is it cost effective?

Is it production ready?

V erification wit h POC

Impetus proprietary





Challenges/Factors

Cost involved in development, release and hosting?

Hosting cost on Cloud- Server, RAM, CPU, Storage

Any specific hardware?

Approach

Analyzing operational cost of hosting on Cloud

Impetus¶ Hadoop deployment toolkit

Leverage knowledge and best practices

Deployment Cost Analysis

Impetus proprietary





Challenges/Factors

Recurring hardware/software cost?

Support and maintenance cost?

Approach

Operational scalability & efficiency analysis

Operational cost analysis

Operational Cost Analysis

Impetus proprietary





Challenges/Factors

Is my IT/Ops team trained to manage development/

deployment?

Hire new resources or train existing team?

Approach

Hadoop is map-reduce APIs built atop of Java- can be

easily learned by programmer

Will not take long to train the team NoSQL database design and implementation training

T raining Cost Analysis

Impetus proprietary





Challenges/Factors

Can we use traditional development processes in

Hadoop/NoSQL?

How to ensure that proper quality guidelines are being

followed?

Approach

Development cycle ± use proven product development

lifecycle management process

Plan for seamless transformation and disruption free

operations

Solution Implementation

Impetus proprietary





Challenges/Factors

How to prepare for production readiness?

How to tune production environment?

How is engineering going to cope with complex

implementations?

Approach

Proper expertise needed for production deployment

Production readiness checks Production environment tuning/optimizations

Proper training/handover to engineering team for

production deployment

Production Readiness &Deployment

Impetus proprietary





The Big Picture

Business

Strategy

Infrastructure

Development

Testing

BI and

Analytics

Data

Warehousing

Enterprise

Architecture

Impact on

IT Strategy

Impetus proprietary





Extension to current IT Strategy

Existing code reusability(business logic)

Minor modifications in test strategy

No major changes in release and build management of

Hadoop/NoSQL module

Quick setup for development/test/production

environments

Migration

Complete DW platform Data storage and archival

BI and Analytics

Application redesign

W hat C hanges Do I Need T o Make?

Impetus proprietary





Hadoop is NOT a new technology but a different

programming paradigm

NoSQL is a non-normalized way of data modeling

The Bottom Line

Hadoop/ NoSQL is SIMPLE!

Impetus proprietary





Case Study

Our Approach

� Assessment of the products that actually require

Hadoop/ NoSQL

� Replaced low latency read-write to NoSQL� Continued the use of ETL and used HDFS

The Challenge

� Where to use Hadoop

� Low RoI on Hadoop infrastructure

� Issues in Hadoop optimization and performance The Client

Global leader in internetservices and media

Client was able to reuse existing investments

Impetus proprietary



Case Study

Our Approach

� Consultation to determine right Big Data solution

�Used our SPDMMmethodology to validate NFR

� Proposed a suitable NoSQL solution

The Challenge

� Product targeting an enormous user base

� Data size ranging in TBs/PBs

� Traditional solutions could not address the

required data sizes The Client

A Fortune-500 enterprise

NoSQL solution saved cost for the client

Impetus proprietary



Questions

Impetus proprietary





Thank youFor more information,

write to us at [email protected]

Impetus proprietary



Planning Your Hadoop NoSQL Projects for 2011

Documents

Transcript of Planning Your Hadoop NoSQL Projects for 2011