Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning...

6
Big Data Engineer

Transcript of Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning...

Page 1: Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning Path Overview Big Data Engineer Duration: 312 Hours Schedule : Full Day Morning

Big Data Engineer

Page 2: Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning Path Overview Big Data Engineer Duration: 312 Hours Schedule : Full Day Morning

All Rights reserved @ www.clslearn.com , Contact us : [email protected] , +201000216660 , +201001692348

Page 3: Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning Path Overview Big Data Engineer Duration: 312 Hours Schedule : Full Day Morning

Learning Path Overview

Big Data Engineer Duration:312 Hours

Schedule :Full Day Morning ( 9-5)

Half Day Evening (6-10)

Weekends Full Day (10-4)

Instructor-Led

Hands-On Training

Delivery Options:

In CLS Classroom.

On site Classroom.

Online Live.

Your Training Comes with

a 100% Satisfaction

Guarantee!

A Big Data Engineer is a person who creates and manages a company’s Big Data infrastructure and tools, and is someone that knows how to get results from vast amounts of data quickly.Responsibilities of a Big Data Engineer:Design, create, build & maintain data pipelinesAggregate & Transform raw data coming from a variety of data sources to fulfill the functional & non-functional business needsPerformance optimization: Automating processes, optimizing data delivery & re-designing the complete architecture to improve performance.Handling, transforming & managing Big Data using Big Data Frameworks & NoSQL databases.Building complete infrastructure to ingest, transform & store data for further analysis & business requirement.The courses in this learning paths covers the Required Skills To Become A Big Data Engineer which includes the following skills :Big Data Frameworks/Hadoop-based technologiesHDFS, YARN, Map Reduce, PIG & HIVE, Flume & Sqoop, Zookeeper, OozieReal-time processing Framework (Apache Spark)Database architectures: SQL-based technologies (e.g. MySQL)NoSQL technologies (e.g. Cassandra, MongoDB, HBaseUNIX/ LinuxCourse Included:1- Big Data Systems Fundamentals 100 Hours2- Big Data Systems Infrastructure 104 Hours3- Big Data Applied Models 108 Hours

All Rights reserved @ www.clslearn.com , Contact us : [email protected] , +201000216660 , +201001692348

Page 4: Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning Path Overview Big Data Engineer Duration: 312 Hours Schedule : Full Day Morning

Learning Path Outline

All Rights reserved @ www.clslearn.com , Contact us : [email protected] , +201000216660 , +201001692348

Big Data Systems Fundamentals 100 Hours* Linux and its Eco-system: History, Open

Source Philosophy, Distributions

* Installation of Linux OS with tips and tricks

* Linux file system

* Linux users groups and permissions

* Introduction to yum & dnf package

managers

* Basic file commands

* Basic process commands

* Basic network commands

* Hadoop and Big Data systems

characteristics

* Hadoop HDFS

* Hadoop Map-Reduce

* Writing Hadoop jobs

* Introduction to Hadoop yarn

* Hadoop eco-system

* Hadoop SQL like DB: Hive

* Hadoop ETL System like: NiFi

* Queueing Systems: Kafka

* Real Time analysis vs Batch processing

* Real Time analysis system implementation

* Real Time analysis using storm

• Storm Use case

Big Data Systems Infrastructure 104 Hours* Intro to Distributed Systems & HDFS

* Exploring Big Data Ecosystem &

Distributions

* Intro to Hadoop

* Introduction to Map Reduce

* Intro to YARN

* Intro to SQL-On-Hadoop

* Intro to Hive

* Basic Implementation of Hive & Hive

Server 2

* Hadoop Architecture In Depth

* Intro to Apache Zookeeper

* Advanced Hive Architecture

* Ingesting Data Into Hive Using Sqoop

* Intro to ETL Concepts

* Intro to Data Flow using Apache Nifi

* Ingesting Data into Hadoop

* Implementing Kafka Cluster

* Spark Implementation

* Simple Data Analysis with Apache Spark

* Using Sqoop to Connect to

RDBMS(Oracle / SQL Server)

* Creating Hive Table on ORC File

* Using Kafka to Connect to RDBMS(Oracle

/ SQL Server)

* Kafka Advanced Cluster Configuration

* Spark Cluster Implementation

* Using PySpark

* Integrating Spark with Hive

* Ingesting Data with Spark into Hive

* Implementing Apache Zeppelin

* Integrating Apache Zeppeling With Spark

Big Data Applied Models 108 Hours * Hadoop Cluster Service Discovery Using

Zookeeper

* Hadoop Cluster Management : HDS

Management and Optimization

* Hadoop Cluster Management : Yarn

Management and Optimization

* Hadoop Cluster Management :

Performance Optimization Using Tez

* Hadoop Cluster Management :

Performance Optimization for Map Reduce

Jobs

* Hadoop Cluster ETL : Workflow

automation Using Oozie

* Hadoop Cluster ETL : Data Manipulation

Using Apache NiFi

* Hadoop Cluster Data Load and Query

Using Pig

Page 5: Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning Path Overview Big Data Engineer Duration: 312 Hours Schedule : Full Day Morning

Learning Path Outcome Audience Profile

* Understand the evolution of data for the last three decades.

* Understand the Challenges in handling huge amount of data.

* Students will be trained as big data Engineers with real world

hands-on experience in Hadoop Administration, ETL

(Batch/Stream), Sql-On-Hadoop and many other technologies

related to Big Data.

* Big Data Cluster Installation , Configuration and Optimization

* Define workflows and tasks automation

* Data Extract , Load and Transform

* Data Analysis using SQL

- This course is for those new to Big Data and interested in

understanding why the Big Data Era has come to be.

- It is for those who want to become conversant with the

terminology and the core concepts behind big data

problems, applications, and systems.

- Who are interested in big data management, data

engineering and data analysis using Big Data

technologies to get hands on training about big data and

NoSQL

- Once you’ve identified a big data issue to analyze, how

do you collect, store and organize your data using Big

Data solutions?

Prerequisites

- Technical Skills :

Basic Knowledge of Java .

- Attending Course :Big Data Systems ( Level 1 )

- To gain the most from the workshop, the following is required:

- Knowledge of Programming, Basic of “Java, Scala, Or Python”

- Attending Course :

Big Data Systems ( Level 1 )

- Attending Course :Big Data Systems Infrastructure ( Level 2 )

Technical Skills:

- Basic Knowledge of Java .

- Basic Knowledge of Linux OS.

All Rights reserved @ www.clslearn.com , Contact us : [email protected] , +201000216660 , +201001692348

Page 6: Big Data Engineer › wp-content › uploads › 2019 › 09 › CLS... · 2020-05-13 · Learning Path Overview Big Data Engineer Duration: 312 Hours Schedule : Full Day Morning

We select the best instructors, who are certified from trustworthy

international vendors. They don’t only provide training program, but they

also share their professional experience with the students, so they can have

hands-on experience on the job market.

CLS facilities are well-equipped with strong hardware and software

technologies that aid both students and trainers lead very effective

smooth training programs.

We provide our clients with the best solutions, Our team of training advisers

answer whatever questions you have.

We have been in the market since 1995, and we kept accumulating

experience in the training business, and providing training for more than

100,000 trainees ever since, in Egypt, and the MENA region.

CLS is an authorized and accredited partner by technology leaders like

Microsoft, EC-Council, Adobe and Autodesk. This means that our

training programs are of the highest quality source materials, the most

up-to-date, and have the highest return on investment ever possible.

We keep tabs on every change in the market and the technology field,

so our training programs will always be updated up to the World-class

latest standards, and adapted to the global shape-shifting job market.

Our clients prefer our training programs not only for the quality

education they get, but also because they are cost effective.

All Rights reserved @ www.clslearn.com , Contact us : [email protected] , +201000216660 , +201001692348