Big Data & Hadoop

Big Data, the technology in which our IT industries are investing at very large scale to manage and maintain large set of data. These data sets are used to be analyzed computationally to reveal patterns, trends, and associations, especially relating to human behavior and interactions. Big data challenges include capturing data, data storage, data analysis, search, sharing, transfer, visualization, querying, updating and information privacy. Our workshop will introduce all the core concepts of big data computing.

Course Highlights
Participants will get aware with concepts related to Bigg Data & Hadoop such as:
  • Hadoop Project Implementation
  • Writing Complex Mapreduce Programs
  • Data Analytics using Pig & Hive
  • Understand Data Loading Techniques using Sqoop and Flume
  • Master the concepts of Hadoop Distributed File System and Map Reduce Framework
  • Work on Big Data Project

 

Projects:
  • Word Count for a large amount of data With Using (MapReduce , Pig , Hive )
  • Temperature Sensor Conversation With Using Pig or HQL
  • DATA Sorting With Large data Set With Pig and Sqoop
  • Write Complex MapReduce programs
  • Master the concepts of Hadoop Distributed File System and MapReduce Framework
  • Work on a Real Life Project on Big Data Analytics and gain Hands on Project Experience

 

Certifications:
  • Certificate of Participation from SkillThon-2018 in Association with ELAN & Nvision IIT Hydrabaad and Microsoft AEP.
  • Certificate of merit from SkillThon-2018 in Association with ELAN & Nvision IIT Hydrabaad and Microsoft AEP.
  • Certificate of Coordination from SkillThon-2018 in Association with ELAN & Nvision IIT Hydrabaad and Microsoft AEP.

 

Participation Benefits:
  • Learn & Interact with renowned Industry Experts.
  • Project Completion letter to each attendee from SkillIT Nation New delhi
  • The Certificate of Participation in association with ELAN & Nvision IIT-H
  • Free CD/DVD containing Software Resource Toolkit
  • Students will get the opportunity to have Free 6 month project based Internship.
  • Top Teams will be awarded as winners with the certificate of honour & respected prizes at IIT-Hydrabaad

 

Agenda:
Day 1
Introduction to Big Data and Hadoop
  • What is a Data?
  • Type of Data
  • Need of Big Data
  • Characteristics of Big Data

 

Different Components of Hadoop
  • HDFS Architecture
  • HDFS internals and use cases
  • HDFS Daemons
  • Files and blocks
  • Name node memory concerns
  • Secondary name node
  • HDFS access options
  • Installing and configuring Hadoop
  • Hadoop daemons
  • Basic Hadoop commands
  • Hands-on exercise

 

Big Data Technology
  • Traditional IT approach
  • Big Data Capabilities
  • Milestones of Hadoop

 

Day 2
Software Introduction
  • VMware Player
  • VMware installed with BIOS system
  • Horton Works Sand Box Introduction

 

Hadoop Architecture
  • Hadoop cluster
  • Hadoop Core Services
  • Hadoop Core Components
  • Map reduce Introduction
  • HDFS

 

Starting with Hadoop
  • Map reduce Analogy
  • Map reduce Example
  • Map Execution
  • Real time Using with Hadoop

 

Day 3:
Cloud computing overview and installing Hadoop on multiple nodes 
  • Cloud computing overview
  • SaaS/PaaS/IaaS
  • Characteristics of cloud computing SaaS/PaaS/IaaS
  • Cluster configurations SaaS/PaaS/IaaS
  • Configuring Masters and Slaves
  • Functional programming concepts
  • List processing
  • Mapping and reducing lists
  • Putting them together in MapReduce
  • Word Count example application
  • Understanding the driver, mapper and reducer
  • Closer look at MapReduce data flow
  • Additional MapReduce functionality
  • Fault tolerance
  • Hands-on exercises

 

Advanced MapReduce concepts
  • Hands-on work on MapReduce
  • Understand combiners & partitioners
  • Understand input and output formats
  • Distributed cache
  • Understanding counters
  • Chaining, listing and killing jobs
  • Hands-On Exercise

 

Day 4:
Pig
  • Introduction to Apache Pig
  • Components of pig
  • How to Works pig, or Data model
  • Pig vs. SQL
  • Pig Execution Modes

 

Pig
  • Map Reduce vs. Apache Pig
  • Different Data Types in Pig
  • Modes of Execution in Pig
  • Local Mode
  • Execution Mechanism
  • Grunt Shell
  • Scrip
  • Pig Commands
  • Examples Of pig
  • Word Count
  • Batting Examples

 

Day 5:
SQOOP
  • Introduction to SQOOP
  • MySQL Client and Server Installation
  • How to Connect to Relational Database
  • Using SQOOP
  • Different SQOOP Commands
  • Different Flavors of Import
  • Export
  • HIVE Imports

 

Introduction of Zookeeper
  • Features of Zookeeper
  • Use of ZooKeeper
  • Zookeeper Data Model

 

Prerequisites: –
  • Can access computer system and make use of internet to perform search over Google.

 

Need to Prepare?
  • Prior programming experience is not required.

 

Tools Expected: –
  • Windows OS based Laptop, Smart phone with Internet, Notebook and Pen

 

Tools Provided (for the session): –
  • Software tools, Sample Codes, PPTs and Reference Material

 

Fees:

  • An amount of INR 2500 will be charged per participants for the NWTP.