Apache Spark and Scala Certification Training
Intellipaat
Course Summary
Intellipaat Spark training let you master real time data processing using spark streaming, spark SQL, Spark RDD, Spark machine learning libraries (spark MLlib). You will learn Apache Spark, Scala programming as well as work on 3 real life use cases in this spark scala course.
-
+
Course Description
About Scala Training Course
What you will learn in this Spark Online Training?
- Understand what is Apache Spark and Scala programming
- Understand the difference between Apache Spark and Hadoop
- Learn Scala and its programming implementation
- Implement Spark on a cluster
- Write Spark Applications using Python, Java and Scala
- Understand RDD and its operation along with implementation of Spark Algorithms
- Define and explain Spark Streaming
- Learn about the Scala classes concept and execute pattern matching
- Learn Scala Java Interoperability and other Scala operations
- Work on Projects using Scala to run on Spark applications
Who should take this Spark and Scala Certification course?
- Software Engineers looking to upgrade Big Data skills
- Data Engineers and ETL Developers
- Data Scientists and Analytics Professionals
- Graduates looking to make a career in Big Data
What are the Prerequisites for this course?
There are no prerequisites for taking up this course. Basic knowledge of database, SQL and query language can help.Why take Apache Spark and Scala training course?
- Apache Spark is an open source computing framework up to 100 times faster than Mapreduce
- Spark is alternative form of data processing unique in batch processing and streaming
- This is a comprehensive course for advanced implementation of Scala
- Prepare yourself for cloudera Hadoop Developer and Spark Professional Certification
- Get professional credibility to your resume so you get hired faster with high salary
-
+
Course Syllabus
Scala Course Content
Introduction of ScalaIntroducing Scala and deployment of Scala for Big Data applications and Apache Spark analytics.Pattern MatchingThe importance of Scala, the concept of REPL (Read Evaluate Print Loop), deep dive into Scala pattern matching, type interface, higher order function, currying, traits, application space and Scala for data analysis.Executing the Scala codeLearning about the Scala Interpreter, static object timer in Scala, testing String equality in Scala, Implicit classes in Scala, the concept of currying in Scala, various classes in Scala.Classes concept in ScalaLearning about the Classes concept, understanding the constructor overloading, the various abstract classes, the hierarchy types in Scala, the concept of object equality, the val and var methods in Scala.Case classes and pattern matchingUnderstanding Sealed traits, wild, constructor, tuple, variable pattern, and constant pattern.Concepts of traits with exampleUnderstanding traits in Scala, the advantages of traits, linearization of traits, the Java equivalent and avoiding of boilerplate code.Scala java InteroperabilityImplementation of traits in Scala and Java, handling of multiple traits extending.Scala collectionsIntroduction to Scala collections, classification of collections, the difference between Iterator, and Iterable in Scala, example of list sequence in Scala.Mutable collections vs. Immutable collectionsThe two types of collections in Scala, Mutable and Immutable collections, understanding lists and arrays in Scala, the list buffer and array buffer, Queue in Scala, double-ended queue Deque, Stacks, Sets, Maps, Tuples in Scala.Use Case bobsrockets packageIntroduction to Scala packages and imports, the selective imports, the Scala test classes, introduction to JUnit test class, JUnit interface via JUnit 3 suite for Scala test, packaging of Scala applications in Directory Structure, example of Spark Split and Spark Scala.Spark Course Content
Introduction to SparkIntroduction to Spark, how Spark overcomes the drawbacks of working MapReduce, understanding in-memory MapReduce,interactive operations on MapReduce, Spark stack, fine vs. coarse grained update, Spark stack,Spark Hadoop YARN, HDFS Revision, YARN Revision, the overview of Spark and how it is better Hadoop, deploying Spark without Hadoop,Spark history server, Cloudera distribution.Spark BasicsSpark installation guide,Spark configuration, memory management, executor memory vs. driver memory, working with Spark Shell, the concept of Resilient Distributed Datasets (RDD), learning to do functional programming in Spark, the architecture of Spark.Working with RDDs in SparkSpark RDD, creating RDDs, RDD partitioning, operations & transformation in RDD,Deep dive into Spark RDDs, the RDD general operations, a read-only partitioned collection of records, using the concept of RDD for faster and efficient data processing,RDD action for Collect, Count, Collectsmap, Saveastextfiles, pair RDD functions.Aggregating Data with Pair RDDsUnderstanding the concept of Key-Value pair in RDDs, learning how Spark makes MapReduce operations faster, various operations of RDD,MapReduce interactive operations, fine & coarse grained update, Spark stack.Writing and Deploying Spark ApplicationsComparing the Spark applications with Spark Shell, creating a Spark application using Scala or Java, deploying a Spark application,Scala built application,creation of mutable list, set & set operations, list, tuple, concatenating list, creating application using SBT,deploying application using Maven,the web user interface of Spark application, a real world example of Spark and configuring of Spark.Parallel ProcessingLearning about Spark parallel processing, deploying on a cluster, introduction to Spark partitions, file-based partitioning of RDDs, understanding of HDFS and data locality, mastering the technique of parallel operations,comparing repartition & coalesce, RDD actions.Spark RDD PersistenceThe execution flow in Spark, Understanding the RDD persistence overview,Spark execution flow & Spark terminology, distribution shared memory vs. RDD, RDD limitations, Spark shell arguments,distributed persistence, RDD lineage,Key/Value pair for sorting implicit conversion like CountByKey, ReduceByKey, SortByKey, AggregataeByKeySpark Streaming & MlibSpark Streaming Architecture, Writing streaming programcoding, processing of spark stream,processing Spark Discretized Stream (DStream), the context of Spark Streaming, streaming transformation, Flume Spark streaming, request count and Dstream, multi batch operation, sliding window operations and advanced data sources. Different Algorithms, the concept of iterative algorithm in Spark, analyzing with Spark graph processing, introduction to K-Means and machine learning, various variables in Spark like shared variables, broadcast variables, learning about accumulators.Improving Spark PerformanceIntroduction to various variables in Spark like shared variables, broadcast variables, learning about accumulators, the common performance issues and troubleshooting the performance problems.Spark SQL and Data FramesLearning about Spark SQL, the context of SQL in Spark for providing structured data processing, JSON support in Spark SQL, working with XML data, parquet files, creating HiveContext, writing Data Frame to Hive, reading JDBC files, understanding the Data Frames in Spark, creating Data Frames, manual inferring of schema, working with CSV files, reading JDBC tables, Data Frame to JDBC, user defined functions in Spark SQL, shared variable and accumulators, learning to query and transform data in Data Frames, how Data Frame provides the benefit of both Spark RDD and Spark SQL, deploying Hive on Spark as the execution engine.Scheduling/ PartitioningLearning about the scheduling and partitioning in Spark,hash partition, range partition, scheduling within and around applications, static partitioning, dynamic sharing, fair scheduling,Map partition with index, the Zip, GroupByKey, Spark master high availability, standby Masters with Zookeeper, Single Node Recovery With Local File System, High Order Functions.Apache Spark – Scala ProjectProject 1: Movie RecommendationTopics – This is a project wherein you will gain hands-on experience in deploying Apache Spark for movie recommendation. You will be introduced to the Spark Machine Learning Library, a guide to MLlib algorithms and coding which is a machine learning library. Understand how to deploy collaborative filtering, clustering, regression, and dimensionality reduction in MLlib. Upon completion of the project you will gain experience in working with streaming data, sampling, testing and statistics.Project 2: Twitter API Integration for tweet AnalysisTopics – With this project you will learn to integrate Twitter API for analyzing tweets. You will write codes on the server side using any of the scripting languages like PHP, Ruby or Python, for requesting the Twitter API and get the results in JSON format. You will then read the results and perform various operations like aggregation, filtering and parsing as per the need to come up with tweet analysis.Project 3: Data Exploration Using Spark SQL – Wikipedia data setTopics – This project lets you work with Spark SQL. You will gain experience in working with Spark SQL for combining it with ETL applications, real time analysis of data, performing batch analysis, deploying machine learning, creating visualizations and processing of graphs.
This course is listed under
Open Source
, Development & Implementations
, Industry Specific Applications
, Data & Information Management
and Server & Storage Management
Community
Related Posts: