Big Data, Data Science Masters Program

Big Data characteristics, understanding Hadoop distributed computing, the Bayesian Law, deploying Storm for real time analytics, the Apache Storm features, comparing Storm with Hadoop, Storm execution, learning about Tuple, Spout, Bolt.

Installing the Apache Storm, various types of run modes of Storm.

Understanding Apache Storm and the data model.

Installation of Apache Kakfa and its configuration.

Understanding of advanced Storm topics like Spouts, Bolts, Stream Groupings, Topology and its Life cycle, learning about Guaranteed Message Processing.

Various Grouping types in Storm, reliable and unreliable messages, Bolt structure and life cycle, understanding Trident topology for failure handling, process, Call Log Analysis Topology for analyzing call logs for calls made from one number to another.

Understanding of Trident Spouts and its different types, the various Trident Spout interface and components, familiarizing with Trident Filter, Aggregator and Functions, a practical and hands-on use case on solving call log problem using Storm Trident.

Various components, classes and interfaces in storm like – Base Rich Bolt Class, i RichBolt Interface, i RichSpout Interface, Base Rich Spout class and the various methodology of working with them.

Understanding Cassandra, its core concepts, its strengths and deployment.

Twitter Boot Stripping, detailed understanding of Boot Stripping, concepts of Storm, Storm Development Environment.

R language for statistical programming, the various features of R, introduction to R Studio, the statistical packages, familiarity with different data types and functions, learning to deploy them in various scenarios, use SQL to apply ‘join’ function, components of R Studio like code editor, visualization and debugging tools, learn about R-bind.

R Functions, code compilation and data in well-defined format called R-Packages, learn about R-Package structure, Package metadata and testing, CRAN (Comprehensive R Archive Network), Vector creation and variables values assignment.

R functionality, Rep Function, generating Repeats, Sorting and generating Factor Levels, Transpose and Stack Function.

Introduction to matrix and vector in R, understanding the various functions like Merge, Strsplit, Matrix manipulation, rowSums, rowMeans, colMeans, colSums, sequencing, repetition, indexing and other functions.

Understanding subscripts in plots in R, how to obtain parts of vectors, using subscripts with arrays, as logical variables, with lists, understanding how to read data from external files.

Generate plot in R, Graphs, Bar Plots, Line Plots, Histogram, components of Pie Chart.

Understanding Analysis of Variance (ANOVA) statistical technique, working with Pie Charts, Histograms, deploying ANOVA with R, one way ANOVA, two way ANOVA.

K-Means Clustering for Cluster & Affinity Analysis, Cluster Algorithm, cohesive subset of items, solving clustering issues, working with large datasets, association rule mining affinity analysis for data mining and analysis and learning co-occurrence relationships.

Introduction to Association Rule Mining, the various concepts of Association Rule Mining, various methods to predict relations between variables in large datasets, the algorithm and rules of Association Rule Mining, understanding single cardinality.

Understanding what is Simple Linear Regression, the various equations of Line, Slope, Y-Intercept Regression Line, deploying analysis using Regression, the least square criterion, interpreting the results, standard error to estimate and measure of variation.

Scatter Plots, Two variable Relationship, Simple Linear Regression analysis, Line of best fit

Deep understanding of the measure of variation, the concept of co-efficient of determination, F-Test, the test statistic with an F-distribution, advanced regression in R, prediction linear regression.

Logistic Regression Mean, Logistic Regression in R.

Advanced logistic regression, understanding how to do prediction using logistic regression, ensuring the model is accurate, understanding sensitivity and specificity, confusion matrix, what is ROC, a graphical plot illustrating binary classifier system, ROC curve in R for determining sensitivity/specificity trade-offs for a binary classifier.

Detailed understanding of ROC, area under ROC Curve, converting the variable, data set partitioning, understanding how to check for multicollinearlity, how two or more variables are highly correlated, building of model, advanced data set partitioning, interpreting of the output, predicting the output, detailed confusion matrix, deploying the Hosmer-Lemeshow test for checking whether the observed event rates match the expected event rates.

Data analysis with R, understanding the WALD test, MC Fadden’s pseudo R-squared, the significance of the area under ROC Curve, Kolmogorov Smirnov Chart which is non-parametric test of one dimensional probability distribution.

Connecting to various databases from the R environment, deploying the ODBC tables for reading the data, visualization of the performance of the algorithm using Confusion Matrix.

Creating an integrated environment for deploying R on Hadoop platform, working with R Hadoop, RMR package and R Hadoop Integrated Programming Environment, R programming for MapReduce jobs and Hadoop execution.

Logistic Regression Case Study

In this case study you will get a detailed understanding of the advertisement spends of a company that will help to drive more sales. You will deploy logistic regression to forecast the future trends, detect patterns, uncover insights and more all through the power of R programming. Due to this the future advertisement spends can be decided and optimized for higher revenues.

Multiple Regression Case Study

You will understand how to compare the miles per gallon (MPG) of a car based on the various parameters. You will deploy multiple regression and note down the MPG for car make, model, speed, load conditions, etc. It includes the model building, model diagnostic, checking the ROC curve, among other things.

Receiver Operating Characteristic (ROC) case study

You will work with various data sets in R, deploy data exploration methodologies, build scalable models, predict the outcome with highest precision, diagnose the model that you have created with various real world data, check the ROC curve and more.

Introduction to Base SAS, Installation of SAS tool, Getting started with SAS, various SAS Windows – Log, Explorer, Output, Search, Editor, etc. working with data sets, overview of SAS Functions, Library Types and programming files

Import/Export Raw Data files, reading and sub setting the data set, various statements like WHERE, SET, Merge

Hands-on Exercise – Import Excel file in workspace, Read data, Export the workspace to save data

Various SAS Operators – Arithmetic, Logical, Comparison, various SAS Functions – NUMERIC, CHARACTER, IS NULL, CONTAINS, LIKE, Input/Put, Date/Time, Conditional Statements (Do While, Do Until, If, Else)

Hands-on Exercise – Apply logical, arithmetic operators and SAS functions to perform operations

Understanding about Input Buffer, PDV (Backend), learning what is Missover

Defining and Using KEEP and DROP statements, apply these statements, Format and Labels in SAS.

Hands-on Exercise – Use KEEP and DROP statements

Understanding Delimiter, dataline rules, DLM, Delimiter DSD, raw data files and execution, list input for standard data.

Hands-on Exercise – Use delimiter rules on raw data files

The various SAS standard Procedures built-in for popular programs – PROC SORT, PROC FREQ, PROC SUMMARY, PROC RANK, PROC EXPORT, PROC DATASET, PROC TRANSPOSE, , PROC CORR etc.

Hands-on Exercise – Use SORT, FREQ, SUMMARY, EXPORT and other procedures

Reading standard and non-standard numeric inputs with Formatted inputs, Column Pointer Controls, Controlling while a record loads, Line pointer control / Absolute line pointer control, Single Trailing , Multiple IN and OUT statements, DATA LINES statement and rules, List Input Method, comparing Single Trailing and Double Trailing.

Hands-on Exercise – Read standard and non-standard numeric inputs with Formatted inputs, Control while a record loads, Control a Line pointer, Write Multiple IN and OUT statements

SAS FORMAT statements – standard and user-written, associating a format with a variable, working with SAS FORMAT, deploying it on PROC Data sets, comparing ATTRIB and FORMAT statements.

Hands-on Exercise – Format a variable, deploy format rule on PROC DATA set, Use ATTRIB statement

Understanding PROC GCHART, various Graphs, Bar Charts – Pie, Bar, 3D, plotting variables with PROC GPLOT.

Hands-on Exercise – Plot graphs using PROC GPLOT Display charts using PROC GCHART

SAS advanced data discovery and visualization, point-and-click analytics capabilities, powerful reporting tools.

Character Functions, Numeric Functions, Converting Variable Type.

Hands-on Exercise – Use Functions in data transformation

Introduction to ODS, Data Optimization, How to generate files (rtf, pdf, html, doc) using SAS

Hands-on Exercise – Optimize data, generate rtf, pdf, html and doc files

Macro Syntax, Macro Variables, Positional Parameters in a Macro, Macro Step

Hands-on Exercise – Write a macro, Use positional parameters

SQL Statements in SAS, SELECT, CASE, JOIN, UNION, Sorting Data

Hands-on Exercise – Create sql query to select and add a condition
Use a CASE in select query

Base SAS web-based interface and ready-to-use programs, advanced data manipulation, storage and retrieval, descriptive statistics.

Hands-on Exercise – Use web UI to do statistical operations

Report Enhancement, Global Statements, User-defined Formats, PROC SORT, ODS Destinations, ODS Listing, PROC FREQ, PROC Means, PROC UNIVARIATE, PROC REPORT, PROC PRINT

Hands-on Exercise – Use PROC SORT to sort the results, List ODS, Find mean using PROC Means, print using PROC PRINT

Introduction to Splunk, Splunk developer roles and responsibilities

Writing Splunk query for search, sharing, saving, scheduling and exporting search resultsHands-on Exercise – Schedule a search, Save a search result, Share and export a search result

Creation of alert, explaining alerts and viewing fired alertsHands-on Exercise – Create an alert, view fired alerts

Introduction to Tags in Splunk, deploying Tags for Splunk search, understanding event types and utility, generating and implementing event types in SearchHands-on Exercise – Deploy tags for Splunk search, generate and implement event types in Search

Using following commands and their functions: addcoltotals, addtotals,top, rare,statsHands-on Exercise – Create reports using following commands and their functions: addcoltotals, addtotals

Explore the available visualizations, create charts and time charts, omit null values and format resultsHands-on Exercise – Create time charts, omit null values and format results

Calculating and analyzing results, value conversion, roundoff and format values, using eval command, conditional statements, filtering calculated search resultsHands-on Exercise – Calculate and analyze results, perform coversion on a data value, roundoff a numbers, use eval command, write conditional statements,apply filters on calculated search results

Learn about data lookups, example, lookup table, defining and configuring automatic lookup, deploying lookup in reports and searchesHands-on Exercise – Define and configure automatic lookup, deploy lookup in reports and searches

Working with raw data for data extraction, transformation, parsing and previewHands-on Exercise – Extract useful data from raw data, perform transformation, parse different values and preview

Introduction to the Splunk 3 tier architecture, understanding the Server settings, control, preferences and licensing, the most important components of Splunk tool, the hardware requirements, conditions for installation of Splunk.

Understanding how to install and configure Splunk, index creation, input configuration in standalone server, the search preferences, installing Splunk in the Linux environment.

Installing Splunk in the Linux environment, the various prerequisites, configuration of Splunk in Linux.

Introduction to the Splunk Distributed Management Console, index clustering, forwarder management and distributed search in Splunk environment, providing the right authentication to users, access control.

Introducing the Splunk app, managing the Splunk app, the various add-ons in Splunk app, deleting and installing apps from SplunkBase, deploying the various app permissions, deploying the Splunk app, apps on forwarder.

Understanding the index time configuration file and search time configuration file.

Learning about the index time and search time configuration files in Splunk, installing the forwarders, configuring the output and inputs.conf, managing the Universal Forwarders.

Deploying the Splunk tool, the Splunk deployment Server, setting up the Splunk deployment environment, deploying the clients grouping in Splunk.

Understanding the Splunk Indexes, the default Splunk Indexes, segregating the Splunk Indexes, learning about Splunk Buckets and Bucket Classification, estimating index storage, creating new index.

Understanding the concept of role inheritance, Splunk authentications, native authentications, LDAP authentications.

Splunk installation, configuration, data inputs, app management, Splunk important concepts, parsing machine-generated data, search indexer and forwarder.

Introduction to Splunk Configuration Files, Universal Forwarder, Forwarder Management, data management, troubleshooting and monitoring.

Converting machine-generated data into operational intelligence, setting up Dashboard, Reports and Charts, integrating Search Head Clustering & Indexer Clustering.

Understanding the input methods, deploying scripted, Windows, network and agentless input types, fine-tuning it all.

Splunk User authentication and Job Role assignment, learning to manage, monitor and optimize Splunk Indexes.

Understanding parsing of machine-generated data, manipulation of raw data, previewing and parsing, data field extraction.

Distributed search concepts, improving search performance, large scale deployment and overcoming execution hurdles, working with Splunk Distributed Management Console for monitoring the entire operation.

The domain of machine learning and its implications to the artificial intelligence sector, the advantages of machine learning over other conventional methodologies.

Introduction to Deep Learning within machine learning, how it differs from all others methods of machine learning, training the system with training data, supervised and unsupervised learning, classification and regression supervised learning, clustering and association unsupervised learning, the algorithms used in these types of learning.

Introduction to TensorFlowopen source software library for designing, building and training Deep Learning models, Python Library behind TensorFlow, Tensor Processing Unit (TPU) programmable AI accelerator by Google.

Mapping the human mind with Deep Neural Networks, the various building block of Artificial Neural Networks, the architecture of DNN, its building blocks, the concept of reinforcement learning in DNN, the various parameters, layers, activation functions and optimization algorithms in DNN.

Introduction to GPUs and how they differ from CPUs, the importance of GPUs in training Deep Learning Networks, the forward pass and backward pass training technique, the GPU constituent with simpler core and concurrent hardware.