the ultimate guide on BIg data Analytics & data science

100 Pages·2017·2.03 MB·English

THE ULTIMATE GUIDE ON
BIG DATA ANALYTICS &
DATA SCIENCE
Table of Contents

What are Big Data Analytics & Data Science? ..................................................................... 5
A. What is Big Data? .................................................................................................................... 6
Big Data Essentials #1- Characteristics ................................................................................... 6
Big Data Essentials #2- Sources ............................................................................................. 12
Big Data Essentials #3- Careers ............................................................................................. 13
Big Data Essentials #4- Importance....................................................................................... 14
Some of the companies using Big Data .................................................................................. 15
B. Big Data Analytics & Data Science – Are they the same? ...................................................... 16
What is Analytics? .................................................................................................................. 16
Analytics way before the Computers‘ age ............................................................................... 17
What does Analytics answer? ................................................................................................. 17
Analysis and Analytics – the same concept? .......................................................................... 17
What is Data Science? ............................................................................................................ 18
Big Data Analytics & Data Science – Is there a winner? ........................................................ 18
Do Big Data Analysts and Data Scientists differ? ................................................................... 18
How do Big Data Analysts and Data Scientists differ? ........................................................... 19
Are there any stark similarities between the two fields? ....................................................... 20
Can you become a Data scientist or an Analyst? ................................................................... 20
Where are Big Data Analytics and Data Science used? .......................................................... 21
C. Introduction to Data Science ................................................................................................ 23
Data- The Modern Age Oil..................................................................................................... 24
What is Data Science? ........................................................................................................... 25
Data Scientist- At a glance ..................................................................................................... 26
What skills are needed to be a Data Scientist? ...................................................................... 27
How can we solve real world problems with Data Science? .................................................. 30
Want to be a Data Scientist? A piece of advice. ...................................................................... 31
Data Science Learning Resources and Communities ............................................................ 32

© 2016 UpX Academy 1
Where are Big Data and Data Science used? ............................................................. 33
I. Banking .................................................................................................................................. 34
The Role of Analytics in Banking .................................................................................................. 34
Prominent use cases for banking analytics ................................................................................... 35
Levels of Analytics ........................................................................................................................ 36
Areas in banking where Analytics has the maximum impact ....................................................... 37
II. Classification of emails – Spam/Ham .................................................................................. 38
The 1990s – Just send it! .............................................................................................................. 38
What does a spam filter do?.......................................................................................................... 38
How does it work?......................................................................................................................... 39
Challenges to spam filters ............................................................................................................. 40
How has Google been addressing these challenges? .................................................................... 40
III. E-commerce ......................................................................................................................... 41
How Predictive Analytics helps Buyers? ....................................................................................... 44
How Predictive Analytics helps Sellers? ....................................................................................... 44
How the giants use Predictive Analytics? ..................................................................................... 44
IV. Retail chains - Starbucks ..................................................................................................... 45
1. Deciding a new Starbucks store location .................................................................................. 45
2. Deciding Starbucks menu offerings .......................................................................................... 46
3. Starbucks loyalty program ........................................................................................................ 47
V. Predicting Election Results ................................................................................................... 49
How data analytics powers 2016 US election ............................................................................... 49
How data analytics affects the election ......................................................................................... 49
How election campaigns make use of Data Analytics .................................................................. 50
Identifying the Swing States .......................................................................................................... 51
Online and offline marketing ........................................................................................................ 52
Big Players in the Election Forecast ............................................................................................. 52
VI. Gaming ................................................................................................................................ 54
VII. Word Cloud, a big thing in a small time! ........................................................................... 59
What is a word cloud? ................................................................................................................... 59
How to create your own word cloud! ............................................................................................ 60

© 2016 UpX Academy 2
Machine Learning ............................................................................................................... 64
A. Introduction to Machine Learning
What is Machine Learning? .......................................................................................................... 64
How does a computer learn? ........................................................................................................ 65
Types of Machine Learning .......................................................................................................... 66
B. The 10 most popular Machine Learning algorithms ............................................................. 68
1. Linear Regression ...................................................................................................................... 68
2. Logistic Regression ................................................................................................................... 69
3. Decision Tree ............................................................................................................................ 70
4. Random Forest ......................................................................................................................... 70
5. Artificial Neural Network ........................................................................................................... 71
6. Support Vector Machine ........................................................................................................... 72
7. K-Means Clustering .................................................................................................................. 73
8. K-Nearest Neighbour ................................................................................................................ 73
9. Naive Bayes classifier ................................................................................................................ 74
10. Ensemble Learning .................................................................................................................. 75
C. Machine Learning in R........................................................................................................... 76
R environment .............................................................................................................................. 76
Basic Functions in R ...................................................................................................................... 77
Get your data and read it! .............................................................................................................. 77
Know your data .............................................................................................................................. 77
Visualisation ............................................................................. Error! Bookmark not defined.
Machine Learning ......................................................................................................................... 78
1. Linear regression ....................................................................................................................... 78
2. Logistic Regression ................................................................................................................... 79
3. K- Nearest Neighbour Classification ........................................................................................ 79
4. Decision Trees ........................................................................................................................... 79
5. K-means Clustering ................................................................................................................... 79
6. Naïve Bayes ............................................................................................................................... 80
7. SVM (Support Vector Machines) .............................................................................................. 80
8. Apriori Algorithm ..................................................................................................................... 80
9. Random Forest .......................................................................................................................... 81

© 2016 UpX Academy 3
D. Comparing Python and R. Which one should you use? ........................................................... 82
#1. The Start .................................................................................................................................. 82
#2. Learning Curve ....................................................................................................................... 82
#3. Libraries .............................................................................. Error! Bookmark not defined.
#4. Community ............................................................................................................................. 83
#5. Speed ....................................................................................................................................... 83
#6. IDEs ........................................................................................................................................ 83
#7. Data Visualization ................................................................................................................... 83
#8. Industry .................................................................................................................................. 83

Data Science Careers ................................................................................................... 85
A. Data Science Roles
Data Analyst .................................................................................................................................. 85
Business Analyst ........................................................................................................................... 85
Data scientist/Statisticians ........................................................................................................... 86
B. What skills are required to be a Data Scientist? .................................................................. 87
Can anyone become a Data Scientist? .......................................................................................... 89
What background do you need to have to be a Data Scientist? .................................................... 89
Are Statistics and programming definite prerequisites? .............................................................. 90
What you should do right away, to become a Data Scientist! ...................................................... 90
C. A Day in a Life of a Data Scientist ......................................................................................... 91
Why has being a Data Scientist become the hottest profession? .................................................. 91
A Data Scientist‘s Day ................................................................................................................... 92
D. How to become a Freelancing Data Scientist? .................................................................... 96
Why be a Freelance Data Scientist? .............................................................................................. 96
How to make your portfolio noticeable? ...................................................................................... 97

© 2016 UpX Academy 4
What are Big Data
Analytics & Data Science?

© 2016 UpX Academy 5
A. What is Big Data?
You must have heard the term ―Big Data‖ a lot. It is indeed gaining a lot of importance these
days. Some are of the opinion that it is the modern age oil! So then, what is Big Data all
about?
Big data refers to the large amount of data generated by web-logs, text, videos, content &
images — mainly created by online activity that demands modern and sophisticated systems
for storage.
Here are 4 Big Data Essentials that you probably didn‘t know!
Big Data Essentials #1- Characteristics
When we talk about Big Data we don‘t necessarily mean the size of the data. Dough Laney
defines Big Data on the basis of 3Vs, viz., Volume, Variety, and Velocity.

© 2016 UpX Academy 6
i. Volume:

The volume of data is increasing daily. As per IBM, 2.5 Exabyte of data is generated every
day. By 2020, the total data will add up to 40,000 Exabyte! A single storage server cannot
store such vast amounts of data. Hence, a need for network of storage devices called SANs
(i.e. Storage Area Networks) arises. Companies find it harder to afford the cost of these
storage servers.

© 2016 UpX Academy 7
ii. Velocity:
Velocity is the speed at which data is generated and the promptness at which it needs to be
processed. Some researchers believe that 90% of the world‘s data was generated in the last
two years alone. Big Data poses a huge challenge for social networking sites. For example,
Facebook needs to store petabytes of data generated by its 1.65 billion active monthly users.
Such streaming data needs to be stored and queries need to be processed in real-time.

© 2016 UpX Academy 8
iii. Variety:
Traditional data types only include structured data, which perfectly fits in the case of an
RDBMS (i.e. Relational Database Management). But most of the data we generate is
unstructured. The digital world has opened up its doors to unstructured data making
RDBMS no longer viable. In fact, Facebook alone generates 30+ petabytes of unstructured
data in the form of web logs, pictures & messages. Almost 80% of the data today is
unstructured and cannot be classified into tables. With the aid of Big Data technologies, it is
now possible to consolidate this data and make sense of it.

However, Big Data has been further classified to include two more Vs i.e. Veracity and
Value.

© 2016 UpX Academy 9