Intro to Apache Spark

150 Pages·2015·12.02 MB·English

Intro to Apache Spark
Big Data TechCon

2015-04-26, Boston"
Paco Nathan @pacoid"
download slides:

training.databricks.com/workshop/intro_spark.pdf
Lecture Outline:
•
Login/quick start on Databricks Cloud

•
Pre-flight check: initial Spark coding exercise

•
Spark Deconstructed: RDDs, lazy-eval, and what
happens on a cluster

•
A Brief History: motivations for Spark and its 

context in Big Data

•
Progressive coding exercises: WC, Join, Workflow

•
Spark Essentials: context, driver, transformations,
actions, persistence, etc.

•
Combine SQL, Streaming, Machine Learning, and 

Graph for Unified Pipelines

•
Resources: certification, events, community, etc.
2
Welcome + 

Getting Started
Getting Started: Step 1
Everyone will receive a username/password for one 

of the Databricks Cloud shards. Use your laptop and
browser to login there:

•

https://class01.cloud.databricks.com/
user: [email protected]

pass: [email protected]

We find that cloud-based notebooks are a simple way
to get started using Apache Spark – as the motto
“Making Big Data Simple” states.

Please create and run a variety of notebooks on your
account throughout the tutorial. These accounts will
remain open long enough for you to export your work.
4
Getting Started: Step 2
Open in a browser window, then click on the
navigation menu in the top/left corner:
5
Getting Started: Step 3
The next columns to the right show folders,

and scroll down to click on
databricks_guide
6
Getting Started: Step 4
Scroll to open the notebook, then
01 Quick Start
follow the discussion about using key features:
7
Getting Started: Step 5
See 

/databricks-guide/01 Quick Start
Key Features:

•
Workspace / Folder / Notebook

•
Code Cells, run/edit/move/comment

•
Markdown

•
Results

•
Import/Export
8
Getting Started: Step 6
Click on the Workspace menu and create your 

own folder (pick a name):
9
Getting Started: Step 7
Navigate to
/_SparkCamp/00.pre-flight-check

hover on its drop-down menu, on the right side:
10