Apache Spark Installation On Ubuntu

Install Java

sudo add-apt-repository ppa:webupd8team/java -y

sudo apt-get update

sudo apt-get install oracle-java8-installer

Check Java version

java -version

Create Directory

mkdir work

Change Directory

cd work

Download spark tar file from Apache mirrors.

wget http://redrockdigimark.com/apachemirror/spark/spark-2.1.0/spark-2.1.0-bin-hadoop2.7.tgz

Unzip  tar file

tar -xzvf spark-2.1.0-bin-hadoop2.7.tgz

Rename directory

mv spark-2.1.0-bin-hadoop2.7 spark

Set the path

vi ~/.bashrc
export SPARK_HOME=/home/ubuntu/work/spark
export PATH=$PATH:$SPARK_HOME/bin

Compile the changes

source ~/.bashrc

Start pyspark(Spark with python)

pyspark

Start spark-shell (Spark with Scala)

spark-shell

 

 

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s