DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

VII SEM – CSE

CS8711 CLOUD COMPUTING LABORATORY

S.NO NAME OF EXPERIMENTS

1. Install Virtualbox/VMware Workstation with different flavours of linux or windows
OS on top of windows7 or8.
2. Install a C compiler in the virtual machine created using virtual box and
execute Simple Programs
3.
Install Google App Engine. Create hello world app and other simple web
applicationsusing python/java
4. Use GAE launcher to launch the webapplications.
5. Simulate a cloud scenario using CloudSim and run a scheduling algorithm that is not
presentin CloudSim.
6. Find a procedure to tansfer the files from one virtual machine to another
virtualmachine.

7. Find a procedure to launch virtual machine using trystack (Online Openstack

DemoVersion)
8.
Install Hadoop single node cluster and run simple applications likewordcount.

Page 2 of 71
 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
VII SEM - CSE

CS8711 CLOUDCOMPUTINGLABORATORYLABORATORY

CYCLE I

1. Install Virtualbox/VMware Workstation with different flavours of linux or windows OS on top

of windows7 or8.
2. Install a C compiler in the virtual machine created using virtual box and execute Simple
Programs
3. Install Google App Engine. Create hello world app and other simple web applicationsusing
python/java.
4. Use GAE launcher to launch the webapplications.

CYCLE II

5. Simulate a cloud scenario using CloudSim and run a scheduling algorithm that is not presentin
CloudSim.
6. Find a procedure to transfer the files from one virtual machine to another virtualmachine.
7. Find a procedure to launch virtual machine using trystack (Online Openstack DemoVersion)
8. Install Hadoop single node cluster and run simple applications likewordcount.

Page 3 of 71
 EX.No:1 Install Virtualbox/VMwareWorkstation Aim:
Find procedure to Install Virtualbox/VMware Workstation with different flavours of linux or
windows OS on top of windows7 or 8.

PROCEDURE TO INSTALL
Step 1- Download Link
Link for downloading the software is https://www.vmware.com/products/workstation-pro/workstation-
pro-evaluation.html. Download the software for windows. Good thing is that there is no signupprocess.
Click and download begins. Software is around 541MB.

Step 2- Download the installer file

It should probably be in the download folder by default, if you have not changed the settings in your browser. File
name should be something like VMware-workstation-full-15.5.1-15018445.exe. This file name can change
depending on the version of

the software currently available for download. But for now, till the next version is available, they will
all be VMware Workstation 15 Pro.

Step 3- Locate the downloaded installer file

For demonstration purpose, I have placed the downloaded installer on my desktop. Find the installer on
your system and double click to launch the application.

Page 4 of 71
VMware workstation 15 pro for windows 10 installer file screenshot.

Step 4- User Access Control (UAC) Warning

Now you should see User Access Control (UAC) dialog box. Click yes to continue.

Initial Splash screen will appear. Wait for the process to complete.
VMware Workstation 12 Pro installer windows 10 UAC screenshot

Page 5 of 71
VMware Workstation 15 Installation Splash Screen

Step 5- VMware Workstation Setup wizard

Now you will see VMware Workstation setup wizard dialog box. Click next to continue.

Page 6 of 71
VMware Workstation 15 Installation – Setup Wizard

Step 6- End User Licence Agreement

This time you should see End User Licence Agreement dialog box. Check “I accept the terms in the
Licence Agreement” box and press next to continue.

Page 7 of 71
VMware Workstation 15 Installation – End User Licence Agreement

Step 7- Custom Setup options

Select the folder in which you would like to install the application. There is no harm in leaving the
defaults as it is. Also select Enhanced Keyboard Driver check box.

Page 8 of 71
VMware Workstation 15 Pro installation – select installation folder

Page 9 of 71
Step 8- User Experience Settings
Next you are asked to select “Check for Updates” and “Help improve VMware Workstation Pro”. Do as
you wish. I normally leave it to defaults that is unchecked.

VMware Workstation 15 Installation – User Experience Settings

Page 10 of 71
Step 9- Application Shortcuts preference
Next step is to select the place you want the shortcut icons to be placed on your system to launch the
application. Please select both the options, desktop and start menu and click next.

VMware workstation 15 pro installation shortcut selection checkbox screenshot.

Page 11 of 71
Step 10- Installation begins
Now you see the begin installation dialog box. Click install to start the installation process.

Screenshot for VMware Workstation 15 pro installation begin confirmation dialog box on windows 10.

Below screenshot shows Installation in progress. Wait for this to complete.

Page 12 of 71
Screenshot for VMware Workstation 15 pro installation process.
At the end you will see installation complete dialog box. Click finish and you are done with the
installation process. You may be asked to restart your computer. Click on Yes to restart.

Page 13 of 71
Step 11- Launch VMware Workstation
After the installation completes, you should see VMware Workstation icon on the desktop. Double click
on it to launch the application.

Screenshot for VMware Workstation 15 Pro icon on windows 10 desktop.

Step 12- Licence Key
If you see the dialog box asking for licence key, click on trial or enter the licence key. Then what you
have is the VMware Workstation 15 Pro running on your windows 10 desktop. If don’t have thelicence
key, you will have 30 daystrial.

Page 14 of 71
 VMware Workstation 15 Pro home screen

Step 13- At some point if you decide to buy

At some point of time if you decide to buy the Licence key, you can enter the Licence key by going
toHelp->EnteraLicenceKey You can enter the 25 character licence key in the dialog box shown below
and click OK. Now you have the licence version of thesoftware.

Result:
Thus the VMware workstation is created on windows7.

Page 15 of 71
Exno 2.a: Setting up C Programming Environment
Aim:
To Setup the virtual machine and install the c compilerin virtual machine.

1. A.Install VirtualBox

I.Visithttp://www.virtualbox.org/wiki/downloads

2. Download VirtualBox platform packages for yourOS

3. Open the Installation Package by double clicking

Page 16 of 71
Page 17 of 71
4. Click continue and finish installingVirtualBox

PC

5. When finished installation, close thewindow.

2. Download Linux
 Visit the pagehttp://www.ubuntu.com/download/ubuntu/download
 Choose the Latest version of Ubuntu and32-bit and click “StartDownload”

CLI
CK

Page 18 of 71
I. Run VirtualBox by double-clicking theicon

2. Click “New” button on the top leftcorner

PC

3. Click “Continue” on the pop-upwindow

4. TypeVMname,select“Linux”fortheOSandchoose “Ubuntu” for theversion.

Page 19 of 71
5. Choose the amount of memory to allocate (I suggest choosing between 5I2 MB to I024MB)

6. Click Continue orNext

PC

7. Choose create a new virtual harddisk

8. Click Continue orNext

PC

Page 20 of 71
9. Choose VDI (VirtualBox DiskImage) I0. Click Continue orNext

PC

II. Choose “DynamicallyAllocated” click continue.

This way, the size of yourVirtual Hard Disk will grow as you use.

PC

Page 21 of 71
I2. Click the folder icon and choose the ubuntu iso file you downloaded.

I3. Select the size of the Virtual Disk (I recommend choosing 8 GB) and click continue

PC

I4. ClickCreate

Page 22 of 71
Exno.2.a : C Programming on Linux
I.OpenTerminal(Applications-Accessories-Terminal)

2. Open gedit by typing “gedit &” onterminal

(You can also use any other Text Editor application)

Page 23 of 71
3. type program as

#include<stdio.h>
Void main()
{ printf(“helloworld”);
}

Save it as helloworld.c
Type “ls” on Terminal to see all files under currentfolder
4. Confirm that “helloworld.c” is in the currentdirectory. Ifnot,type cd DIRECTORY_PATH to go to the directory that has
“helloworld.c”
5. Type “gcc helloworld.c” to compile, and type “ls” to confirm that a new executable file “a.out” iscreated

6. Type “./a.out” on Terminal to run theprogram

Page 24 of 71
 7. If you see “HelloWorld” on the next line,
you just successfully ran your first C program!

8. Try other codes from “A Shotgun Introduction to C” on professor Edwards’s webpage.You can also find many C
programing guides online. (jusgoogleit!)

Result:
Thus the virtual machine is created and the c program was executed.

Page 25 of 71
 Ex no 3: Installing and Running the Google App Engine on Windows

Aim:

To perform the installation of the Google App Engine Software Development Kit
(SDK) on a Microsoft Windows and running a simple “hello world” application.

Procedure:

 The App Engine SDK allows you to run Google App Engine Applications on your
local computer. It simulates the run--‐time environment of the Google App Engine
infrastructure.

Step1: To install python

Pre--Requisites: Python 2.5.4

If you don't already have Python 2.5.4 installed in your computer, download
and Install Python 2.5.4 from:

http://www.python.org/download/releases/2.5.4/

Step 2: to install Google App Engine

Download and Install
You can download the Google App Engine SDK by going to:
http://code.google.com/appengine/downloads.html and download the
appropriate installpackage.

Page 26 of 71
Download the Windows installer – the simplest thing is to download it to
your Desktop or another folder that youremember.

Double Click on the GoogleApplicationEngine installer.

Page 27 of 71
 Click through the installation wizard, and it should install the App Engine. If you
do not have Python 2.5, it will install Python 2.5 as well.

Once the install is complete you can discard the downloaded installer
After installation Google app engine looks that,

Step 3: Making of the FirstApplication

Now we need to create a simple application. We could use the “+” option to have
the launcher make us an application – but instead we will do it by hand to get a
better sense of what is goingon.

Make a folder for your Google App Engine applications. I am going to make the
Folder on my Desktop called “apps” – the path to this folder is:
C:\Documents and Settings\csev\Desktop\apps

Page 28 of 71
 Andthenmakeasub--‐folderinwithinappscalled“ae--01--trivial”–thepathtothis folder
would be:
C:\ Documents and Settings \csev\Desktop\apps\ae--01--trivial
Using a text editor such as JEdit (www.jedit.org), create a file called app.yaml in the
ae--01--trivial folder with the following contents:
application: ae-01-
trivial version: 1
runtime: python
api_version: 1
handlers:
- url: /.*
script: index.py

Then create a file in the ae--01--trivial folder called index.py with three lines in it:

print 'Content-
Type:text/plain' print ' '
print 'Hello there Chuck'

Step 4: Run the program

Page 29 of 71
Output:

Once you have selected your application and press Run. After a few moments your
application will start and the launcher will show a little green icon next to your
application. Then press Browse to open a browser pointing at your application which is
running at http://localhost:8080/
Page 30 of 71
 Paste http://localhost:8080 into your browser and you should see
your application as follows:

Result:

Thus the python application program was executed.

Ex no 4: Use GAE launcher to launch the web applications on Windows

Aim:
Page 31 of 71
 To deploy the GAE launcher on a Microsoft Windows and running a simple
“hello world” application.

Procedure:

Step 1. Download the basic housekeeping stuff

No matter what platform you build products on, there is always some housekeeping stuff you need to put in
place before you can hit the ground running. And deploying apps within the Google App Engine is no
exception.

1. Download Python 2.7

As of when this article was written, the Google App Engine standard environment supports Python
only upto version 2.7. However, it is only a matter of time before support for Python 3.x is added.
You can check the App Engine docs for the latest info.
2. Download Google Cloud SDK
This will allow you to fork apps onto your local machine, make changes (edit and develop the app),
and deploy your app back to the cloud.
3. Set the Python path in the Google App Engine launcher
After downloading the SDK, launch the App Engine launcher, go to Edit -> Preferences and make
sure you set the path for where you installed Python in step 1 above.

Page 32 of 71
Set the Python path in Google App Engine launcher

That’s all you need. Your local machine should now be ready to build webapps.

Step 2. App Engine sign-up

This is often the most confusing part of the entire setup. Things you should know when you sign-up:

1. Currently, App Engine offers a free trial for one year.

2. The trial includes $300 of credit that can be used during the one year trial period.
3. You will need to add a credit card to sign-up (for verification purposes).
4. You will not be charged during the sign-up process.
5. You will not be charged during the trial period as long as you do not cross the credit limit offered.

Here are the steps you need to follow to sign-up:

1. Go to the Google Cloud landing page

2. Follow the sign-up process and go to your App Engine dashboard

Page 33 of 71
Most of the hard work is complete after a successful sign-up.

Step 3. Create a new project

The next step is to create a new Python project that you can work on. Follow the screenshots below to create
a new project.

Launch the new project wizard.

Image courtesy.

https://console.cloud.google.com/home

Image courtesy https://console.cloud.google.com/home

Give your app a name and make a note of your project ID.

Page 34 of 71
 Image courtesy.
https://console.cloud.google.com/home

Hit the create button and Google should take a few minutes to set up all that is necessary for your newly
created app.

Step 4. Fork the app to develop it locally

The next step in the process is to fork the app on your local machine. This will allow you to make changes to
the app locally and deploy it whenever you wish to.

Go to Google App Engine launcher and create a new application.

Page 35 of 71
Enter the project ID of your newly created app. Also, provide the folder (local destination) where you wish to
store the app locally. Make sure you select the Python 2.7 as your runtime engine.

Hit the create button, and you should see your app listed on the window that follows. You should also check
that you now see some files in your local storage (the directory you chose in the screenshot above) after this
step.

Page 36 of 71
Step 5. Run the app locally

Before you go ahead and make some changes to the app, it is important to check whether or not you have
executed all the above steps correctly. This can be done by simply running the app locally.

Select the app and hit the run button on the window.

Wait for a few seconds until you can hit the Browse button. Once the Browse button becomes clickable,
click it. This should take you to the browser, and you should see the hello world text appear in your browser
window. Alternatively, you can manually go to the browser and use the port specified to access the app.

As long as you see the above screen, you are all set.

Step 6. Understand the app structure

Page 37 of 71
It is finally time to look at the lines of code which are running this webapp. Open your app folder in the text
editor of your choice. I recommend Sublime text or VS Code. However, feel free to choose the one you
prefer.

Here is a description of the various files.

app.yaml

This file is a basic markup file that stores information (some metadata) about the app. It is important to note
the following crucial parts of the file.

1. application
This is the project ID which you should never change. This is the unique identifier for the app
2. url -> script
This is the homepage for the app. In other words, this file will be rendered in your browser when you
launch the app
3. libraries
This is where you can include external libraries to use within the webapp

Page 38 of 71
 app.yaml
file in the webapp folder

main.py program

This is the homepage of the app (as discussed above). Note that the hello world text in the browser window
(step 5) is due to the code you see highlighted below.

Page 39 of 71
 main.py file in the webapp folder

Step 7. Make your changes and deploy the new app

No hello world app is ever complete without the developer changing the hello world text to something else
just to make sure that everything happening behind the scenes is working as it should.

Go ahead and change the text in the above screenshot to something else.

Page 40 of 71
Output:

Save the changes, go to the browser and refresh the page. You should see the page with the text “MEOW”
displayed.

Finally, it is time to deploy your changes to the cloud to make them globally accessible via a URL. Go to the
App Engine launcher, select the app, and hit the Deploy button.

Page 41 of 71
This will ensure your app gets deployed onto Google Cloud. To check whether or not everything worked just
fine, go to the URL below:

https://<yourProjectID>.appspot.com/

You should see the exact same window as above, expect now, it is a URL that is globally accessible.

Result:
Thus the python web application is executed with the google app engine .

Page 42 of 71
Exno 5: CloudSim in Eclipse

Aim:
To Simulate a cloud scenario using CloudSim and run a scheduling algorithm that is
not present in CloudSim.

Procedure:

CloudSim is written in Java. The knowledge you need to use CloudSim is basic Java
programming and some basics about cloud computing. Knowledge of programming
IDEs such as Eclipse or NetBeans is also helpful. It is a library and, hence, CloudSim
does not have to be installed. Normally, you can unpack the downloaded package in
any directory, add it to the Java classpath and it is ready to be used. Please verify
whether Java is available on your system.

To use CloudSim in Eclipse:

1. Download CloudSim installablefilesfrom https://code.google.com/p/cloudsim/downloads/list

and unzip
2. OpenEclipse
3. Create a new Java Project: File ->New
4. Import an unpacked CloudSim project into the new JavaProject
5. The first step is to initialise the CloudSim package by initialising the CloudSim
library,as follows:

CloudSim.init(num_user, calendar, trace_flag)

6. Data centres are the resource providers in CloudSim; hence, creation of data
centres is a second step. To create Datacenter, you need the DatacenterCharacteristics
object that stores the properties of a data centre such as architecture, OS, list of
machines, allocation policy that covers the time or spaceshared, the time zone and
itsprice:

Page 43 of 71
Datacenter datacenter9883 = new Datacenter(name, characteristics,
new VmAllocationPolicySimple(hostList),s

7. The third step is to create abroker:

DatacenterBroker broker = createBroker();

8. The fourth step is to create one virtual machine unique ID of the VM, userId ID of
theVM’s owner, mips, number Of Pes amount of CPUs, amount of RAM, amount of
bandwidth, amount of storage, virtual machine monitor, and cloudletScheduler policy
forcloudlets:

Vm vm = new Vm(vmid, brokerId, mips, pesNumber, ram, bw, size, vmm, new
CloudletSchedulerTimeShared())

9. Submit the VM list to thebroker:

broker.submitVmList(vmlist)

10. Create a cloudlet with length, file size, output size, and utilisationmodel:
Cloudlet cloudlet = new Cloudlet(id, length, pesNumber, fileSize, outputSize, utilizationModel,
utilizationMode

11. Submit the cloudlet list to thebroker:

broker.submitCloudletList(cloudletList)

12. Start
thesimulation:
CloudSim.startSimulation()

Sample Output from the Existing Example:

Starting

Page 44 of 71
CloudSimExample1...
Initialising...
Starting CloudSim version 3.0 Datacenter_0 is starting...

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>null
Broker is
starting...
Entities
started.
: Broker: Cloud Resource List received with
1resource(s) 0.0: Broker: Trying to Create VM #0
inDatacenter_0
: Broker: VM #0 has been created in Datacenter #2, Host#0
0.1: Broker: Sending cloudlet 0 to VM #0
400.1: Broker: Cloudlet 0 received
: Broker: All Cloudlets
executed.Finishing... 400.1: Broker:
Destroying VM#0
Broker is shutting down...
Simulation: No more
future events
CloudInformationService: Notify all CloudSim entities for
shutting down. Datacenter_0 is shutting down...
Broker is shutting
down...
Simulationcomplet
ed.
Simulation completed.

Page 45 of 71
 OUTPUT :
Cloudlet ID STATUS Data center ID VM ID Time Start Time FinishTime
0 SUCCESS 20 400 0.1 400.1
*****Datacenter:Datacenter_0***** Userid Debt
3 35.6

CloudSimExample1 finished!

RESULT:

Thus the scheduling algorithm is created with the CloudSim.

Page 46 of 71
 Exno 6: File Transfer from one virtual machine to another virtual machine

Aim:

To find a procedure to transfer the files from one virtual machine to another virtual machine.

Procedure:

Method1: Creating a Shared Folder in VirtualBox

A shared folder is a folder which makes its files available on both the guest machine and the host
machine at the same time. Creating a shared folder between the guest and the host allows you to easily
manage files which should be present on both machines. The course virtual machines are ready to use shared
folders right away, but if you are using the virtual machine on your personal computer you will need to
specify which folder to use as shared storage.

Shared Folders on SCS Lab Computers using Course VMs:

If you are using a course VM on a lab computer, it is likely that a shared folder has already been setup for
you. On the desktop of your course VM you should notice a folder titled SharedFolders. Inside of this you
will find any folders that have been shared between the course VM and lab computers.
You should see two folders that have already been configured for you: Z_DRIVE and Temp.
Z_DRIVE gives you access to your Windows Account Z:\ drive. This is storage that is persistent to your
SCS account and available as a network drive on the lab computers.
Temp gives you access to the folder found at D:\temp on the lab computer. Files stored in this folder are
local to the machine, meaning that they can be accessed faster, but will delete from the system when you
log out.
Page 47 of 71
 If you are working with data that you will need to use again, use the Z_DRIVE for your shared folder. If you
need faster read/write speed, use the Temp folder, but remember to backup your files or they will be deleted
when you log off the computer.

Shared Folders on Personal Computers:

If you are using your own personal machine, you will need to configure VirtualBox to look in the right place
for your shared files.

First, click on the guest machine you intend to share files with. From there, you can select the
guestSettings and navigate to Shared Folders on the left side menu. To create a new shared folder, either
click the New Folder icon on the right menu or right click the empty list of shared folders and click Add
Shared Folder. From here, there are six options:

 Folder Path: The folder name on the host machine. Click the drop down menu and navigate to the folder
you would like to share.

 Folder Name: This is the name of the folder as it will appear on the guest machine.

 Read-Only: If you check read-only, the guest machine will be unable to write changes to the folder. This is
valuable when you only want to send files to the virtual machine, but do not want to risk having the files
modified by the guest.

 Auto-Mount: When any external storage is connected to a computer it must be mounted in order to be used.
It is recommended that you turn on auto-mounting, unless you are familiar with the process of mounting a
drive yourself.

 Mount Point: Unless you already know about mount points, leave this blank.

 Make Permanent: If you check this, the shared folder will be a permanent machine folder. If it is not
checked, the folder will not be shared after a shutdown.
On the course virtual machines, when you load into the desktop, you should see a folder
labelledSharedFolders. In there you will see any folders that are currently mounted and being shared.
Steps:

1. Select the guest machine you wish to share files with

Page 48 of 71
2. Click Settings > Shared Folders
3. Right-click and select Add Shared Folder and use the following settings:

Folder Path: Click the dropdown arrow, select Other, and navigate to the folder you
would like to share
Folder Name: Anything to identify it on the guest machine
Read-Only: Unchecked (Checked, if you are exclusively pulling files from the host)
Auto-Mount: Checked
Mount Point: Leave blank

4. Click OK

Method 2: Dragging and Dropping Files in VirtualBox

If you only need to transfer a few files quickly, you can simply drag and drop the files in. On the top bar of
the running guest machine, click on Devices > Drag and Drop and make sure that Bidirectional is selected.
This means that you will be able to drag files from the host to the guest and from the guest to the host. Once
bidirectional drag and drop is checked, you should be able to begin dragging and dropping files.
NOTE: Sometimes when dragging files into the course VM, you may not be able to drag into the file
browser directly. If you encounter this issue, you should drag your files onto the Desktop and move the files
around from there. You should see the cursor change when it is ready to drop files.
You can also drag files from the guest machine into the host. To do this, simply open the file browser on the
host to where you would like to drop the files and drag the files from the virtual machine into the file
browser of the host. File transfers should be pretty quick; if the virtual machine seems stuck when
transferring, simply cancel the transfer and try again.

Method 3: Managing Files with NextCloud

On any virtual machine, including VirtualBox, VMWare, or the virtual machines hosted on the SCS
OpenStack, you can access the SCS NextCloud services to move files between multiple machines and
your SCS Windows Account storage. NextCloud offers you all of your SCS storage in one remote location,
similar to how you might use other file hosting services like Dropbox or Google Drive. Before trying to use
NextCloud, you should check that you can access the service by logging in here.
If you can access the NextCloud services, you can browse the various file storage services available to you:

Page 49 of 71
 Linux Home: These are the files from your SCS Linux Account

 Windows Home: These are the files from your SCS Windows Account and your lab Z:\ drive.

 NextCloud: In addition to the other storage accounts provided to you by the SCS, you can also upload up to
20GB of files directly to NextCloud.
With NextCloud, you can upload your files from any machine with an internet connection and download
them onto any other machine with an internet connection. For example, you can move project files off of
your virtual machine, onto the NextCloud storage, and then download them on your personal laptop.

Alternatively, you can upload files from your personal PC onto the NextCloud storage, place it into
theWindows Home folder, and access those files from either the lab Z:\ drive or download them on a virtual
machine like VirtualBox or OpenStack.

Uploading Files to NextCloud from a Lab Computer:

If you would like to upload files from a lab computer, the easiest way to do this is to place the files you
would like to transfer into your Z:\ drive. These files will be automatically backup into your NextCloud
storage under the Windows Home folder. After that, you can move them into the main NextCloud storage or
choose to keep them in your Z:\drive.

Uploading Files to NextCloud from a VM or Other PC:

If you would like to upload files from either a VM or any other computer, you can login to the NextCloud
service using any of the available interfaces, such as the web interface. Press the “+” icon in the top left of
the file browser and select Upload File. From here, you can choose to keep it in the main NextCloud
storage, move it into your Windows Account storage (the Windows Home folder), or into your Linux
Account storage (the Linux Home folder).

Downloading NextCloud Files to a VM or Other PC:

Once your files are uploaded you will be able to download those files onto any machine which can connect
to NextCloud. First, log in to your preferred NextCloud interface (eg. the web interface). N to the folder
which contains the files you would like to download. Once you are in the target folder, click the checkbox
next to each file you would like to download. Above the file listing you should notice the context bar

Page 50 of 71
changing to tell you how many files you have selected and a button labelled Actions. ClickActions >
Download.
If you have selected a single file, it will prompt you to confirm the download. If you have chosen more than
one file, NextCloud will place all of the selected files into a zip archive. Before you can use the files, you
will need to extract them from the archive. Once you have downloaded your file, or extracted your archive,
you are ready to use your files on your machine.

Result:

Thus the file is transfered from one virtual machine to another virtual machine.

Page 51 of 71
 Exno 7:Launch virtual machine using trystack (Online Openstack Demo Version)

Aim:

To find a procedure to launch virtual machine using trystack (Online Openstack Demo Version).

OpenStack is an open-source software cloud computing platform. OpenStack is primarily used for
deploying an infrastructure as a service (IaaS) solution like Amazon Web Service (AWS). In other
words, you can make your own AWS by using OpenStack. If you want to try out OpenStack,
TryStack is the easiest and free way to do it.
In order to try OpenStack in TryStack, you must register yourself by joining TryStack
FacebookGroup. The acceptance of group needs a couple days because it’s approved manually.
After you have been accepted in the TryStack Group, you can log in TryStack.

TryStack.org Homepage
I assume that you already join to the Facebook Group and login to the dashboard.
After youlog in to the TryStack, you will see the Compute Dashboard like:

Page 52 of 71
OpenStack Compute Dashboard

Overview: What we will do?

In this post, I will show you how to run an OpenStack instance. The instance will be accessible
through the internet (have a public IP address). The final topology will like

:
Page 53 of 71
Step 1: Create Network

Network? Yes, the network in here is our own local network. So, your instances will be not
mixed up with the others. You can imagine this as your own LAN (Local Area Network) in the
cloud.
1. Go to Network > Networks and then click CreateNetwork.
2. In Network tab, fill Network Name for example internal and then clickNext.
3. In Subnettab,
1. Fill Network Address with appropriate CIDR, for example 192.168.1.0/24. Use private
network CIDR block as the bestpractice.
2. Select IP Version with appropriate IP version, in this caseIPv4.
3. ClickNext.
4. In Subnet Details tab, fill DNS Name Servers with 8.8.8.8 (Google DNS) and then
clickCreate.

Step 2: Create Instance

Now, we will create an instance. The instance is a virtual machine in the cloud, like AWS EC2.
You need the instance to connect to the network that we just created in the previous step.
1. Go to Compute > Instances and then click LaunchInstance.
2. In Detailstab,
1. Fill Instance Name, for example Ubuntu1.
2. Select Flavor, for examplem1.medium.
3. Fill Instance Count with1.
4. Select Instance Boot Source with Boot fromImage.
5. Select Image Name with Ubuntu 14.04 amd64 (243.7 MB) if you want installUbuntu
14.04 in your virtual machine.
3. In Access & Securitytab,
1. Click [+] button of Key Pair to import key pair. This key pair is a public and private keythat
we will use to connect to the instance from ourmachine.
2. In Import Key Pairdialog,
1. Fill Key Pair Name with your machine name (for exampleEdward-Key).

Page 54 of 71
 2. Fill Public Key with your SSH public key (usually is in ~/.ssh/id_rsa.pub). See
description in Import Key Pair dialog box for more information. If you are using
Windows, you can use Puttygen to generate keypair.
3. Click Import keypair.
3. In Security Groups, mark/checkdefault.
4. In Networkingtab,
1. In Selected Networks, select network that have been created in Step 1, for exampleinternal.
5. ClickLaunch.
6. If you want to create multiple instances, you can repeat step 1-5. I created one more instance
with instance name Ubuntu2.

Step 3: Create Router

I guess you already know what router is. In the step 1, we created our network, but it is isolated.
It doesn’t connect to the internet. To make our network has an internet connection, we need a
router that running as the gateway to the internet.

1. Go to Network > Routers and then click CreateRouter.

2. Fill Router Name for example router1 and then click Createrouter.
3. Click on your router name link, for example router1, Router Detailspage.
4. Click Set Gateway button in upperright:
1. Select External networks withexternal.
2. ThenOK.
5. Click Add Interfacebutton.
1. Select Subnet with the network that you have been created in Step 1.
2. Click Addinterface.
6. Go to Network > Network Topology. You will see the network topology. In the example, there
are two network, i.e. external and internal, those are bridged by a router. Thereare
instances those are joined to internal network.

Step 4: Configure Floating IP Address

Floating IP address is public IP address. It makes your instance is accessible from the internet.
When you launch your instance, the instance will have a private network IP, but no public IP. In
OpenStack, the public IPs is collected in a pool and managed by admin (in our case is TryStack).
Page 55 of 71
 You need to request a public (floating) IP address to be assigned to your instance.
1. Go to Compute >Instance.
2. In one of your instances, click More > Associate FloatingIP.
3. In IP Address, click Plus[+].
4. Select Pool to external and then click AllocateIP.
5. ClickAssociate.
6. Now you will get a public IP, e.g. 8.21.28.120, for yourinstance.

Step 5: Configure Access & Security

OpenStack has a feature like a firewall. It can whitelist/blacklist your in/out connection. It is
called Security Group.
1. Go to Compute > Access & Security and then open Security Groupstab.
2. In default row, click ManageRules.
3. Click Add Rule, choose ALL ICMP rule to enable ping into your instance, and then clickAdd.
4. Click Add Rule, choose HTTP rule to open HTTP port (port 80), and then clickAdd.
5. Click Add Rule, choose SSH rule to open SSH port (port 22), and then clickAdd.
6. You can open other ports by creating newrules.

Step 6: SSH to Your Instance

Now, you can SSH your instances to the floating IP address that you got in the step 4. If you are
using Ubuntu image, the SSH user will be ubuntu.

Result:
Thus the virtual machine is launched using trystack.

Page 56 of 71
 Exno 8a : Install Hadoopsingle node cluster

Aim:
To Install Hadoopand create thesingle node cluster.

Step 1: To download the Java 8 Package. Save this file in your home directory.

Step 2: Extract the Java Tar File.

Command: tar -xvf jdk-8u101-linux-i586.tar.gz

Fig: Hadoop Installation – Extracting Java Files

Step 3: Download the Hadoop 2.7.3 Package.

Command:wget https://archive.apache.org/dist/hadoop/core/hadoop-2.7.3/hadoop-
2.7.3.tar.gz

Fig: Hadoop Installation – Downloading Hadoop

Step 4: Extract the Hadoop tar File.

Command: tar -xvf hadoop-2.7.3.tar.gz

Page 59 of 71
 Fig: Hadoop Installation – Extracting Hadoop Files

Step 5: Add the Hadoop and Java paths in the bash file (.bashrc). Open.
bashrc file. Now, add Hadoop and Java Path as shown below.
Command: vi .bashrc

Fig: Hadoop Installation – Setting Environment Variable

Then, save the bash file and close it.

For applying all these changes to the current Terminal, execute the source command.

Command: source .bashrc

Page 60 of 71
 Fig: Hadoop Installation – Refreshing environment variables

To make sure that Java and Hadoop have been properly installed on your system and can
be accessed through the Terminal, execute the java -version and hadoop version
commands.

Command: java -version

Fig: Hadoop Installation – Checking Java Version

Command: hadoop version

Fig: Hadoop Installation – Checking Hadoop Version

Step 6: Edit the Hadoop Configuration files.

Command: cd hadoop-2.7.3/etc/hadoop/
Command: ls

Page 61 of 71
All the Hadoop configuration files are located in hadoop-2.7.3/etc/hadoop directory as
you can see in the snapshot below:

Fig: Hadoop Installation – Hadoop Configuration Files

Step 7: Open core-site.xml and edit the property mentioned below inside configuration tag:

core-site.xml informs Hadoop daemon where NameNode runs in the cluster. It contains
configuration settings of Hadoop core such as I/O settings that are common to HDFS &
MapReduce.

Command: vi core-site.xml

Page 62 of 71
 Fig: Hadoop Installation – Configuring core-site.xml

1
<?xmlversion="1.0"encoding="UTF-8"?>
2 <?xml-stylesheettype="text/xsl"href="configuration.xsl"?>
3 <configuration>
4 <property>
5 <name>fs.default.name</name>
6 <value>hdfs://localhost:9000</value>
</property>
7 </configuration>
8

Step 8: Edithdfs-site.xmlandeditthepropertymentioned belowinside

configuration tag:

hdfs-site.xml contains configuration settings of HDFS daemons (i.e. NameNode, DataNode,

Secondary NameNode). It also includes the replication factor and block size of HDFS.

Command: vi hdfs-site.xml

Fig: Hadoop Installation – Configuring hdfs-site.xml

Page 63 of 71
 1
<?xmlversion="1.0"encoding="UTF-8"?>
2
<?xml-stylesheettype="text/xsl"href="configuration.xsl"?>
3 <configuration>
4 <property>
5 <name>dfs.replication</name>
6 <value>1</value>
</property>
7 <property>
8 <name>dfs.permission</name>
9 <value>false</value>
10 </property>
</configuration>
11 9: Edit the mapred-site.xml file and edit the property mentionedbelow
Step
12
inside configuration tag:

mapred-site.xml contains configuration settings of MapReduce application like number of

JVM that can run in parallel, the size of the mapper and the reducer process, CPU cores
available for a process,etc.
In some cases, mapred-site.xml file is not available. So, we have to create the mapred-
site.xml file using mapred-site.xml template.
Command: cp mapred-site.xml.template mapred-site.xml

Command: vi mapred-site.xml.

Fig: Hadoop Installation – Configuring mapred-site.xml

Page 64 of 71
 1
2 <?xmlversion="1.0"encoding="UTF-8"?>
<?xml-stylesheettype="text/xsl"href="configuration.xsl"?>
3 <configuration>
4 <property>
5 <name>mapreduce.framework.name</name>
6 <value>yarn</value>
7 </property>
</configuration>
8

Step 10: Edit yarn-site.xml and edit the property mentioned below inside configuration tag:

yarn-site.xml contains configuration settings of ResourceManager and NodeManager like

application memory management size, the operation needed on program & algorithm, etc.

Command: vi yarn-site.xml

Fig: Hadoop Installation – Configuring yarn-site.xml

Page 65 of 71
 1
2
<?xmlversion="1.0">
3 <configuration>
4 <property>
5 <name>yarn.nodemanager.aux-services</name>
6 <value>mapreduce_shuffle</value>
</property>
7 <property>
8 <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</
9 name>
1 <value>org.apache.hadoop.mapred.ShuffleHandler</value>
0 </property>
</configuration>
1
1

Step 11: Edit hadoop-env.sh and add the Java Path as mentioned below:

hadoop-env.sh contains the environment variables that are used in the script to run
Hadoop like Java home path, etc.

Command: vi hadoop–env.sh

Fig: Hadoop Installation – Configuring hadoop-env.sh

Page 66 of 71
 Step 12: Go to Hadoop home directory and format the NameNode.
Command: cd

Command: cd hadoop-2.7.3
Command: bin/hadoop namenode -format

Fig: Hadoop Installation – Formatting NameNode

This formats the HDFS via NameNode. This command is only executed for the first
time. Formatting the file system means initializing the directory specified by the
dfs.name.dir variable.

Never format, up and running Hadoop filesystem. You will lose all your data stored in
theHDFS.

Step 13: Once the NameNode is formatted, go to hadoop-2.7.3/sbin directory and start all the
daemons.

Command: cd hadoop-2.7.3/sbin

Either you can start all daemons with a single command or do it individually.

Command: ./start-all.sh

The above command is a combination of start-dfs.sh, start-yarn.sh &mr-jobhistory-

daemon.sh
Or you can run all the services individually as below:

Page 67 of 71
 Start NameNode:
The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all
files stored in the HDFS and tracks all the file stored across the cluster.
Command: ./hadoop-daemon.sh start namenode

StartDataNode:
– Starting NameNode

On startup, a DataNode connects to the Namenode and it responds to the requests from the Namenode

for differentoperations.

Command: ./hadoop-daemon.sh start datanode

Fig: Hadoop Installation – Starting DataNode

Start ResourceManager:
ResourceManager is the master that arbitrates all the available cluster resources and thus
helps in managing the distributed applications running on the YARN system. Its work
is to manage each NodeManagers and the each application’s ApplicationMaster.
Page 68 of 71
 Command: ./yarn-daemon.sh start resourcemanager

Fig: Hadoop Installation – Starting ResourceManager

Start NodeManager:
The NodeManager in each machine framework is the agent which is responsible for
managing containers, monitoring their resource usage and reporting the same to the
ResourceManager.

Command: ./yarn-daemon.sh start nodemanager

Fig: Hadoop Installation – Starting NodeManager

Start JobHistoryServer:
JobHistoryServer is responsible for servicing all job history related requests from client.
Command: ./mr-jobhistory-daemon.sh start historyserver

Page 69 of 71
Step 14: To check that all the Hadoop services are up and running, run the below command.

Command: jps

Fig: Hadoop Installation – Checking Daemons

Step 15: Now open the Mozilla browser and go

to localhost:50070/dfshealth.html to check the NameNode interface.

Fig: Hadoop Installation – Starting WebUI

 Congratulations, you have successfully installed a single node Hadoop cluster

Result:
Thus the Hadoop single node cluster was created successfully.

Page 70 of 71
Exno8b.Wordcount program to demonstrate the use of Map and Reduce tasks

Aim:
To write a wordcount program to demonstrate the use of Map and Reduce tasks.

Procedure:
step1

hduser@nspublin:/usr/local/hadoop/sbin$ mkdir /home/hduser/wc

step2:Compiling the java file - WordCount.java

hduser@nspublin:/usr/local/hadoop/sbin$ sudo /usr/lib/jvm/java-8-oracle/bin/javac -classpath

/home/hduser/hadoop-core-1.2.1.jar -d /home/hduser/wc /home/hduser/WordCount.java

step3:Creating jar file for wordCount.java:

hduser@nspublin:/usr/local/hadoop/sbin$ jar -cvf /home/hduser/wc.jar -C /home/hduser/wc/ .
added manifest
adding: WordCount$IntSumReducer.class(in = 1739) (out= 739)(deflated 57%)
adding: WordCount$TokenizerMapper.class(in = 1736) (out= 753)(deflated 56%)
adding: WordCount.class(in = 1491) (out= 814)(deflated 45%)

step4.Executing jar file for WordCount.java

hduser@ksrietcsevb:/usr/local/hadoop/sbin$ hadoop jar /home/hduser/wc.jar WordCount /user/input
/user/output
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
16/09/12 10:52:53 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
using builtin-java classes where applicable
Step5: to check the file in Output file
hduser@ksrietcsevb:/usr/local/hadoop/sbin$ hadoop fs -ls /user/output
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
16/09/12 10:56:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
using builtin-java classes where applicable
Found 2 items

Page 71 of 71
-rw-r--r-- 1 hdusersupergroup 0 2016-09-12 10:56 /user/output/_SUCCESS
-rw-r--r-- 1 hdusersupergroup 182 2016-09-12 10:56 /user/output/part-r-00000
Program:
/home/hduser/WordCount.java:

import java.io.IOException; import

java.util.StringTokenizer;
import org.apache.hadoop.conf.Configuration; import
org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable; import
org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job; import
org.apache.hadoop.mapreduce.Mapper; import
org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; public class
WordCount {
public static class TokenizerMapper
extends Mapper<Object, Text, Text, IntWritable>{ private final static
IntWritable one = new IntWritable(1); private Text word = new
Text();
public void map(Object key, Text value, Context context ) throws
IOException, InterruptedException {
StringTokenizeritr = new StringTokenizer(value.toString());
while (itr.hasMoreTokens()) {
word.set(itr.nextToken()); context.write(word,
one); } }
}
public static class IntSumReducer
extends Reducer<Text,IntWritable,Text,IntWritable> { private
IntWritable result = new IntWritable();
public void reduce(Text key, Iterable<IntWritable>values,Context context) throws IOException,

Page 72 of 71
InterruptedException { int sum = 0;
for (IntWritableval : values) { sum += val.get();
} result.set(sum); context.write(key, result); } }
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf, "word count");
job.setJarByClass(WordCount.class);
job.setMapperClass(TokenizerMapper.class);
job.setCombinerClass(IntSumReducer.class);
job.setReducerClass(IntSumReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(IntWritable.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1); }
}
Input:
hai i am in gcc lab
Output
hduser@ksrietcsevb:/usr/local/hadoop/sbin$ hadoop fs -cat /user/output/*
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
16/09/12 10:56:32 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
using builtin-java classes where applicable
am 1
gcc 1
hai 1
i 1
in 1
lab 1
hduser@ksrietcsevb:/usr/local/hadoop/sbin$
Result:
Thus the word count program was executed using hadoop mar reduce function.

Page 73 of 71