Web Based Collaborative Big Data Analytics On Big Data As A Service Platform
Web Based Collaborative Big Data Analytics On Big Data As A Service Platform
Right from the advent of computers, developers have been facing a lot of
problem maintaining and processing data. Varied formats and differing sizes have
made most of the data platform dependent. Frameworks like Hadoop and other
open source technologies have made dealing with big data easy and flexible.
Making cloud and Internet as platform of storage and analytics of data we have
drawn our motivation.
The varied sizes and different types of data have made the processing and
maintenance of the data in a wide space like Internet far from flexible and
accessibility. Persons from various locations find it difficult to access data from
remote locations.
2. LITERATURE SURVEY
2.1 INTRODUCTION
Data is always admired for its omnipresence and its accessibility. Right from the
generation of data, its maintenance has been a cause of issue. All of the data used in
social networking sites is stored and processed securely. One has to be very careful with
data stored or processed in public domains and social networking sites. Data is of various
kinds in existence and has its reach in different industries. Large amount of data requires
sophisticated approach in storing and performance.
As a model that serves on cloud, one of the basic concerns would be the security
of the data being shared. One does not really want to lose their data to an eavesdropper or
a professional hacker. This application being dealt on internet needs to be secure and
reliable such that no author or owner of a confidential information.
Security of the file is under stake when the data is worked on an external platform.
The application carries the paradigm that of a real time big data cloud applications but
falls short in the efficiency of execution.
Yet, today people reply on SNS (Social Network Service) to update their
messages. Sometimes, a few seconds’ old messages is discarded or updated because users
are not interested in any more. They often remove old messages and pay attention to
recent updates. This high velocity data illustrates big data. Lastly, data can be stored in
multiple formats. For example, the data would be stored in a simple text file or stored in
the form of video, SMS, pdf or user defined format. In this case, we should consider how
to process a variety formats of data. This variety of data describes big data.
ECONOMICAL FEASIBILITY
TECHNICAL FEASIBILITY
SOCIAL FEASIBILITY
ECONOMICAL FEASIBILITY
This study is carried out to check the economic impact that the system will have
on the organization. The amount of fund that the company can pour into the research and
development of the system is limited. The expenditures must be justified. Thus the
developed system as well within the budget and this was achieved because most of the
technologies used are freely available. Only the customized products had to be purchased.
TECHNICAL FEASIBILITY
This study is carried out to check the technical feasibility, that is, the technical
requirements of the system. Any system developed must not have a high demand on the
available technical resources. This will lead to high demands on the available technical
resources. This will lead to high demands being placed on the client. The developed
system must have a modest requirement, as only minimal or null changes are required for
implementing this system.
SOCIAL FEASIBILITY
The aspect of study is to check the level of acceptance of the system by the user.
This includes the process of training the user to use the system efficiently.
The user must not feel threatened by the system, instead must accept it as a
necessity. The level of acceptance by the users solely depends on the methods that are
employed to educate the user about the system and to make him familiar with it. His level
of confidence must be raised so that he is also able to make some constructive criticism,
which is welcomed, as he is the final user of the system.
Simple
Architecture neutral
Object oriented
Portable
Distributed
High performance
Interpreted
Multithreaded
Robust
Dynamic
Secure
5
Every Java interpreter, whether it’s a development tool or a Web browser that
can run applets, is an implementation of the Java VM. Java byte codes help make
“write once, run anywhere” possible. You can compile your program into byte
codes on any platform that has a Java compiler. The byte codes can then be run on
any implementation of the Java VM. That means that as long as a computer has a
Java VM, the same program written in the Java programming language can run on
Windows 2000, a Solaris workstation, or on an iMac.
It’s the base for the Java platform and is ported onto various hardware-based
platforms. The Java API is a large collection of ready-made software components that
provide many useful capabilities, such as graphical user interface (GUI) widgets. The
Java API is grouped into libraries of related classes and interfaces; these libraries are
known as
6
packages. The next section, What Can Java Technology Do? Highlights what
functionality some of the packages in the Java API provide.
The following figure depicts a program that’s running on the Java platform. As the figure
shows, the Java API and the virtual machine insulate the program from the hardware.
Native code is code that after you compile it, the compiled code runs on a specific
hardware platform. As a platform-independent environment, the Java platform can be a
bit slower than native code. However, smart compilers, well-tuned interpreters, and just-
in-time byte code compilers can bring performance close to that of native code without
threatening portability.
Through the classes in java.net, Java programs can use TCP or UDP to
communicate over the Internet. The URL, URLConnection, Socket, and ServerSocket
classes all use TCP to communicate over the network. The Datagram Packet, Datagram
Socket, and MulticastSocket classes are for use with UDP. The TCP/IP stack is shorter
than the OSI one:
Fig: TCP/IP Architecture
7
TCP is a connection-oriented protocol; UDP (User Datagram Protocol) is a
connectionless protocol.
IP datagram’s
The IP layer supplies a checksum that includes its own header. The header
includes the source and destination addresses. The IP layer handles routing through an
Internet. It is also responsible for breaking up large datagram into smaller ones for
transmission and reassembling them at the other end.
UDP
UDP is also connectionless and unreliable. What it adds to IP is a checksum for the
contents of the datagram and port numbers. These are used to give a client/server model -
see later.
TCP
TCP supplies logic to give a reliable connection-oriented protocol above IP. It provides a
virtual circuit that two processes can use to communicate.
Internet addresses
In order to use a service, you must be able to find it. The Internet uses an address scheme
for machines so that they can be located. The address is a 32 bit integer which gives the
IP address. This encodes a network ID and more addressing. The network ID falls into
various classes according to the size of the network address.
Network address
Class A uses 8 bits for the network address with 24 bits left over for other addressing.
Class B uses 16 bit network addressing. Class C uses 24 bit network addressing and class
Subnet address
Internally, the UNIX network is divided into sub networks. Building 11 is currently on
one sub network and uses 10-bit addressing, allowing 1024 different hosts.
8
Host address
8 bits are finally used for host addresses within our subnet. This places a limit of 256
machines that can be on the subnet.
Port addresses
A service exists on a host, and is identified by its port. This is a 16 bit number. To send a
message to a server, you send it to the port for that service of the host that it is running
on. This is not location transparency! Certain of these ports are "well known".
Sockets
A socket is created using the call socket. It returns an integer that is like a file descriptor.
In fact, under Windows, this handle can be used with and functions.
Remote Method Invocation (RMI)
The Java Remote Method Invocation (RMI) system allows an object running in
one Java Virtual Machine (JVM) to invoke methods on an object running in another Java
VM. RMI provides for remote communication between programs written in the Java
programming language. RMI can be used to develop distributed applications as has been
demonstrated by this project.
RMI provides the user with facilities to make method calls remotely. RMI is transparent to
the local modules of the application. Thus, there would be no difference between making a
call to a local method and making a remote method call to a method on a remote machine
using RMI. The user is provided with the same interface by the underlying layer.
9
Awt
Swing's configurability is a result of a choice not to use the native host OS's GUI
controls for displaying itself. Swing "paints" its controls programmatically through the
use of Java 2D APIs, rather than calling into a native user interface toolkit. Thus, a Swing
component does not have a corresponding native OS GUI component, and is free to
render itself in any way that is possible with the underlying graphics APIs.
However, at its core every Swing component relies on an AWT container, since (Swing's)
JComponent extends (AWT's) Container.
Swing component paints its rendition on the graphic device in response to a call to
component. paint (), which is defined in (AWT) Container. But unlike AWT components,
which delegated the painting to their OS-native "heavyweight" widget, Swing
components are responsible for their own rendering.
The set is the same across Unix, Windows and the Macintos
The user interface components can be used for applets and for applications
The AWT (Abstract Window Toolkit) has been present in all versions of Java
The AWT objects are built above native code objects, giving a native look-and-
feel
The AWT objects are a least common denominator of all platforms
The Swing objects are in pure Java, and have the same look-and-feel on all
platforms
In JDK 1.2, the Swing objects are part of the Java Foundation Classes
10
The AWT objects will decrease in importance over time Events may need to be
``blocked'' from reaching objects sometime in password entry, keystrokes need to be
caught by the application but not reach the Text object for display
File Dialog
The mode can be LOAD or SAVE - which affects the labeling of the dialog
Major methods are
Lightweight Objects
AWT Hierarchy
11
``Primitive'' objects are directly derived from Component
Layout Manager is used for geometry management
LayoutManager2 fixes some deficiencies in LayoutManager (from JDK 1.1 on)
AWT Event encodes all the event information
ComponentPeer objects are used to give the native implementation, and should be
ignored most of the time
AWT Primitive Components
12
As much as possible, each Swing component has the same methods and behavior
as its corresponding AWT component
Container Components
Container objects are derived from Component. The main ones derived from
that are
Dialog
File Dialog
Frame
Panel
Window
MenuComponents
Menus are implemented by sub classing Menu Component which is derived from
Object
CheckboxMenuItem
Menu
Menu Bar
Menu Item
11
Layout Objects
Layout objects derive from Object and are used for geometry management
BorderLayout
CardLayout
FlowLayout
13
GridLayout
GridBagLayouts
Components
Component Objects
Component has a number of methods inherited by all objects derived from it.
setBackground(Color)
setFont(Font)
boolean isEnabled()
boolean isShowing()
Color getBackground()
Label
It is left-to-right only
Label (2)
label.setVisible(false);
14
if (label.isShowing()) ...
label.setText("Hello");
label.setAlignment(Label.LEFT);
String s = label.getText();
Label
It is left-to-right only
Label (2)
label.setVisible (false);
if (label.isShowing()) ...
label.setText("Hello");
label.setAlignment(Label.LEFT);
String s = label.getText();
* Swing- Swing is a set of classes that provides more powerful and flexible
components that are possible with AWT. In addition to the familiar components, such
as button checkboxes and labels, swing supplies several exciting additions, including
tabbed panes, scroll panes, trees and tables.
Swing buttons and labels can display images instead of, or in addition to text.
User can easily add or change the borders drawn around most swing
components.
15
Users can easily change the behavior or appearance of a swing component by
either invoking methods on it or creating a subclass of it.
Swing components don’t have to be rectangular, buttons, for e.g., can be
round.
Assistive technologies such as screen renders can get information from swing
components.
Swing let’s to specify which look and feel a program’s GUI uses.
Swing components are implemented with no native code and aren’t restricted
to the least common denominator.
Introduction to swing
This introduction to using Swing in Java will walk you through the basics of
Swing. This covers topics of how to create a window, add controls, position the controls,
and handle events of the control.
The main window almost all GUI applications have a main or top-level window.
In Swing, such window is usually instance of JFrame or JWindow. The difference
between those two classes is in simplicity – JWindow is much simpler than JFrame (most
noticeable are visual differences - JWindow does not have a title bar, and does not put a
button in the operating system task bar). So, your applications will almost always start
with a JFrame. Though you can instantiate a JFrame and add components to it, a good
practice is to encapsulate and group the code for a single visual frame in a separate class.
Usually, I subclass the JFrame and initialize all visual elements of that frame in the
constructor. Always pass a title to the parent class constructor – that String will be
displayed in the title bar and on the task bar. Also, remember to always initialize frame
size (by calling set Size (width, height)), or your frame will not be noticeable on the
screen. package com.neuri.handsonswing.ch1;
import javax.swing.JFrame;
public class MainFrame extends JFrame
public MainFrame()
16
super("My title");
setSize(300, 300);
}}
ss}
Now you have created your first frame, and it is time to display it. Main frame is
usually displayed from the main method – but resist the urge to put the main method in
the frame class. Always try to separate the code that deals with visual presentation from
the code that deals with application logic – starting and initializing the application is part
of application logic, not a part of visual presentation. A good practice is to create an
Application class that will contain initialization code.
package com.neuri.handsonswing.ch1;
}
If you run the code now, you will see an empty frame. When you close it,
something not quite obvious will happen (or better said, will not happen). The application
will not end. Remember that the Frame is just a visual part of application, not application
logic – if you do not request application termination when the window closes, your
program will still run in the background (look for it in the process list). To avoid this
problem, add the following line to the MainFrame constructor:
17
Fig: JFrame
window listener and then act on the window closing event by stopping the application.
Since Java2 1.3, you can specify a simple action that will happen when a window is
closed with this shortcut. Other options are HIDE_ON_CLOSE (the default – window is
closed but application still runs) and DO_NOTHING_ON_CLOSE (rather strange option
that ignores a click on the X button in the upper right corner).
Adding Components
Now is the time to add some components to the window. In Swing (and the Swing
predecessor, AWT) all visual objects are subclasses of Component class. The Composite
pattern was applied here to group visual objects into Containers, special components that
can contain other components. Containers can specify the order, size and position of
embedded components (and this can all be automatically calculated, which is one of the
best features of Swing).
JButton is a component class that represents a general purpose button – it can
have a text caption or an icon, and can be pressed to invoke an action. Let’s add the
button to the frame (note: add imports for javax.swing.* and java.awt.* to the MainFrame
source code so that you can use all the components).
When you work with JFrame, you want to put objects into it’s content pane – special
container intended to hold the window contents. Obtain the reference to that container
with the getContentPane() method.
18
content.add(new JButton("Button 1"));
If you try to add more buttons to the frame, most likely only the last one added
will be displayed. That is because the default behavior of JFrame content pane is to
display a single component, resized to cover the entire area.
Grouping Componets
To put more than one component into a place intended for a single component,
group them into a container. JPanel is a general purpose container, that is perfect for
grouping a set of components into a “larger” component. So, let’s put the buttons into a
JPanel:
JPanel panel=new JPanel();
content.add(panel)
19
Fig: Mulltiple buttons widget on JFrame
Layout for a container is defined using the setLayout method (or usually in
the constructor). So, you could change the layout of content pane to FlowLayout and add
several components, to see them all on the screen.
20
The best choice for the window content pane is usually a BorderLayout with a
central content part and a bottom status (or button) part. The top part can contain a
toolbar, optionally.
Now, let’s combine several components and layouts, and introduce a new
component – JTextArea. JTextArea is basically a multiline editor. Initialize the frame
content pane explicitly to BorderLayout, put a new JTextArea into the central part and
move the button panel below.
package com.neuri.handsonswing.ch1;
import java.awt.*;
import javax.swing.*;
public class MainFrame extends JFrame
public MainFrame()
super("My title");
setSize(300,300);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
content.setLayout(new BorderLayout());
JPanel panel = new JPanel(new FlowLayout());
panel.add(new JButton("Button 1"));
21
panel.add(new JButton("Button 2"));
content.add(panel, BorderLayout.SOUTH);
content.add(new JTextArea(), BorderLayout.CENTER);
Scrolling
Now, type a few lines of text into the text area – once you type enough lines (or
press Enter enough times), the cursor will disappear from the screen. Swing is different
from other GUI toolkits, and visual components do not support scrolling unless you tell
them to – but, on the other hand, this way you can put scrollbar around almost anything.
To enable scrolling, just put the component (or a container with other components) into a
JScrollPane – this is a special container that adds scrollbars to it’s content. Change the
last line of the previous class to content.add(new JScrollPane(new JTextArea()),
BorderLayout.CENTER);to see a scrolling text area inside your window.
Fig: Frame with scrolling option
22
Fig:JFrame with drop-down options
panel.add(new JTextField(10));
String options[] = new String[]{ "Option 1","Option 2","Option 2"
}; panel.add(new JComboBox(options)); content.add(panel,
BorderLayout.NORTH);
Now that you have learned how to put components on the screen, you will learn
how to react to user actions with those components. The central mechanism for this is
the Observer pattern, implemented in Swing with event listeners. Components publish
notifications about events, and event listeners receive these notifications.For example, to
execute a code when user presses a button, you should define a listener for the button
pressing event and register it with that button.
23
Fig: JFrame with onClick event Listener
package com.neuri.handsonswing.ch1;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import javax.swing.*;
{
public MainFrame()
super("My title");
setSize(300,300);
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
content.setLayout(new BorderLayout());
JPanel panel = new JPanel(new FlowLayout());
JButton button1 = new JButton("Button 1");
panel.add(button1);
button1.addActionListener( new
MyButtonListener(this)); panel.add(new JButton("Button
2")); panel.add(new JButton("Button 3"));
content.add(panel, BorderLayout.SOUTH);
24
panel = new JPanel(new FlowLayout());
panel.add(new JTextField(10));
MyButtonListener(JFrame parentComponent)
this.parentComponent=parentComponent;
}
}
25
button1.addActionListener( new ActionListener()
JOptionPane.showMessageDialog(MainFrame.this,"BUTTON PRESSED!");
})
For now, just note that there are also other ways of receiving event notification
then installing a listener for every object on the screen. For example, the following few
lines installs a global listener for the F1 key pressing:
KeyStroke ks=KeyStroke.getKeyStroke(KeyEvent.VK_F1,0);
topComponent.getInputMap(JComponent.WHEN_ANCESTOR_OF_FOCUSED_COMP
ONENT).put(ks,"HELP");
topComponent.getActionMap().put("HELP",
new AbstractAction()
{
// do something here, display a dialog or whatever
});
JDBC
26
3. ANALYSIS
3.1 INTRODUCTION
Data owner can view, how many file has been uploaded to the
corresponding cloud servers
1. Usability
The system is designed with completely automated process hence there is
no or less user intervention.
2. Reliability
The system is more reliable because of the qualities that are inherited
from the chosen platform java (JSP). The code built by using java is more reliable.
3. Performance
This system is developing in the high level languages and using the
advanced front-end and back-end technologies it will be responsive and execute in
very less time.
27
4. Supportability
5. Implementation
HARDWARE REQUIREMENTS:
SOFTWARE REQUIREMENTS:
Operating System : Windows95/98/2000/XP
Front End : HTML, JSP
Scripts: JavaScript.
Server side Script :AWT, swings.
Database : MySQL 5.0
DatabaseConnectivity : JDBC
28
5. IMPLEMENTATION
5.1 INTRODUCTION
The most crucial phase of any project is the implementation. This includes
all those activities that take place to convert from the old system to the new system. It
involves setting up of the system for use by the concerned end user. A successful
implementation involves a high level of interaction between the analyst, programmers
and the end user. The most common method of implementation is the phased approach,
which involves installation of the system concurrently with the existing system. This has
its advantage in that the normal activity carried out, as part of the existing system is
anyway hampered. The end users are provided with sufficient documentation and
adequate training in the form of demonstration/presentation in order to familiarize with
the system.
<HTML XMLNS="HTTP://WWW.W3.ORG/1999/XHTML">
<HEAD>
8859-1" />
TYPE="TEXT/CSS" />
<SCRIPT TYPE="TEXT/JAVASCRIPT"
43
SRC="LAYOUT/SCRIPTS/JQUERY.MIN.JS"></SCRIPT>
<SCRIPT TYPE="TEXT/JAVASCRIPT"
SRC="LAYOUT/SCRIPTS/JQUERY.JCAROUSEL.PACK.JS"></SCRIPT>
<SCRIPT TYPE="TEXT/JAVASCRIPT"
SRC="LAYOUT/SCRIPTS/JQUERY.JCAROUSEL.SETUP.JS"></SCRIPT>
</HEAD>
<BODY ID="TOP">
<DIV ID="HEADER">
<DIV ID="LOGO">
</DIV>
<DIV ID="TOPNAV">
<UL>
<LI><A HREF="BDAAS.JSP">BDAAS</A>
</LI>
</UL>
</DIV>
</DIV>
</DIV>
44
HOME PAGE
DATA OWNER LOGIN PAGE
45
DATA OWNER REGISTRATION PAGE
DATA LIST PAGE WITH ALL FILES
46
DATA MODIFICATION PAGE
.DATA MONITORING PAGE
47
DATA SCIENTIST HOME PAGE
DATA SCIENTIST VERIFICATION PAGE
48
PAGE FOR CHECKING ALL VERIFIED FILES
SERVICE DEVELOPER LOGIN PAGE
49
.PAGE WITH LIST OF FILES THAT ARE VERIFIED
.BDAAS HOME PAGE
50
REQUEST VALIDATION PAGE
GRANTED REQUEST PAGE
51
6. TESTING, VALIDATION AND RESULTS
6.1 INTRODUCTION
Software system meets its requirements and user expectations and does not fail in an
unacceptable manner. There are various types of test. Each test type addresses a specific
testing requirement.
Unit testing
Unit testing involves the design of test cases that validate that the internal program
logic is functioning properly, and that program inputs produce valid outputs. All decision
branches and internal code flow should be validated. It is the testing of individual
software units of the application .it is done after the completion of an individual unit
before integration. This is a structural testing, that relies on knowledge of its construction
and is invasive. Unit tests perform basic tests at component level and test a specific
business process, application, and/or system configuration. Unit tests ensure that each
unique path of a business process performs accurately to the documented specifications
and contains clearly defined inputs and expected results.
Integration testing
Integration tests are designed to test integrated software components to determine
if they actually run as one program. Testing is event driven and is more concerned with
the basic outcome of screens or fields. Integration tests demonstrate that although the
components were individually satisfaction, as shown by successfully unit testing, the
combination of components is correct and consistent. Integration testing is specifically
aimed at exposing the problems that arise from the combination of components.
52
Functional test
Valid Input
: identified classes of valid input must be accepted.
Invalid Input
: identified classes of invalid input must be rejected.
Functions
: identified functions must be exercised.
Output
: identified classes of application outputs must be exercised.
System testing ensures that the entire integrated software system meets
requirements. It tests a configuration to ensure known and predictable results. An
example of system testing is the configuration oriented system integration test. System
testing is based on process descriptions and flows, emphasizing pre-driven process
links and integration points.
White Box Testing is a testing in which in which the software tester has
knowledge of the inner workings, structure and language of the software, or at least its
53
purpose. It is purpose. It is used to test areas that cannot be reached from a black box
level.
Black Box Testing is testing the software without any knowledge of the inner
workings, structure or language of the module being tested. Black box tests, as most other
kinds of tests, must be written from a definitive source document, such as specification or
requirements document, such as specification or requirements document. It is a testing in
which the software under test is treated, as a black box .you cannot “see” into it. The test
provides inputs and responds to outputs without considering how the software works.
Unit testing is usually conducted as part of a combined code and unit test phase of
the software lifecycle, although it is not uncommon for coding and unit testing to be
conducted as two distinct phases.
Field testing will be performed manually and functional tests will be written in
detail.
Test objectives
54
Features to be tested
Integration Testing
Test Results: All the test cases mentioned above passed successfully. No defects
encountered.
Acceptance Testing
User Acceptance Testing is a critical phase of any project and requires significant
participation by the end user. It also ensures that the system meets the functional
requirements.
55
7. CONCLUSION
7.1 CONCLUSION
The most crucial phase of any project is the implementation. This includes
all those activities that take place to convert from the old system to the new system. It
involves setting up of the system for use by the concerned end user. A successful
implementation involves a high level of interaction between the analyst, programmers
and the end user. The most common method of implementation is the phased approach,
which involves installation of the system concurrently with the existing system. This has
its advantage in that the normal activity carried out, as part of the existing system is
anyway hampered. The end users are provided with sufficient documentation and
adequate training in the form of demonstration/presentation in order to familiarize with
the system.
Being a prototype it would not promise much but the module can be trained with
algorithms of machine learning such that the application will become good enough to deal
with the type of data it gets from a new user. This way we can expect minimum errors
and exceptions.
56
8. BIBLIOGRAPHY
REFERENCES
3. S. Lohr, “The age of big data, “ New York Times, vol. 11, 2012.
5. Z. Zheng, J. Zhu, and M. R. Lyu, "Service-generated big data and big data-as-A-
service: An overview," pp. 403-410, 2013.
57