Blood Bank Management System Using Amazon Elastic Cloud Computing Report
Blood Bank Management System Using Amazon Elastic Cloud Computing Report
CHAPTER 1 INTRODUCTION
The term cloud is used as a metaphor for the Internet, based on the cloud drawing used to depict the Internet in computer network diagrams as an abstraction of the underlying infrastructure it represents. Typical cloud computing providers deliver common business applications online which are accessed from a web browser, while the software and data are stored on servers.
Dept Of CSE/BTLIT/BATCH 5
While clouds are currently mainly used for personal and commercial use (e.g. for web applications), their large number of storage and computational resources, high accessibility, reliability and simple cost model make them very attractive for HPC applications as well.
Dept Of CSE/BTLIT/BATCH 5
1.3 Objective
This thesis takes the idea step further and proposes to use the concept of cloud computing (Amazon Web Services) in order to effectively distribute large amount of data stored at Amazon Simple Storage Service (S3) to the all highly computational Elastic Compute Cloud (EC2) nodes before or during the run. In a cloud, these data are typically stored in a separate storage service. Distributing data from the storage service to all compute nodes is essentially a multicast operation. The simplest solution is to let all nodes download the same data directly from the storage service.
S3
E 2 C
E 2 C
E 2 C
Figure 1.1 Basic block diagram shows working of S3 & EC2 Nodes. The data stored at Amazon S3 is distributed to the Amazon EC2 Instances. Multicast Operations is carried out to distribute data from the storage service to all compute nodes.
Dept Of CSE/BTLIT/BATCH 5 3
Blood bank management system using amazon elastic cloud computing The algorithm used here consists of two phases:
(1) The scatter phase and (2) The allgather phase. In the scatter phase, the root node (S3) divides the data to be multicast into blocks of equal size depending on the number of EC2 nodes. These blocks are then sending to each corresponding node using a binomial tree. After all the nodes have receives the divided blocks, they start the allgather phase in which the missing blocks are exchanged and collected by adjacent EC2 nodes.
1.6 Motivation
Many optimization methods and algorithms for multicast operations have been developed for different environments such as high-performance collective communication algorithms for parallel distributed systems (e.g. clusters and grids) & data streaming and file sharing on P2P systems.
Dept Of CSE/BTLIT/BATCH 5 4
Each of these approaches essentially performs a multicast operation, but has to take different assumptions and settings into consideration depending on the target environment. While cloud platforms are similar to parallel distributed systems, they also share some characteristics with P2P systems. First, cloud computing services generally provide virtualized computing environments that are shared with other cloud users. The available bandwidth within a cloud can therefore change dynamically. Moreover, the underlying physical network topology and activity of other users is generally unknown.
Amazon Web Services (AWS) is a collection of remote computing services (also called web services) that together make up a cloud computing platform, offered over the Internet by Amazon.com. The most central and well-known of these services are Amazon EC2 and Amazon S3. Amazon Web Services offerings are accessed over HTTP, using Representational State Transfer (REST) and SOAP protocols. All services are billed on usage. AWS is comprehensive cloud computing platform & is more than a collection of infrastructure services. With pay as you go pricing, you can save time by incorporating compute, database, storage, messaging, payment, and other services that will give you a head start on delivering for your business. All AWS services can be used independently or deployed together to create a complete computing platform in the cloud.
Dept Of CSE/BTLIT/BATCH 5
Standard Instances
Instances of this family are well suited for most applications. Small Instance (Default) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1
EC2 Compute Unit), 160 GB of local instance storage, 32-bit platform. Large Instance 7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2
Compute Units each), 850 GB of local instance storage, 64-bit platform. Extra Large Instance 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2
EC2 Compute Units each), 1690 GB of local instance storage, 64-bit platform.
Micro Instances
Instances of this family provide a small amount of consistent CPU resources and allow you to burst CPU capacity when additional cycles are available. They are well suited for lower throughput applications and web sites that consume significant compute cycles periodically. Micro Instance 613 MB of memory, up to 2 ECUs (for short periodic bursts), EBS
High-Memory Instances
Instances of this family offer large memory sizes for high throughput applications, including database and memory caching applications. High-Memory Extra Large Instance 17.1 GB memory, 6.5 ECU (2 virtual cores with
3.25 EC2 Compute Units each), 420 GB of local instance storage, 64-bit platform. High-Memory Double Extra Large Instance 34.2 GB of memory, 13 EC2 Compute
Units (4 virtual cores with 3.25 EC2 Compute Units each), 850 GB of local instance storage, 64bit platform.
Dept Of CSE/BTLIT/BATCH 5
Units (8 virtual cores with 3.25 EC2 Compute Units each), 1690 GB of local instance storage, 64-bit platform.
High-CPU Instances
Instances of this family have proportionally more CPU resources than memory (RAM) and are well suited for compute-intensive applications. High-CPU Medium Instance 1.7 GB of memory, 5 EC2 Compute Units (2 virtual cores
with 2.5 EC2 Compute Units each), 350 GB of local instance storage, 32-bit platform. High-CPU Extra Large Instance 7 GB of memory, 20 EC2 Compute Units (8 virtual
cores with 2.5 EC2 Compute Units each), 1690 GB of local instance storage, 64-bit platform.
Dept Of CSE/BTLIT/BATCH 5
4. A paper on Optimization of collective communication operations in MPICH, by R. Thakur, R. Rabenseifner, and W. Gropp.
In these papers [3] [4] it is found that the optimization methods for collective operations. In particular, optimization of multicast communication has been researched in message passing systems like MPI and their collective operation algorithms. Target applications in this environment are mainly HPC applications, so these optimization techniques focus on making a multicast operation as fast as possible. 5. A paper on Incentives build robustness in BitTorrent, by B. Cohen. 6. A paper on Splitstream: High-bandwidth multicast in cooperative environments, by M. Castro, P. Druschel, A. Kermarrec, A. Nandi, A. Rowstron, and A. Singh. In these papers [5] [6], It is observed that Optimization of multicast communication is also studied within the context of P2P overlay networks. Examples include file sharing applications like BitTorrent and data streaming protocols like SplitStream. Consequently, the main focus of multicast communication protocols on P2P systems is being robust. They divide the data to multicast into small pieces that are exchanged with a few neighbor nodes. All nodes tell their neighbors which pieces they have and request pieces they lack. 7. A paper on The Cost of Doing Science on the Cloud: The Montage Example, by E. Deelman, G. Singh, M. Livny, B. Berriman, and J. Good. In this paper it was proposed that the cost of running an application on a cloud depends on the compute, storage and communication resources it will provision and consume. Different execution plans of the same application may result in significantly different costs& to evaluate the cost of running data-intensive applications on Amazon EC2/S3 in terms of different execution and resource provisioning plans.
Dept Of CSE/BTLIT/BATCH 5
10
This section will characterize various multicast methods for large amounts of data on parallel distributed systems and P2P networks, & specific issues with optimizing multicast operations on clouds. 2.3.1. Multicast on Parallel Distributed Systems For parallel distributed systems, such as clusters and grids, there are many types of optimization methods for collective operations. In particular, optimization of multicast communication has been researched in message passing systems like MPI and their collective operation algorithms. Target applications in this environment are mainly HPC applications, so these optimization techniques focus on making a multicast operation as fast as possible. The optimization of multicast communication for parallel distributed systems makes several assumptions: 1) Network performance is high and stable. 2) Network topology does not change. 3) Available bandwidth between nodes is symmetric. Based on these assumptions, optimized multicast algorithms generally construct one or more optimized spanning trees by using network topology information and other monitoring data. The data is then forwarded along these spanning trees from the root node to all others. These multicast techniques are therefore sender-driven (i.e. push-based). For large amounts of data, some optimization algorithms try to construct multiple spanning trees that maximize the available bandwidth of nodes. The data is then divided into small pieces that are transferred efficiently to each node by using the pipelining technique. For example, Stable Broadcast uses depth-first search to find multiple spanning pipeline trees based on estimated network topology information of multiple clusters, and maximizes available bandwidth of all nodes by reducing the effect of slow bandwidth nodes. 2.3.2 Overlay multicast on P2P systems Optimization of multicast communication is also studied within the context of P2P overlay networks. Examples include file sharing applications like BitTorrent and data streaming
Dept Of CSE/BTLIT/BATCH 5
11
protocols like Splitstream. The target environment of these applications differs from parallel distributed systems; 1) Network performance is very dynamic. 2) Nodes can join and leave at will. 3) Available bandwidth between nodes can be asymmetric. Consequently; the main focus of multicast communication protocols on P2P systems is being robust. They divide the data Multicast into small pieces that are exchanged with few neighbor nodes. All nodes tell their neighbors which pieces they have and request pieces they lack. Multicast communication in P2P networks is therefore receiver-driven (i.e. pull-based). Receiver-driven multicast can also improve throughput. For example, MOB optimizes multicast communication between multiple homogeneous clusters connected via a WAN. The MOB protocol is based on BitTorrent, and takes node locality into account to reduce the number of wide-area transfers. Nodes in the same cluster are grouped into mobs. Each node in a mob steals an equal part of all data from peers in remote clusters, and distributes the stolen pieces locally. This way, each piece is transferred to each cluster only once, which greatly reduces the amount of wide-area traffic compared to BitTorrent. The incoming data is also automatically spread over all nodes in a mob, which works very well when the NICs of the nodes are the overall bandwidth bottleneck instead of the wide-area links.
2.3.3. Multicast on Clouds Cloud services such as Amazon EC2/S3 provide virtualized application execution environments which are becoming increasingly popular as a platform for HPC applications. For example, Deelman et al. tried to evaluate the cost of running data-intensive applications on Amazon EC2/S3 in terms of different execution and resource provisioning plans. Their target application is Montage, which is a portable software toolkit for science-grade mosaics of the sky by composing multiple astronomical images. In this case, if users continuously use and access the data sets stored on cloud storage many times, it is good to store the generated mosaic, because the storage costs are cheaper than the CPU and the data transfer costs. Some works have shown the calculation performance of Amazon EC2 by using LINPACK and NAS Parallel Benchmark.
Dept Of CSE/BTLIT/BATCH 5 12
Cloud systems are based on large clusters in which nodes are densely connected, as opposed to P2P environments in which nodes are sparsely connected. Contrary to traditional clusters, the computational and storage resources provided by clouds are fully or partly virtualized. A multicast algorithm for clouds can therefore not assume anything about the exact physical infrastructure. Similar t o P2P environments, the network performance within clouds is dynamic. The performance of the uplink and downlink of a virtual compute node can be affected by other virtual compute nodes that are running on the same physical host. Routing changes and load balancing will also affect network performance. Furthermore, the data to distribute is often located in the clouds storage service, which introduces another potential shared bottleneck. Existing multicast solutions for parallel distributed systems, like the pipelined trees described in Section 2.3.1, rely on monitoring data and topology information or a map of the available to optimize data transfers for the current network conditions. However, obtaining such information is tedious and hard, and keeping it up-to-date is almost impossible. Our multicast algorithms therefore apply solutions commonly used in P2P environments to handle the dynamic network performance in clouds. These solutions are much easier to deploy and more reactive to changes in network performance. 2.3.4 The cloud usage model of data intensive applications Our multicast algorithms are designed for effective hosting of data-intensive applications on clouds. We will assume that a typical data-intensive application would be started on multiple compute resources in a cloud in parallel, in order to process a large data set. The data set would be stored in the clouds storage service for flexibility, cost and durability. While storing the data at a user site and uploading it to each instance in the cloud would be too costly and slow, as well as users own storage. Hence, the ideal solution is to let each compute resource initially transfer the data set from the cloud storage service to its local storage before the application can start. In this Project, we adopt Amazon EC2/S3 for the aforementioned usage scenario,
Dept Of CSE/BTLIT/BATCH 5 13
although we believe our results would also apply to other clouds. Our objective is to distribute a large amount of data stored on S3 to all EC2 instances of an application, as fast as possible. To maintain the practicality of the usage scenario, we assume that EC2 instances and S3 objects are located in the same region as with the current EC2/S3 infrastructure, transferring data from one region to another would be too costly and slow for processing large amounts of data.
systems ( or were of minor significance), which leads to the redefining of priorities of the activities that go into developing software. As systems grew more complex, it became evident that the goals of the entire system could not be easily comprehended. Hence the need for more rigorous requirements analysis arose. In System Engineering & Software Engineering, Requirement analysis encompasses those tasks that go into determining the requirements of the various users. Requirements analysis is critical to the success of the project. Requirements must be measurable, testable, related to identified business needs, opportunities, and specific customers, and defined to the level of detail sufficient for system design. Since the Software Requirements Specification contains both Functional & non-functional requirements the following document states both these requirements according to the IEEE standards.
3.2.1 Functional requirements This section describes the functional Requirements of the system for those that refer to the functionality of the system, i.e., what services it will provide to the user. This system developed consists of various requirements, which can be broadly classified accordingly to their functionality as given below. 3.2.1.1 Creation of Amazon Web Services (AWS) Account.
Dept Of CSE/BTLIT/BATCH 5 15
AWS account provides cloud computing platform which is essential to fulfill the cloud services such as storage and computing resources. 3.2.1.2 Sign up for Amazon S3 & Amazon EC2 services. In AWS, We need to sign up for the Amazon S3 & Amazon EC2 services. AWS credentials are verified before the user can sign up for any services. 3.2.1.3 Create Bucket and specify region & bucket name. In AWS, Once user has signed up for the S3 service, they are allowed to perform many operations on it, Such as creation of bucket to store your data by specifying the Bucket Name & choose a region where your bucket and object(s) reside to optimize latency, minimize costs, or address regulatory requirements. 3.2.1.4 Upload Objects in the Specified Buckets & perform operations. In Amazon S3, User can perform many operations such as uploading objects into bucket; perform write, read & delete operations on the S3 objects & its contents. 3.2.1.5 Choose the appropriate AMI & Launch EC2 instances. In AWS, Once user has signed up for the EC2 service, they are allowed to create an Amazon Machine Image (AMI) containing your applications, libraries, data, and associated configuration settings by configuring security & network access for Amazon EC2 instance. Then user can choose Instance type, OS for the required EC2 & launch the configured instances for the selected AMI. 3.2.1.6 Perform the Non Steal & Steal algorithm in EC2 and get the objects from Bucket of S3. Once the communications is established between S3 & EC2 Instances, We can run both the multicast algorithms and get all the contents of the objects from bucket to EC2 computational Systems. 3.2.2 Non-Functional requirements
Dept Of CSE/BTLIT/BATCH 5
16
Nonfunctional (supplementary) requirements pertain to other information needed to produce the correct system and are detailed separately. These are requirements that are not functional in nature, i.e., these are constraints within which the system must work. The program must be self-contained so that it can easily be moved from one Computer to another. It is assumed that JDK (Java Development Kit) will be available on the computer on which the program resides. Capacity, scalability and availability The system shall achieve 100 per cent availability at all times, & shall be scalable to support additional clients and volunteers. Maintainability. The system should be optimized for supportability, or ease of maintenance as far as possible. This may be achieved through the use documentation of coding standards, naming conventions, class libraries and abstraction. Randomness, verifiability and load balancing. The system should be optimized for supportability, or ease of maintenance as far as possible. This may be achieved through the use documentation of coding standards, naming conventions, class libraries and abstraction. It should have randomness to check the nodes and should be load balanced.
Menu List having Create Bucket & Object Upload Download Show Share
The user has to enter the data through interface provided as a front end which will be displaying Amazon S3 operations so that will be place of interaction between user and the system. The input & output requirements are as mentioned below.
3.3.1.1 Input Requirements 1. If a bucket is to be created, click on Menu Item create bucket, enter the Bucket name and click on create button. 2. If a file is to be uploaded , click on MenuItem File Upload ,click browse button, select the file ,click ok button. 3. If a Object is to be downloaded, select the Object and click on download button. 4. To get the Object list of neighboring nodes, click on show button. 5. To get the missing Object Contents from neighboring nodes, click on share button.
3.3.1.2. Output Requirements The EC2 nodes receives the Object list from the S3 server and object contents should be distributed from the S3 Server to the respective EC2 nodes (i,e Objects or data stored in S3 bucket should be downloaded to the respective nodes) and nodes communicates with each other for exchanging missing Object / Data.
Dept Of CSE/BTLIT/BATCH 5
18
3.4 Design Constraints This section should indicate any design constraints on ht system being built. A design constraint represents design decisions that have been mandated and must be adhered to while developing the product.
3.4.1 Hardware Requirements Processor: Pentium IV or AMD 1.0 GHZ or above. Memory: 1 GB RAM (Min). Hard disk space: 3.5 GB (Min). Keyboard, mouse and other peripherals.
3.4.2 Software Requirements Operating System: Windows XP / Vista. Browser: Google Chrome / IE 7. Language used: Java 1.5 or higher. Java Swing front end. Networking-Socket programming.
Dept Of CSE/BTLIT/BATCH 5
19
Dept Of CSE/BTLIT/BATCH 5
20
1. High-Level Design. 2. Detailed-Level Design. In the High-Level Design, the Technical Architect of the project will study the proposed applications functional & non functional (quantitative) requirements and the design overall solution architecture of the applications, which can handle those needs. High-Level Design discusses an overall view of how something should work and the top level components that will comprise the proposed solutions. It should have very little detail on implementation, i.e. no explicit class definitions, and in some cases not even details such a databases type (Relational or object) and programming language and platform. In this chapter we give an overview of the design of the system and how it is organized and the flow of data through the system. By reading this document the user should have an overall understanding of the problem and its solution. I have also discussed about the problems encountered during the design of the system and justified the use of the design. Also the Data Flow Diagrams [DFD] is given in this chapter.
Dept Of CSE/BTLIT/BATCH 5
Upload Objects in the Specified Buckets & perform operations. Choose the appropriate AMI & Launch EC2 instances. Access Objects stored in S3 from EC2 instances. Perform the Non Steal & Steal algorithm in EC2 and get the objects from Bucket of S3. Finally all the running EC2 Instances will have the contents of the objects.
Scalable: Amazon S3 can scale in terms of storage, request rate, and users to support an unlimited number of web-scale applications. It uses scale as an advantage: Adding nodes to the system increases, not decreases, its availability, speed, throughput, capacity, and robustness.
Reliable: Store data with up to 99.999999999% durability, with 99.99% availability. There can be no single points of failure. All failures must be tolerated or repaired by the system without any downtime.
Fast: Amazon S3 must be fast enough to support high-performance applications. Serverside latency must be insignificant relative to Internet latency. Any performance bottlenecks can be fixed by simply adding nodes to the system.
Inexpensive: Amazon S3 is built from inexpensive commodity hardware components. All hardware will eventually fail and this must not affect the overall system. It must be hardware-agnostic, so that savings can be captured as Amazon continues to drive down infrastructure costs.
Simple: Building highly scalable, reliable, fast, and inexpensive storage is difficult. Doing so in a way that makes it easy to use for any application anywhere is more difficult. Amazon S3 must do both.
Data flow diagrams are commonly used during problem analysis. They are quite general and are not limited to problem analysis alone during the software requirements specification. Data Flow Diagrams indicate the flow of data through the system. It views the system as a function that transforms the inputs into desired outputs. Since a complex system will not undergo this transformation in a single step, data must typically undergo a series of transformation before it becomes the output. The DFD aims to capture the transformations that take place within a system to the input data so that eventually the output data is produced. It should be noted that DFD is not a flowchart. A DFD represents the flow of data, while a flowchart represents the flow of control. A DFD does not represent procedural information. The agent that performs the transformation of data from one state to another is called a process (denoted by Bubble). So the DFD basically shows the movement of data through the various transformations or processes in a system. This gives a clear understanding as to how the data flows through the system and therefore helps in improving the understanding of the working of the system.
4.3.1 DFD Level 0 Context level DFD represents the fundamental system model. The entire software element is represented as a single bubble. In this level 0 Data flow diagram has two entities like user and output. Valid Input Processed Input
1.0 INPUT
Invalid Input Fig 4.1 Level-0 Data flow diagram. In this diagram the system is shown to comprise of simple entities like the user, the whole system which is described as Amazon Web Services (AWS) and output are depicted in this DFD. Each of
Dept Of CSE/BTLIT/BATCH 5 23
AWS
OUTPUT
these entities will have a specific work assigned to it and they will have to be performed by the respective modules. User will be interacting with the system to provide the input which is to perform the operations on Amazon S3 & EC2. Here the validity of the input is checked and matched with the AWS access credentials and then accepted to be valid if not suitable action will be taken. The AWS module consists of many sub-modules to get register & login into Amazon S3 & Amazon EC2. Each of these sub-modules will have their own set of responsibilities which needs to be completed by respective modules. Finally, all these sub modules are represented to make an Amazon Web Service. Output will be shown to the user by the interface and this will be an implementation using java swing which consists of library functions. Thus the process can be completed using this comprised of above said modules.
4.3.2 DFD Level 1 The level-1 data flow diagram explains in greater details the various entities and modules of the USER project and their interoperability, as well as the various inputs and expected outputs. As can be seen from the figure, 4.2 there are entities like User, GUI, AWS, Check AWS credentials & output. Here each of modules work will be explained to understand the role of the system. User will be interacting with the system to provide the input which is to perform the operations on Amazon S3 & EC2. Here the validity of the input is checked and matched with the AWS access credentials and then accepted to be valid if not suitable action will be taken. 1.1 GUI 1.2 AWS 1.3 Files Containing AWS Access CHECK_ Credentials AWS Secret Key
24
Instruction
Interaction
Dept Of CSE/BTLIT/BATCH 5
OUTPUT
Key
Invalid Input
Data
Processed
Fig 4.2 Level-1 Data flow diagram. Graphical user interface GUI is made using the java swings which is designed to be small and efficient, but still flexible enough to allow the programmer freedom in the interfaces created. Java Swings allows the programmer to use a variety of standard user interfaces widgets such as push, radio & check buttons, menus, lists & frame in the system. Check_ AWS performs the checking of the users AWS credentials and verify the access key and secret key and allow user to perform the operations on Amazon S3 & Amazon EC2. By creating buckets in S3 and uploading few files into it. Output will be shown to user by the interfaces and this will be implemented using which again uses the interface GUI made using java swings. Thus the process can be completed using this strategy comprised of above said modules.
Dept Of CSE/BTLIT/BATCH 5
25
2.2 Sign Up for AWS \ 2.1 Sign Up For Amazon S3 Access S3 Services & Functionality CHECK AWS Access EC2 Services & Functionality Amazon EC2 ACCESS S3/EC2
The level-2 data flow diagram explains in greater details the various entities and modules and their interoperability, as well as the various inputs and expected outputs. As can be seen in the figure 4.3, the input to the diagram is from AWS management Console; here we need to sign up for the Amazon S3 & Amazon EC2 services. AWS credentials are verified before the user can sign up for any services, then once user have signed up for these services then they can access the functionality.
S3
S3 Task
Dept Of CSE/BTLIT/BATCH 5
26
The level-3 data flow diagram explains in greater details the various entities and modules and their interoperability, as well as the various inputs and expected outputs. As can be seen in the figure 4.4, the input to the diagram is from Amazon S3 service, once user has signed up for the S3 service, they are allowed to perform many operations on it, Such as creation of bucket to store your data by specifying the Bucket Name & choose a region where your bucket and object(s) reside to optimize latency, minimize costs, or address regulatory requirements. Then user can perform many operations such as uploading objects into bucket; perform write, read & delete operations on the S3 objects & its contents. Few main features of Amazon S3 are
Write, read, and delete objects containing from 1 byte to 5 terabytes of data each. The number of objects you can store is unlimited.
Each object is stored in a bucket and retrieved via a unique, developer-assigned key. A bucket can be stored in one of several Regions. You can choose a Region to optimize for latency, minimize costs, or address regulatory requirements. Amazon S3 is currently available in the US Standard, EU (Ireland), US West (Northern California), Asia Pacific (Singapore), and Asia Pacific (Tokyo) Regions. The US Standard Region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps.
Objects stored in a Region never leave the Region unless you transfer them out. For example, objects stored in the EU (Ireland) Region never leave the EU.
Authentication mechanisms are provided to ensure that data is kept secure from unauthorized access. Objects can be made private or public, and rights can be granted to specific users.
Uses standards-based REST and SOAP interfaces designed to work with any Internetdevelopment toolkit.
Dept Of CSE/BTLIT/BATCH 5
27
Built to be flexible so that protocol or functional layers can easily be added. The default download protocol is HTTP. A BitTorrent protocol interface is provided to lower costs for high-scale distribution.
EC2
4.1 Create Amazon EC2 Machine Image (AMI) Configure Security & Network Access for EC2. Task
4.2 Choose Instance EC2 Type & OS for your EC2 Launch & Start instance for selected AMI
Fig 4.5 Level-4 Data flow diagram. The level-2 data flow diagram explains in greater details the various entities and modules and their interoperability, as well as the various inputs and expected outputs. As can be seen in the figure 4.3, the input to the diagram is from Amazon EC2 service, once user has signed up for the EC2 service, we are allowed to create an Amazon Machine Image (AMI) containing your applications, libraries, data, and associated configuration settings by configuring security & network access for Amazon EC2 instance. Then user can choose Instance type, OS for the required EC2 & launch the configured instances for the selected AMI. Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of operating systems, load them with your custom application environment, manage your networks access permissions, and run your image using as many or few systems as you desire. To use Amazon EC2, you simply:
Dept Of CSE/BTLIT/BATCH 5 28
Select a pre-configured, template image to get up and running immediately. Or create an Amazon Machine Image (AMI) containing your applications, libraries, data, and associated configuration settings.
Configure security and network access on your Amazon EC2 instance. Choose which instance type(s) and operating system you want, then start, terminate, and monitor as many instances of your AMI as needed, using the web service APIs or the variety of management tools provided.
Determine whether you want to run in multiple locations, utilize static IP endpoints, or attach persistent block storage to your instances. Pay only for the resources that you actually consume, like instance-hours or data transfer.
Our project has modules namely Bucket Operations, Non Steal & Steal Algorithm & EC2 Configuration. The following figures show the various modules of my project, and their interaction with the other modules and its functionality.
IF AWS_CREDENTIALS MATCHES Sign Up For Amazon S3 Create a Bucket Select a Bucket name & Region
IF AWS_CREDENTIALS Sign Up For Amazon EC2 MATCHES Create Amazon Machine Image (AMI); Configure Security & Network Access for EC2.
Upload Files / Objects Into Bucket Operations - Write, read, and delete objects present in bucket
Choose Instance Type & OS for your EC2 Launch & Start instance for selected AMI
Dept Of CSE/BTLIT/BATCH 5
Perform the Non Steal & Steal algorithm in EC2 and get the objects from Bucket of S3
30
This project contains the following modules. Server Module for Non Steal & Steal Algorithm Client Module for Non- Steal Algorithm. Client Module for Steal Algorithm.
Step 5: Run the algorithm for carrying out scatter & allgather phase on Object. Input The input to this module is user commands to carry Amazon S3 operations. Output The output is the results on GUI showing S3 tasks. Start Amazon Web Services
Divide & Distribute Object Content using Non-Steal & Steal Algorithm
Dept Of CSE/BTLIT/BATCH 5 33
Scatter Phase
Allgather Phase
Dept Of CSE/BTLIT/BATCH 5
34
Nodes Fig 4.7 Flowchart on Server Module showing both Non-Steal & Steal Algorithm
Input The input to this module is user commands to carry Amazon S3 operations to check Object contents. Output
Dept Of CSE/BTLIT/BATCH 5
35
The output is the creation of text files that are having all the contents of the objects. All the nodes will have the object contents after non-steal algorithm is executed.
Store Access key and Secret key from Amazon Web Services
Input The input to this module is user commands to carry Amazon S3 operations to check Object contents. Output The output is the creation of text files that are having all the contents of the objects. All the nodes will have the object contents after steal algorithm is executed. Here the fastest node will steal some work from the neighboring nodes and download the contents of the object on their behalf and then exchange the contents.
Dept Of CSE/BTLIT/BATCH 5
38
Store Access key and Secret key from Amazon Web Services
Dept Of CSE/BTLIT/BATCH 5
40
CHAPTER 5 IMPLEMENTATION
5.1 Programming language Selection
The language used in this project is JAVA and compilations are done in the terminal provided for the user is Eclipse and also it is necessary as front end is done using Swings. The major reason for using JAVA in project was the ease and the control, the language gives on system calls and interface design. The entire project stands on the JAVA API provided by Amazon Web Services to access the functionality of EC2 & S3. Hence it was important to choose a language which supports the design of a very good interface to interact with the users of the system. JAVA source code can be optimized much more than higher-level languages because the language set are relatively small & very efficient.
Dept Of CSE/BTLIT/BATCH 5 41
That leads to a third advantage that JAVA has is its application in developing the front end for the user is easy and efficient.
JAVA is an object oriented programming language and it was intended to serve as a new way to manage software complexity. JAVA refers to a number of computer software products and specifications from Sun Microsystems that together provides a system for developing applications software and deploying it in a cross-platform environment.
Dept Of CSE/BTLIT/BATCH 5
42
storage: The available throughput should scale according to the number of nodes, so the multicast algorithm achieves maximum utilization of the available aggregate download throughput despite any rapid changes in the underlying throughputs of individual nodes.
Minimization of multicast completion time of each node: usually all the nodes not
only need to start to calculate as early as possible, but they also need to be able to commence the calculation by and large simultaneously to avoid the long-tail effect resulting in resource underutilization. A multicast operation should be stable, i.e. the data transfer finishes without any outliers even if the users allocation scales to tens of thousands of nodes.
network topology: various previous works we have mentioned usually tried to achieve the above policies by network monitoring and/or estimation of the physical network topologies. Such an approach is undesirable in clouds, as cloud compute resources are dynamically provisioned by the underlying cloud system. Any measurement results would generate significant overhead due to extensive monitoring requirements while they would likely be not be applicable across runs.
Table 5.1 shows the different requirements and characteristics of multicast algorithms for clusters, P2P systems, and clouds.
Dept Of CSE/BTLIT/BATCH 5 43
Multicast algorithms for clusters can achieve high performance by using an expensive algorithm that requires monitoring data to construct a high-throughput spanning tree, but they cannot adapt well to dynamic and unstable changes in network throughput. Multicast algorithms for P2P systems are scalable and can adapt well to network performance changes. However, it is difficult to use P2P algorithms to achieve high performance because they overly assume that not only the network but also the availability of the data and the nodes is unstable, resulting in significant overhead. Our algorithms achieve both high performance and scalability by combining the advantages of multicast algorithms or clusters and P2P systems. In particular, we do not perform network monitoring, and balance the network load by using work stealing technique.
Cluster Multicast topology Communication type Network performance Node proximity Node-to-node performance Storage-to-node performance Underlying network topology Correspond to dynamic change Spanning tree(s) Push High Dense Homogeneous Homogeneous Stable Bad
Clouds Tree+ overlay Pull Middle Dense Heterogeneous Heterogeneous (un)stable Good
Table 5.1 Features of general and proposed multicast algorithms on each environment.
5.3.2 A brief overview of Non-Steal algorithm The multicast algorithm proposed by van de Geijn et al. is a well known algorithm for clusters and multi-clusters environments. It achieves high performance multicast operations, and is often used in efficient MPI collectives implementations.
Dept Of CSE/BTLIT/BATCH 5 44
In the scatter phase, the root node divides the data to be multicast into blocks of equal size depending on the number of nodes. These blocks are then sending to each corresponding node using a binomial tree. After all the nodes have receives the divided blocks, they start the allgather phase in which the missing blocks are exchanged and collected by using the recursive doubling technique.
Our non-steal algorithm is inspired by this algorithm. It also consists of a scatter phase and an allgather phase. All nodes cooperate to download and forward data from S3 to each EC2 node. Initially, none of the nodes has any parts of the data stored in S3, so S3 corresponds to a multicast root node. Figure 5.1 depicts the two phases the algorithm:
Dept Of CSE/BTLIT/BATCH 5
45
Phase 1 (Non-Steal): The file to distribute is logically divided into P fixed-sized pieces, e.g. 32KB each, numbered from 0 to P 1. When the number of nodes is N, each node i is assigned a range of pieces:
------------------ (1) Node i then downloads all the pieces in its range from S3. Once a node finishes downloading the assigned range of pieces, it waits until all the other nodes have finished too.
Phase 2: After constructing a full overlay network between the all the nodes, each
node continuously exchanges information with its neighbors in the mesh about which pieces they already obtained, and fetches missing pieces from them until all pieces are downloaded.
5.3.2.1 Example of Non-Steal Algorithm Consider three nodes (A, B and C) that download the same 300 MB file from S3. Node B has a fast connection to S3 (10 MB /sec), while A and C have a slow connection to S3 (2 MB /sec). The file will first be logically split into 9600 pieces of 32KB each (9600 * 32KB = 300 MB). Initially, each node requests the assigned 100 MB from S3 (i.e. Node A, B, and C request pieces 0-3199, 3200-6399 and 6400-9599, respectively). After approximately 10 seconds, node B will finish downloading its range. Node A and C, on the other hand, achieve slower throughput and will finish after 50 seconds. Since all nodes wait until everybody has finished, the total completion time of phase 1 is 50 seconds. Once phase 1 is finished, all nodes start phase 2 and exchange the pieces using a BitTorrent-like protocol. Each node connects to some neighbor nodes and sends a possession list indicating which pieces it has. For example, node i send its possession list to node j. If the list contains piece p that
Dept Of CSE/BTLIT/BATCH 5 46
node j has not yet obtained, node j requests node i to send it piece p which is then returned by node i. After node j obtains piece p, node j informs all its neighbors that it now has piece p. The informed nodes then update their possession list of node j. All nodes communicate this way until they have obtained all pieces.
5.3.3. A brief overview of Steal algorithm In the non-steal algorithm, all nodes globally synchronize with each other once they have downloaded their assigned range of pieces from S3. As shown in Figure 5.1, however, the completion time may vary greatly among the nodes due to significant and unpredictable variance in the download throughput. The Steal algorithm resolves this issue by stealing: when a node has downloaded its assigned range of pieces, it actively asks other nodes whether they still have pieces to download. If the node asked is still downloading, it splits its own assignments, and returns some of its pieces. This approach is similar to work stealing, in that a fast node steals some download work from a slower node. To make the steal algorithm efficient, we adapt the amount of stolen work to the download bandwidth from S3 observed by both nodes. The steal algorithm consists of two phases:
Phase 1 (Scatter): similar to the non-steal algorithm, The file to distribute is logically split into P equally-sized pieces, and each node i is assigned a range of pieces as shown in Equation 1. When node i has finished downloading its pieces, it asks other nodes whether they have any work remaining and reports its own download throughput Bi for the download just completed. Now assume that node j has W remaining pieces, and its download throughput is currently Bj. Node j then divides W into Wi and Wj such that:
Dept Of CSE/BTLIT/BATCH 5 47
Wi = [W * Bi / ( Bi + Bj)] Wj = [W * Bj / ( Bi + Bj)]
Node j then returns Wi pieces to node i. Node i and j can then concurrently download Wi and Wj pieces, respectively. Hence, the amount of work they download is
proportional to their download bandwidth. Phase 2 (allgather) : similar to the non-steal algorithm, each node exchanges pieces within EC2 by using a BitTorrent-like protocol until all the pieces have been obtained. Note that in phase 1, we seem to use the heuristic that the download throughput is stable over a short period of time, yet the instability we have demonstrated in our earlier experiment would render such a heuristic ineffective. However, we claim that it is an effective strategy for the following reason: even if the new bandwidth Bi turns out to be slower than the previous Bi, it would be relatively OK: if Wi is small it will quickly
terminate in any case, and if Wi is large it could be subject to further split by some alternative node that will have finished its work, i.e. it will be effectively in the position of node. 5.3.3.1 Example of Steal Algorithm Let us now apply the steal algorithm in the same example scenario we described previously in Section 5.3.2.1. Initially Node A, B and C are assigned ranges 0-3199, 3200-6399, and 6400-9599. From the start of the algorithm, after approximately 10 seconds the fast node B will finish its work. Node B will then request more work (i.e. steal) from, for example, node A. By then, node A will have downloaded about 20 MB of its assigned total 100MB (10 seconds * 2 MB/sec = 20 MB). This is equivalent to 640 pieces, and leaves 2560 remaining pieces to download from S3. Node B will then steal work from node A according to equations (2) and (3), i.e. [2560 * 10/ (2 + 10)] = 2134 pieces (indices 1066-3199). These pieces are returned to node B as
Dept Of CSE/BTLIT/BATCH 5 48
new work. Node A then restarts downloading [2560 * 2/ (2 + 10)] = 426 pieces (numbers 640-1065). This process is repeated until all pieces have been downloaded from S3. Finally, the nodes exchange all downloaded pieces with each other in phase 2 of the algorithm. Our steal algorithm can generally apply not only to the Amazon EC2/S3, but also to the other cloud systems (e.g. Windows Azure Storage), moreover, it is possible to apply to the some parallel distributed file systems (e.g. Lustre and GFS) in order to get higher throughput. This is because bandwidth fluctuation per connection potentially will occur in these environments. For example, GFS divides file into a lot of fixed chunks and distributes these chunks to multiple chunk servers, so user can achieve high I/O throughput to/from chunk servers by aggregating each I/O connection. When some nodes access to the same chunk server simultaneously, however, I/O contention occurring access or I/O contention to same chunk server, however, it is confirmed that bandwidth performance in some links decreases.
Dept Of CSE/BTLIT/BATCH 5
49
Blood bank management system using amazon elastic cloud computing 4. Launching the EC2 instance & configuring it as per our requirements. 5. Importing / Copying the Amazon AWS Jar Files & API.
Some of the strategies used to tackle the challenges were the following 1. Always run the project in availability of high speed internet for proper execution & getting results. 2. By choosing the right AMI & EC2 instance, We have selected the Windows 2008 Express Edition Server AMI & chose Micro EC2 instance as it satisfy the requirements of the project.
3. S3 Bucket should have unique name & region. While creating case should be taken to
avoid the creation of bucket with a Capital Alphabet letters. Always use small letters.
4.
By selecting the correct Security Group & providing it with secure communication ports which enables us to communicate easily & securely among EC2 nodes.
5. By importing the AWS JAVA API into codes ease our S3 operations & then by copying the jar files in the JAVA SDK modules enables us to work with AWS.
Dept Of CSE/BTLIT/BATCH 5
50
Dept Of CSE/BTLIT/BATCH 5
51
Fig. 5.3 Access Key ID & Secret Key generated as Access credentials.
Dept Of CSE/BTLIT/BATCH 5
52
Dept Of CSE/BTLIT/BATCH 5
53
Dept Of CSE/BTLIT/BATCH 5
58
/** * This code was edited or generated using CloudGarden's Jigloo * SWT/Swing GUI Builder, which is free for non-commercial * use. If Jigloo is being used commercially (ie, by a corporation, 48. * company or business for any purpose whatever) then you 49. * should purchase a license for each developer using Jigloo. 50. * Please visit www.cloudgarden.com for details. 51. * Use of Jigloo implies acceptance of these licensing terms. 52. * A COMMERCIAL LICENSE HAS NOT BEEN PURCHASED FOR 53. * THIS MACHINE, SO JIGLOO OR THIS CODE CANNOT BE USED 54. * LEGALLY FOR ANY CORPORATE OR COMMERCIAL PURPOSE. 55. */ 56. public class Insert extends javax.swing.JFrame implements ActionListener{ 57. 58. { 59. //Set Look & Feel 60. try { 61. javax.swing.UIManager.setLookAndFeel("com.sun.java.swing.plaf.windows.W indowsLookAndFeel"); 62. } catch(Exception e) { 63. e.printStackTrace(); 64. } 65. } 66. 67. private JMenuBar jMenuBar1; 68. private JMenu jMenu5; 69. private JMenuItem helpMenuItem; 70. private JLabel jLabel4; 71. private JLabel jLabel5; 72. private JComboBox cboxGender; 73. private JComboBox cboxCity; 74. private JComboBox cboxBGrp; 75. private JButton btnClear; 76. private JButton btnSubmit; 77. private JTextField txtContact; 78. private JTextField txtAge; 79. JTextField txtName; 80. private JTextField txtLoc; 81. private JLabel jLabel9;
Dept Of CSE/BTLIT/BATCH 5
59
Dept Of CSE/BTLIT/BATCH 5
60
jLabel1.setBorder(BorderFactory.createBevelBorder(BevelBorder.RAISED, new java.awt.Color(255,0,0), new java.awt.Color(128,255,0), null, new java.awt.Color(0,255,255))); 134. jLabel1.setDisabledIcon(new ImageIcon(getClass().getClassLoader().getResource("blood1.JPG"))); 135. //jLabel8.setIcon(new ImageIcon(getClass().getClassLoader().getResource("1.jpg"))); 136. } 137. { 138. insert = new JLabel(); 139. getContentPane().add(insert,"2,3,1,1"); 140. insert.setText("Add Details"); 141. insert.setFont(new java.awt.Font("Andalus",3,22)); 142. insert.setBounds(60, 68, 153, 32); 143. insert.setForeground(new java.awt.Color(255,0,0)); 144. 145. } 146. { 147. jLabel2 = new JLabel(); 148. getContentPane().add(jLabel2, "2, 4 1 1"); 149. jLabel2.setText("Donor Name"); 150. jLabel2.setFont(new java.awt.Font("Tahoma",1,11)); 151. jLabel2.setBounds(49, 110, 108, 24); 152. } 153. { 154. jLabel3 = new JLabel(); 155. getContentPane().add(jLabel3, "2, 6 1 1"); 156. jLabel3.setText("Gender"); 157. jLabel3.setFont(new java.awt.Font("Tahoma",1,11)); 158. jLabel3.setBounds(47, 149, 106, 19); 159. } 160. { 161. jLabel4 = new JLabel(); 162. getContentPane().add(jLabel4, "2, 6 1 1"); 163. jLabel4.setText("Age"); 164. jLabel4.setBounds(47, 182, 81, 23); 165. jLabel4.setFont(new java.awt.Font("Tahoma",1,11)); 166. }
Dept Of CSE/BTLIT/BATCH 5
61
jLabel5 = new JLabel(); getContentPane().add(jLabel5); jLabel5.setText("Blood Group"); jLabel5.setBounds(47, 226, 81, 14); jLabel5.setFont(new
jLabel6 = new JLabel(); getContentPane().add(jLabel6); jLabel6.setText("Location/City"); jLabel6.setBounds(47, 266, 81, 14); jLabel6.setFont(new
jLabel7 = new JLabel(); getContentPane().add(jLabel7); jLabel7.setText("Contact No."); jLabel7.setBounds(47, 344, 88, 14); jLabel7.setFont(new
txtName.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERED, null, new java.awt.Color(128,128,0), null, null)); 193. } 194. { 195. ComboBoxModel cboxGenderModel = 196. new DefaultComboBoxModel( 197. new String[] { "<Select Gender", "M","F" }); 198. cboxGender = new JComboBox(); 199. getContentPane().add(cboxGender); 200. cboxGender.setModel(cboxGenderModel); 201. cboxGender.setBounds(158, 223, 158, 20); 202. cboxGender.setBounds(157, 148, 159, 20); 203. cboxGender.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERE D, null, new java.awt.Color(128,128,0), null, null)); 204. } 205. { 206. txtAge = new JTextField(); 207. getContentPane().add(txtAge);
Dept Of CSE/BTLIT/BATCH 5
62
txtAge.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERED, null, new java.awt.Color(128,128,0), null, null)); 210. } 211. { 212. txtContact = new JTextField(); 213. getContentPane().add(txtContact); 214. txtContact.setBounds(158, 338, 156, 20); 215. txtContact.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERE D, null, new java.awt.Color(128,128,0), null, null)); 216. } 217. { 218. btnSubmit = new JButton(); 219. getContentPane().add(btnSubmit); 220. btnSubmit.setText("Submit"); 221. btnSubmit.setBounds(158, 399, 77, 28); 222. btnSubmit.setFont(new java.awt.Font("Tahoma",1,11)); 223. btnSubmit.setBorder(BorderFactory.createBevelBorder(BevelBorder.RAISED, new java.awt.Color(255,255,0), new java.awt.Color(255,0,0), new java.awt.Color(255,255,0), new java.awt.Color(255,0,0))); 224. btnSubmit.addActionListener(this); 225. } 226. { 227. btnClear = new JButton(); 228. getContentPane().add(btnClear); 229. btnClear.setBounds(245, 399, 83, 27); 230. btnClear.setText("Clear"); 231. btnClear.setFont(new java.awt.Font("Tahoma",1,11)); 232. btnClear.setBorder(BorderFactory.createBevelBorder(BevelBorder.RAISED, new java.awt.Color(255,255,0), new java.awt.Color(255,0,0), new java.awt.Color(255,255,128), new java.awt.Color(255,0,0))); 233. btnClear.setForeground(new java.awt.Color(0,0,0)); 234. btnClear.addActionListener(this); 235. } 236. { 237. ComboBoxModel cboxBGrpModel = 238. new DefaultComboBoxModel( 239. new String[] { "<Select BloodGroup>", "Aplus","Aminus","Bplus","Bminus","Oplus","Ominus" }); 240. cboxBGrp = new JComboBox();
Dept Of CSE/BTLIT/BATCH 5
63
cboxBGrp.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERED, null, new java.awt.Color(128,128,0), null, null)); 245. } 246. { 247. ComboBoxModel cboxCityModel = 248. new DefaultComboBoxModel( 249. new String[] { "<Select City>", "Bengaluru","Mysore","Hyderabad" }); 250. cboxCity = new JComboBox(); 251. getContentPane().add(cboxCity); 252. cboxCity.setModel(cboxCityModel); 253. cboxCity.setBounds(158, 260, 158, 20); 254. cboxCity.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERED, null, new java.awt.Color(128,128,0), null, null)); 255. } 256. { 257. jLabel8 = new JLabel(new ImageIcon("b.jpg")); 258. getContentPane().add(jLabel8); 259. //jLabel8.setText("Image"); 260. //jLabel8.setIcon("purshi.jpg"); 261. jLabel8.setBounds(393, 154, 203, 180); 262. jLabel8.setBorder(BorderFactory.createBevelBorder(BevelBorder.RAISED, new java.awt.Color(255,0,0), new java.awt.Color(255,0,0), new java.awt.Color(0,255,0), new java.awt.Color(0,255,0))); 263. jLabel8.setIcon(new ImageIcon(getClass().getClassLoader().getResource("1.jpg"))); 264. 265. } 266. { 267. errLab = new JLabel(); 268. getContentPane().add(errLab); 269. errLab.setBounds(383, 93, 176, 29); 270. } 271. { 272. jLabel9 = new JLabel(); 273. getContentPane().add(jLabel9); 274. jLabel9.setText("Location"); 275. jLabel9.setBounds(49, 304, 88, 14); 276. jLabel9.setFont(new java.awt.Font("Tahoma",1,11)); 277. } 278. {
Dept Of CSE/BTLIT/BATCH 5
64
txtLoc.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERED)); 283. } 284. } 285. this.setSize(700, 550); 286. { 287. jMenuBar1 = new JMenuBar(); 288. setJMenuBar(jMenuBar1); 289. jMenuBar1.setBackground(new java.awt.Color(0,255,255)); 290. { 291. jMenu3 = new JMenu(); 292. jMenuBar1.add(jMenu3); 293. jMenu3.setText("Home"); 294. jMenu3.setBackground(new java.awt.Color(0,128,0)); 295. { 296. insertMenuItem = new JMenuItem(); 297. jMenu3.add(insertMenuItem); 298. insertMenuItem.setText("Insert"); 299. } 300. { 301. searchMenuItem = new JMenuItem(); 302. jMenu3.add(searchMenuItem); 303. searchMenuItem.setText("Search"); 304. searchMenuItem.addActionListener(this); 305. } 306. { 307. jSeparator2 = new JSeparator(); 308. jMenu3.add(jSeparator2); 309. } 310. { 311. exitMenuItem = new JMenuItem(); 312. jMenu3.add(exitMenuItem); 313. exitMenuItem.setText("Exit"); 314. exitMenuItem.addActionListener(this); 315. } 316. } 317. { 318. jMenu5 = new JMenu(); 319. jMenuBar1.add(jMenu5); 320. jMenu5.setText("Help"); 321. jMenu5.setBackground(new java.awt.Color(128,128,64));
Dept Of CSE/BTLIT/BATCH 5
65
Dept Of CSE/BTLIT/BATCH 5
66
Dept Of CSE/BTLIT/BATCH 5
67
Dept Of CSE/BTLIT/BATCH 5
68
Dept Of CSE/BTLIT/BATCH 5
69
Dept Of CSE/BTLIT/BATCH 5
70
//Set Look & Feel try { javax.swing.UIManager.setLookAndFeel("com.sun.java.swing.plaf.windows.Windows LookAndFeel"); } catch(Exception e) { e.printStackTrace(); } } private private private private private private private private private private private private private private private private private JMenuBar jMenuBar1; JMenu jMenu5; JMenuItem helpMenuItem; JLabel jLabel5; static JLabel jLabel4; JLabel jLabel3; JComboBox cboxBGrp; JComboBox cboxCity; JButton btnClear; JButton btnSubmit; JLabel jLabel2; JLabel jLabel1; JMenuItem exitMenuItem; JSeparator jSeparator2; JMenuItem insertMenuItem; JMenuItem searchMenuItem; JMenu jMenu3;
/** * Auto-generated main method to display this JFrame */ public static void main(String[] args) {
Dept Of CSE/BTLIT/BATCH 5
71
SwingUtilities.invokeLater(new Runnable() { public void run() { Search inst = new Search(); inst.setLocationRelativeTo(null); inst.setVisible(true); inst.setBounds(0, 0, 700, 550); } }); } public Search() { super(); initGUI(); } private void initGUI() { try { { getContentPane().setLayout(null); getContentPane().setBackground(new java.awt.Color(21,193,230)); getContentPane().setForeground(new java.awt.Color(0,255,0)); this.setFont(new java.awt.Font("Arial",0,10)); //this.setIconImage(new ImageIcon(getClass().getClassLoader().getResource("purshi.jpg")).getImage()); { jLabel1 = new JLabel(); getContentPane().add(jLabel1, "2, 0 1 1"); jLabel1.setText("Blood Bank Management System Using Clouds"); jLabel1.setFont(new java.awt.Font("Andalus",3,22)); jLabel1.setBounds(60, 37, 538, 32); jLabel1.setForeground(new java.awt.Color(255,0,0)); } { jLabel2 = new JLabel(); getContentPane().add(jLabel2, "2, 4 1 1"); jLabel2.setText("Location"); jLabel2.setFont(new java.awt.Font("Tahoma",1,11)); jLabel2.setBounds(59, 150, 108, 24); } { jLabel5 = new JLabel(); getContentPane().add(jLabel5); jLabel5.setText("Blood Group"); jLabel5.setBounds(60, 200, 81, 14); jLabel5.setFont(new java.awt.Font("Tahoma",1,11));
Dept Of CSE/BTLIT/BATCH 5
72
java.awt.Font("Tahoma",1,11));
btnClear = new JButton(); getContentPane().add(btnClear); btnClear.setBounds(233, 297, 83, 27); btnClear.setText("Clear"); btnClear.setFont(new
java.awt.Font("Tahoma",1,11)); btnClear.setBorder(BorderFactory.createBevelBorder(BevelBorder.RAISED, null, new java.awt.Color(255,0,128), new java.awt.Color(0,128,0), null)); btnClear.addActionListener(this); } { ComboBoxModel cboxCityModel = new DefaultComboBoxModel( new String[] { "<Select City>", "Bengaluru","Mysuru","Hyderabad" }); cboxCity = new JComboBox(); getContentPane().add(cboxCity); cboxCity.setModel(cboxCityModel); cboxCity.setBounds(158, 152, 158, 20); cboxCity.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERED, null, null, null, new java.awt.Color(0,128,0))); } { ComboBoxModel cboxBGrpModel = new DefaultComboBoxModel( new String[] { "<Select BloodGroup>", "Aplus","Aminus","Bplus","Bminus","Oplus","Ominus" }); cboxBGrp = new JComboBox(); getContentPane().add(cboxBGrp); cboxBGrp.setModel(cboxBGrpModel); cboxBGrp.setBounds(158, 196, 158, 20); cboxBGrp.setBorder(BorderFactory.createBevelBorder(BevelBorder.LOWERED, null, null, null, new java.awt.Color(0,128,0))); } {
Dept Of CSE/BTLIT/BATCH 5
73
java.awt.Font("Tahoma",3,14)); java.awt.Color(255,0,0)); } {
jLabel4 = new JLabel(new ImageIcon("1.jpg")); getContentPane().add(jLabel4); jLabel4.setIcon(new ImageIcon(getClass().getClassLoader().getResource("1.jpg"))); jLabel4.setBounds(377, 122, 203, 189); jLabel4.setBorder(BorderFactory.createBevelBorder(BevelBorder.RAISED, new java.awt.Color(0,255,0), new java.awt.Color(255,0,0), new java.awt.Color(255,128,0), new java.awt.Color(255,255,0))); } { test", 10, 80); JScrollPane(resultTA);
} } this.setSize(700, 550); { jMenuBar1 = new JMenuBar(); setJMenuBar(jMenuBar1); jMenuBar1.setBackground(new java.awt.Color(0,255,255)); { jMenu3 = new JMenu(); jMenuBar1.add(jMenu3); jMenu3.setText("Home"); jMenu3.setBackground(new java.awt.Color(64,128,128)); { insertMenuItem = new JMenuItem(); jMenu3.add(insertMenuItem); insertMenuItem.setText("Insert"); insertMenuItem.addActionListener(this); } { searchMenuItem = new JMenuItem(); jMenu3.add(searchMenuItem); searchMenuItem.setText("Search"); searchMenuItem.addActionListener(this);
Dept Of CSE/BTLIT/BATCH 5
74
java.awt.Color(128,128,64));
} }
@Override public void actionPerformed(ActionEvent e) { // TODO Auto-generated method stub if(e.getSource()==exitMenuItem) { System.exit(0); } else if(e.getSource()==insertMenuItem) { this.setVisible(false); Insert ins= new Insert(); ins.setVisible(true); }
else if(e.getSource()==btnSubmit) { String bucketName=(String)cboxBGrp.getSelectedItem(); String key=(String) cboxCity.getSelectedItem(); try { PropertiesCredentials( AmazonS3 s3Client = new AmazonS3Client(new
Dept Of CSE/BTLIT/BATCH 5
75
GetObjectRequest(
bucketName, key); S3Object objectPortion = s3Client.getObject(rangeObjectRequest); System.out.println("Printing bytes retrieved."); displayTextInputStream(objectPortion.getObjectContent(),key); } catch(IOException e1){} catch (AmazonServiceException ase) { System.out.println("Caught an AmazonServiceException, which" + " means your request made it " + "to Amazon S3, but was rejected with an error response" + " for some reason."); System.out.println("Error Message: " + ase.getMessage()); System.out.println("HTTP Status Code: " + ase.getStatusCode()); System.out.println("AWS Error Code: " + ase.getErrorCode()); System.out.println("Error Type: " + ase.getErrorType()); System.out.println("Request ID: " + ase.getRequestId()); } catch (AmazonClientException ace) { System.out.println("Caught an AmazonClientException, which means"+ " the client encountered " + "an internal error while trying to " + "communicate with S3, " + "such as not being able to access the network."); System.out.println("Error Message: " + ace.getMessage()); } }
Dept Of CSE/BTLIT/BATCH 5
76
} }
private static void displayTextInputStream(InputStream input, String key) throws IOException { PrintWriter pw=new PrintWriter(new FileWriter(key)); FileWriter fstream = new FileWriter(key,true); BufferedWriter out = new BufferedWriter(fstream); try{ BufferedReader reader = new BufferedReader(new InputStreamReader(input)); while (true) { String line = reader.readLine(); if (line == null) { break; } pw.println(line); //System.out.println("written successfully"+line); //if (line == null) break; //out.write(line); //out.newLine(); //line = reader.readLine(); // System.out.println(" } }catch(Exception e){} finally{ if (pw != null){ jLabel4.setText("Data Downloaded Successfully.......!"); pw.close(); //out.close(); } } } " + line);
Dept Of CSE/BTLIT/BATCH 5
77
Dept Of CSE/BTLIT/BATCH 5
79
Dept Of CSE/BTLIT/BATCH 5
80
Figure 7.1 Amazon S3 Cloud Operations. This Screen Shot shows the first point of the project execution.
Dept Of CSE/BTLIT/BATCH 5
81
Figure 7.2 Various Options for Amazon S3 Cloud Operations. This Screen Shot displays the various operations that can be done on Amazon S3 such as Bucket Creation & Deletion, Object Upload & Download, Listing the objects in the specified bucket.
Dept Of CSE/BTLIT/BATCH 5
82
Figure 7.3 Bucket Names listed at Amazon S3. This Screen Shot displays the list of buckets already present in the Users Amazon S3 Account. This is the result of clicking the Refresh Bucket Listing from the Menu List.
Dept Of CSE/BTLIT/BATCH 5
83
Figure 7.4 Creation of New Bucket at Amazon S3. This Screen Shot displays the creation of new bucket at S3. This is the result of clicking the Create Bucket from the Menu List.
Dept Of CSE/BTLIT/BATCH 5
84
Figure 7.5 shows the updated Bucket list at Amazon S3. This Screen Shot displays the updated list showing the new bucket bgs-sjbit. This is the result of clicking the Refresh Bucket Listing from the Menu List.
Dept Of CSE/BTLIT/BATCH 5
85
Figure 7.6 Deletion of a bucket from Amazon S3. This Screen Shot display the deletion of the already existing bucket at S3, the contents of the buckets should be deleted first then the bucket can be deleted. This is the result of clicking the Delete Bucket from the Menu List.
Dept Of CSE/BTLIT/BATCH 5
86
Figure 7.7 Uploading an Object at Bucket at Amazon S3. This Screen Shot display the Uploading of objects in the bucket at S3, This is the result of clicking the Object Update from the Menu List. User can upload any files / objects on to the bucket. Figure 7.8, 7.9 & 7.10 shows the detailed process of object updating.
Dept Of CSE/BTLIT/BATCH 5
87
Dept Of CSE/BTLIT/BATCH 5
88
Figure 7.11 Listing Objects at Amazon S3 bucket. This Screen Shot display the object list of the highlighted bucket. This is the result of clicking the Object Listing from the Menu List.
Dept Of CSE/BTLIT/BATCH 5
90
Figure 7.12 Objects Listed at Bucket named Animesh. This Screen Shot display the object list of the Animesh bucket showing its objects contents, currently this bucket has only one object named n1bucket.java.
Dept Of CSE/BTLIT/BATCH 5
91
Figure 7.13 Downloading Objects from Amazon S3 bucket. This Screen Shot displays the downloading the selected objects from the bucket. This is a result of clicking of Object Download from Main Menu.
Dept Of CSE/BTLIT/BATCH 5
92
Figure 7.14 Number to nodes to get downloaded objects. This Screen Shot displays the downloading the selected objects from the bucket to the computing nodes. Here user can to specify the number of nodes to which the contents of the objects to be downloaded. This after successful execution of algorithms, all the nodes will have the complete object content.
Dept Of CSE/BTLIT/BATCH 5
93
There are three EC2 nodes executing Non Steal Algorithm, the Design of all the three EC2 nodes are shown below from figure 7.15 to 7.17, These nodes has several buttons to support the execution of the algorithm. List Button displays the Objects range for the Specified node Download Button is activated once the object range is selected & then the contents present in that range are downloaded to the files. Show Button displays the objects range of the neighboring nodes Share Button is activated once the object range of the neighboring nodes is selected & then the contents are shared among all other nodes.
Dept Of CSE/BTLIT/BATCH 5
94
There are three EC2 nodes executing Steal Algorithm, the Design of all the three EC2 nodes are shown below from figure 7.21 to 7.17, These nodes has several buttons to support the execution of the algorithm. List Button displays the Objects range for the Specified node Download Button is activated once the object range is selected & then the contents present in that range are downloaded to the files. Show Button displays the objects range of the neighboring nodes Share Button is activated once the object range of the neighboring nodes is selected & then the contents are shared among all other nodes.
Dept Of CSE/BTLIT/BATCH 5
97
Recv Button is only present at EC2 Node 1 & will receive the work request to download object content from Node 2 & Node 3. Req Button is present at both Node 2 & Node 3, which is activated to request the download from the Node 1. The range of the remaining object is send to Node 1, which download the contents and then send to the requesting node. Receive Button is present at both Node 2 & Node 3, which is activated to receive the downloaded contents from the Node 1.
Dept Of CSE/BTLIT/BATCH 5
98
Dept Of CSE/BTLIT/BATCH 5
99
Figure 7.24 EC2 Node 2 Requesting EC2 Node 1to download the specified range of objects contents
Dept Of CSE/BTLIT/BATCH 5
100
Dept Of CSE/BTLIT/BATCH 5
101
Dept Of CSE/BTLIT/BATCH 5
102
Dept Of CSE/BTLIT/BATCH 5
103
7.2 CONCLUSION
We have addressed cloud characteristics and summarized different multicast algorithms for distributed parallel and P2P systems. We have focused on Amazon EC2/S3 which is the most commonly used cloud platform, and revealed some multicast performance problems on there when using the most simple algorithm by which all EC2 compute nodes download files from S3 directly. There are two main types of problems with optimizing multicast performance on clouds
1. The network performance within clouds is dynamic. It means the network performance
changes not only between compute nodes but also especially between the cloud storage and each compute node.
Dept Of CSE/BTLIT/BATCH 5
104
Blood bank management system using amazon elastic cloud computing 2. Keeping network monitoring data and topology information up-to-date is almost
impossible. It means it is difficult to know the underlying clouds physical infrastructures and construct one or more available bandwidth maximized spanning trees, which is well known optimization technique for cluster systems. Based o n these findings, we have presented two high performance multicast algorithm. These algorithms make it possible to transfer large Amounts of data stored in S3 to multiple EC2 nodes efficiently. The proposed algorithms combine optimization ideas from multicast algorithms used in parallel systems, and they have three salient features;
(a) They can construct an overlay network on clouds without network topology
distributed systems
and P2P
information,
(b) T hey can optimize the total throughput between compute nodes dynamically, and (c) They can increase the download throughput from cloud storage by letting nodes
cooperate with each other, resembling work stealing techniques. We have implemented the two proposed algorithms and evaluated their performance on Amazon EC2/S3, The steal algorithm achieved higher throughput than the non-steal algorithm since the steal algorithm could adapt to the dynamic changes of the download throughput from S3.
Dept Of CSE/BTLIT/BATCH 5
105