VISVESVARAYA TECHNOLOGICAL UNIVERSITY

BELAGAVI - 590018, Karnataka

Synopsis
On

“Automated HTML Code Generation from Hand Drawn Images

using Machine Learning Methods”
For the requirement of 7th semester B.E in Computer Science and Engineering

Submitted By

<NAME1 > (USN)

<NAME2 >(USN)

Under the Guidance of

<Internal Guide Name >
Abstract
1. Introduction
1.1 Introduction of the Project
1.2 Scope
1.3 Objectives of the Project
1.4 Applications
2. Literature Survey
2.1 Survey on Technology
2.2 Survey on Methods
2.3 Gaps Identified
3. Requirement Specifications
3.1 Functional Requirements
3.2 Non-functional Requirements
3.3 Hardware Requirements
3.4 Software Requirements
4. Methodology
4.1 Existing System
4.2 Proposed System
 ABSTRACT
As a first step of designing of website is start to built the mock-up images for the particular
web pages by operated with the hands or using mock-up developer tools. It is efficiently used
for the developer to transferring web pages mock-up to the coding. It’s generating the
proposed system to creating the wireframe to the layout interfaces there are two techniques
mostly used first is computer vision and second is deep systematic analysis. The automatic
code generation is time reducing and cost effective. We have design structured an outline the
design.
CHAPTER-1
INTRODUCTION
1.1Introduction of the Project
Nowadays, the internet is the most important in our day to day life. Websites are presents
in every fields. The design cycle for a website opening due to creating mock-up for
separate webpage further away drawing by hand or by drawing in paint designs and
intensive mock-up formation apparatus. The mock-up images were then transformed to
HTML by software engineer. This proceeding is recast extra instant as far as the wanted
template is not get. Our main target is to equip html code out of hand drawn images. We
use convolution neural network, computer vision technique and also deep learning was
used for our proposed system. In today’s world websites review the institution, hotels,
business, people, etc. Websites are use in each and every factor. From education to
knowledge, from training to social work. At the front end of every site that concert with
the user. It is actual relevant to give a surface a certain attraction in the user, it is very
easy to use and it has sufficing advanced attribute. In other way, creating webpage which
gives active respond expertly for this it required a very tiring pathway. In the developing
of webpage, many software engineering developer are working together to designing the
front view of the webpage. Software designer built code to design the webpage basis on
the draft. The resulting webpage can convert depend on feedback received by the as in
user. For the elements built the code with same feature with page format converting
instant turn into the steps difficult. This emerges the need for expanding more improved
feature in a webpage format. The proposal of structured the webpage by crating automatic
code is very interesting as a research subject. Generation of automatic webpage minimize
coding instant, steps price as well as resource. So by this way thanking to the speedy
pattern steps, the final website is create in a very less time period. In our survey, methods
were used for automatically developing the hand drawn images by generating code for it.
It’s aim to observe the factor create the hand drawing by making encrypt the system in the
way of the webpage format.
1.2Scope
• Increasing system versatility by allowing users to create front-end designs using a
variety of image formats.
• Improving the system in order to produce more appealing designs.
• Improving system quality by adding a new function that allows users to customise the
CSS for the website's front end.

1.3Objectives of the Project

• To recognize the mock-up drawing's components and encode them according to the
web page hierarchy.
• To improve the accuracy in script generation.
• To help the web designer to design the page as per the user sketch.
• To reduce the labor cost and time in creation of website.

1.4Applications
• Web designing
• Web development
• AI
CHAPTER-2
LITERATURE SURVEY
A literature survey or a literature review in a project report shows the various
analyses and research made in the field of interest and the results already published, taking
into account the various parameters of the project and the extent of the project. Literature
survey is mainly carried out in order to analyze the background of the current project which
helps to find out flaws in the existing system & guides on which unsolved problems we can
work out. So, the following topics not only illustrate the background of the project but also
uncover the problems and flaws which motivated to propose solutions and work on this
project.
A literature survey is a text of a scholarly paper, which includes the current
knowledge including substantive findings, as well as theoretical and methodological
contributions to a particular topic. Literature reviews use secondary sources, and do not
report new or original experimental work. Most often associated with academic-oriented
literature, such as a thesis, dissertation or a peer-reviewed journal article, a literature
review usually precedes the methodology and results sectional though this is not always
the case. Literature reviews are also common in are search proposal or prospectus (the
document that is approved before a student formally begins a dissertation or thesis). Its
main goals are to situate the current study within the body of literature and to provide
context for the particular reader. Literature reviews are a basis for researching nearly
every academic field. demic field. A literature survey includes the following:

• Existing theories about the topic which are accepted universally.

• Books written on the topic, both generic and specific.

• Research done in the field usually in the order of oldest to latest.

• Challenges being faced and on-going work, if available.

Literature survey describes about the existing work on the given project. It deals with the
problem associated with the existing system and also gives user a clear knowledge on how to
deal with the existing problems and how to provide solution to the existing problems
Objectives of Literature Survey
• Learning the definitions of the concepts.

• Access to latest approaches, methods and theories.

 • Discovering research topics based on the existing research

• Concentrate on your own field of expertise– Even if another field uses the
same words, they usually mean completely.
• It improves the quality of the literature survey to exclude sidetracks–
Remember to explicate what is excluded.

2.1 Survey on Technology

Survey methodology is "the study of survey methods". As a field of applied statistics
concentrating on human-research surveys, survey methodology studies the sampling
of individual units from a population and associated techniques of survey data
collection, such as questionnaire construction and methods for improving the number
and accuracy of responses to surveys. Survey methodology targets instruments or
procedures that ask one or more questions that may or may not be answered.
Researchers carry out statistical surveys with a view towards making statistical
inferences about the population being studied; such inferences depend strongly on the
survey questions used. Polls about public opinion, public-health surveys, market-
research surveys, government surveys and censuses all exemplify quantitative
research that uses survey methodology to answer questions about a population.
Although censuses do not include a "sample", they do include other aspects of survey
methodology, like questionnaires, interviewers, and non-response follow-up
techniques. Surveys provide important information for all kinds of public-information
and research fields, such as marketing research, psychology, health-care provision and
sociology.
2.2 Survey on Methods
2.2.1 Title: Automatic HTML Code Generation from Mock-up Images Using Machine
Learning Techniques
Author: Batuhan Aşıroğlu; Büşta Rümeysa Mete; Eyyüp Yıldız; Yağız
Nalçakan; Alper Sezen
Year: 2019
Abstract:
The design cycle for a web site starts with creating mock-ups for individual
web pages either by hand or using graphic design and specialized mock-up creation
tools. The mockup is then converted into structured HTML or similar markup code by
software engineers. This process is usually repeated many more times until the
desired template is created. In this study, our aim is to automate the code generation
process from hand-drawn mock-ups. Hand drawn mock-ups are processed using
computer vision techniques and subsequently some deep learning methods are used to
implement the proposed system. Our system achieves 96% method accuracy and 73%
validation accuracy.
Methodologies: CNN
Limitations:
 It is limited to the specific dataset images.
 It is not suitable for our own images.

2.2.2 Title: Convolutional Neural Network (CNN) for Image Detection and
Recognition
Author: Rahul Chauhan; Kamal Kumar Ghanshala; R.C Joshi
Year: 2018
Abstract:
Deep Learning algorithms are designed in such a way that they mimic the
function of the human cerebral cortex. These algorithms are representations of deep
neural networks i.e. neural networks with many hidden layers. Convolutional neural
networks are deep learning algorithms that can train large datasets with millions of
parameters, in form of 2D images as input and convolve it with filters to produce the
desired outputs. In this article, CNN models are built to evaluate its performance on
image recognition and detection datasets. The algorithm is implemented on MNIST
and CIFAR-10 dataset and its performance are evaluated. The accuracy of models on
MNIST is 99.6 %, CIFAR-10 is using real-time data augmentation and dropout on
CPU unit.
Methodologies: CNN
Limitations:
 It is suitable to detect the objects only.

 It is not suitable to generate HTML scripts

2.2.3 Title: P2A: A Tool for Converting Pixels to Animated Mobile Application
User Interfaces
Author: Rahul Chauhan; Kamal Kumar Ghanshala; R.C Joshi
Year: 2018
Abstract:
Developing mobile applications is typically a labor-intensive process in which
software engineers manually re-implement in code screen designs, inter-screen
transitions, and in-screen animations developed by user interface and user experience
experts. Other engineering domains have used computer vision techniques to
automate human perception and manual data entry tasks. The P2A tool adopts
computer vision techniques for developing animated mobile applications. P2A infers
from mobile application screen designs the user interface portion of an application's
source code and other assets that are ready to be compiled and executed on a mobile
phone. Among others, inferred mobile applications contain inter-screen transitions
and in-screen animations. In our experiments on screenshots of 30 highly-ranked
third-party Android applications, the P2A-generated application user interfaces
exhibited high pixel-to-pixel similarity with their input screenshots. P2A took an
average of 26 seconds to infer in-screen animations.
Methodologies: computer vision techniques
Limitations:
 It is not suitable to generate HTML scripts.
2.2.4 Title: Machine learning based prototyping of graphical user interfaces for
mobile apps
Author: Kevin Moran; Carlos Bernal-Cárdenas; Michael Curcio; Richard
Bonett;
Year: 2018
Abstract:
It is common practice for developers of user-facing software to transform a
mock-up of a graphical user interface (GUI) into code. This process takes place both
at an application's inception and in an evolutionary context as GUI changes keep pace
with evolving features. Unfortunately, this practice is challenging and time-
consuming. In this paper, we present an approach that automates this process by
enabling accurate prototyping of GUIs via three tasks: detection, classification, and
assembly. First, logical components of a GUI are detected from a mock-up artifact
 using either computer vision techniques or mock-up metadata. Then, software
repository mining, automated dynamic analysis, and deep convolutional neural
networks are utilized to accurately classify GUI-components into domain-specific
types (e.g., toggle-button). Finally, a data-driven, K-nearest-neighbors algorithm
generates a suitable hierarchical GUI structure from which a prototype application can
be automatically assembled. We implemented this approach for Android in a system
called ReDraw. Our evaluation illustrates that ReDraw achieves an average GUI-
component classification accuracy of 91 percent and assembles prototype applications
that closely mirror target mock-ups in terms of visual affinity while exhibiting
reasonable code structure. Interviews with industrial practitioners illustrate ReDraw's
potential to improve real development workflows.
Methodologies: CNN and K-nearest neighbors.
Limitations:
 It is not suitable to detect objects.
 It is not suitable to generate HTML scripts.
2.2.5 Title: Handwritten Recognition Using SVM, KNN and Neural Network
Author: Norhidayu Abdul Hamid, Nilam Nur Amir Sjarif
Year: 2018
Abstract:
Handwritten recognition (HWR) is the ability of a computer to receive and
interpret intelligible handwritten input from source such as paper documents,
photographs, touch-screens and other devices. In this paper we will using three (3)
classification t o re cognize the handwritten which is SVM, KNN and Neural
Network.
Methodologies: SVM, KNN, Neural Network .
Limitations:
 It is not suitable to detect objects.
 It is not suitable to generate HTML scripts.

2.3 Gaps Identified

 Some of the existing paper limited to the specific dataset images.
 Some of the existing paper is not suitable for our own images.
 Some of the existing papers is not suitable to generate HTML scripts.
CHAPTER-3
SYSTEM REQUIREMENTS AND SPECIFICATIONS
3.1 Functional Requirements
1. Create a desktop application using python tkinter framework.
2. User should load the sketch.
3. System will extract the image features and detect the objects.
4. System will apply the RCNN model to recognize the HTML components.
5. System will generate HTML code for recognized component using RCNN.
6. Application should accurately generate HTML code from the user’s input sketchs
automatically.
3.2 Non-functional Requirements
These are requirements that are not functional in nature, that is, these are constraints within
which the system must work.
 The program must be self-contained so that it can easily be moved from one
Computer to another. It is assumed that network connection will be available on the
computer on which the program resides.
 Capacity, scalability and availability.
The system shall achieve 100 per cent availability at all times.
The system shall be scalable to support additional clients and volunteers.
 Maintainability.
The system should be optimized for supportability, or ease of maintenance as far as
possible. This may be achieved through the use documentation of coding standards,
naming conventions, class libraries and abstraction.
 Randomness, verifiability and load balancing.
The system should be optimized for supportability, or ease of maintenance as far as
possible. This may be achieved through the use documentation of coding standards,
naming conventions, class libraries and abstraction. It should have randomness to
check the nodes and should be load balanced.
3.3 Hardware Requirements
• Processor Type : Intel i5
• RAM :4 GB RAM
• SD disk : 64 GB
• Input device : Mouse, Keyboard

3.3.1 CPU- INTEL CORE i5

Fig 3.1 INTEL CORE i5

Intel Core is a brand name that Intel uses for various mid-range to high-end consumer
and business microprocessors. As of 2015 the current line up of Core processors
included the Intel Core i7, Intel Core i5, and Intel Core i3. 5th generation Intel® Core™
i5 processors empower new innovations like Intel® Real Sense™ technology—bringing
you features such as gesture control, 3D capture and edit, and innovative photo and
video capabilities to your devices. Enjoy stunning visuals, built-in security, and an
automatic burst of speed when you need it with Intel® Turbo Boost Technology 2.0.
3.3.2 RAM

Fig 3.2 RAM 4 GB

When you load up an application on to your computer it loads into your available RAM
memory. It is very quick type of memory. The more programs you load up, the more
RAM is taken up. At the point where you have loaded up enough apps to take up all your
free available physical RAM, your OS will create a swap-file on your hard drive. This
file is used as a reserve for all additional apps you run.
The trouble with that is that hard drives are a lot slower to read and write from than
RAM memory is. Therefore, your computer will perform much slower at that point.
Although new generation of SSD hard drives are much faster than your traditional spinning
drive, it is still best to have enough RAM available. If you are using Windows and want to
want to know how much RAM you are using up, you can right click on task bar, then select
start "Task Manager" and on the "performance" tab you will see a green bar indicating
"Memory".

3.4 Software Requirements

 Operating System : Raspbian 64-bit

 Technology : Python
 IDE : PythonIDLE
 Python Version : Python 3.9
3.4.1 Introduction to Python

Python is a widely used general-purpose, high level programming language. It was

created by Guido van Rossum in 1991 and further developed by the Python Software
Foundation. It was designed with an emphasis on code readability, and its syntax allows
programmers to express their concepts in fewer lines of code. Python is a programming
language that lets you work quickly and integrate systems more efficiently.

There are two major Python versions: Python 2 and Python 3. Both are quite different.

Beginning with Python programming:

1) Finding an Interpreter:
 Before we start Python programming, we need to have an interpreter to interpret and
run our programs.

Windows: There are many interpreters available freely to run Python scripts like IDLE
(Integrated Development Environment) that comes bundled with the Python software
downloaded from http://python.org/.

Linux: Python comes preinstalled with popular Linux distros such as Ubuntu and
Fedora. To check which version of Python you’re running, type “python” in the
terminal emulator. The interpreter should start and print the version number.

macOS: Generally, Python 2.7 comes bundled with macOS. You’ll have to manually
install Python 3 from http://python.org/.

3.4. Importance of Python

Python is a high-level, interpreted, interactive, and object-oriented scripting language.
Python was designed to be highly readable which uses English keywords frequently
whereas other languages use punctuation and it has fewer syntactical constructions
than other languages.
It is used in:
1. Software Development
2. Web Development
3. System Scripting
4. Mathematics

Python is Interpreted

It means that each line is processed one by one at runtime by the interpreter and you do not
need to compile your program before executing it.
 Python is Interactive

It means that you can actually sit at a Python prompt and interact with the interpreter directly,
to write and execute your programs.

Python is Object-Oriented

Python supports the Object-Oriented style or technique of programming that encapsulates

code within objects.
 Python is Beginner's Language

Python is an excellent language for beginning programmers and facilitates the construction of
a wide range of programs ranging from simple text processing to web browsers to games.
Python does not have pointers, which is one of the main challenges that many of us have
encountered when programming.

Easy-to-maintain

Python's success is that its source code is fairly easy-to-maintain. One reason for that is, it is
read and written like a lot of everyday English.

A Broad Standard Library

One of Python's greatest strengths is the bulk of the library, which makes it very portable and
cross-platform compatible. Python has libraries for almost everything one can think of.
 Interactive Mode

Support for an interactive mode in which you can enter results from a terminal right to the
language, allowing interactive testing and debugging of snippets of code.
 Portable

Python can run on a wide variety of hardware platforms and has the same interface on all
platforms. You can run the same python program on Windows, Linux, Mac, Raspberry Pi,
Mango Pi, Android, etc.

Extendable

You can add low-level modules to the Python interpreter. These modules enable
programmers to add to or customize their tools to be more efficient. Generally, we do that
using the PIP command.

Databases

Python provides interfaces to all major commercial databases. It has packages to

communicate with SQL, NoSQL, etc. databases, ranging from MongoDB to MySQL.

GUI Programming

Python supports GUI applications that can be created and ported to many system calls,
libraries, and windows systems, such as Windows, Macintosh, and the X Window system of
Unix. It has libraries like Tkinter, WxPython, etc.
 Scalable

Python provides a better structure and support for large programs than shell scripting.

Apart from the above-mentioned features, Python has a big list of good features, few of them
are-

 Support for functional and structured programming methods as well as OOP.

 It can be used as a scripting language or can be compiled to byte-code for building
large applications.
 Very high-level dynamic data types and supports dynamic type checking.
 Supports automatic garbage collection.
 It can be easily integrated with C, C++, COM, ActiveX, CORBA, and Java.
CHAPTER-4
METHODOLOGY
4.1EXISTING SYSTEM
A large gap occurs between graphic designers' conceptual designs and functional UI code
while building the UI code of a mobile application. Programmers often close this gap by
rewriting the conceptual drawings in code, which is time-consuming and costly. To fill
this need, we pioneered the first method for automatically reverse engineering mobile app
user interfaces (REMAUI). REMAUI uses computer vision and OCR methods to identify
UI components on a given input image. REMAUI-generated UIs were close to the
originals in our tests on 488 screenshots of over 100 major third-party programmes. We
intend to expand the export process to more platforms, including iOS and cross-platform
JavaScript frameworks. Currently, REMAUI separates each input screen into its own
programme. We want to provide a graphical notation that users can use to connect many
inputs screen designs, which REMAUI can then utilize to produce a single application
with several screens and matching screen transitions. REMAUI will be integrated with
tools that produce mobile app functionality using keyword-based code search or high-
level models.) We want to index a screenshot corpus by executing REMAUI on it and
saving the intermediate results of REMAUI. By exposing this index through a query
interface, a user might look for screenshots based on their structure and features.
Building classifiers for picture detection and recognition on multiple datasets using
various machine learning had advanced significantly in recent years. Deep learning, in
particular, has improved accuracy across a variety of datasets. The below are summaries
of some of the works. SVM (support vector machines), KNN (K-nearest Neighbour), and
CNN were used to evaluate the performance on MNIST datasets (convolutional neural
networks). On that platform, the Multilayer Perceptron didn't function well since it didn't
attain the global minimum but rather stayed caught in the local optimum and couldn't
reliably distinguish digits 9 and 6.Other classifiers performed well, and it was decided
that by building the model on the Keras platform, performance on CNN may be
enhanced. On the MNIST dataset, implement DNN (deep neural networks), DBF (deep
belief networks), and CNN (convolutional neural networks) and compare the results.
According to the research, DNN fared the best, with an accuracy of 98.08 percent, while
others had error rates and execution times that differed.
4.2PROPOSED SYSTEM
Our main aim is to convert the hand drawn mockups which contain textbox, buttons,
picture into the HTML code to make a website template or front end of our websites
according to hand-drawn mock-up which is conceptual drawing. In order to convert this
hand –drawn image into the HTML code to frontend template we used computer vision
technique CNN model, object recognition, cropping etc. This work on object detection
algorithm to detect the component from the hand-drawn image. This work on object
cropping algorithm to crop the object like button, textbox, dropdown etc. This works on
object reorganization algorithm by using CNN model to train our purposed system by
using data set we tested it on windows platform. It work successfully & result have been
obtained. The author work on HTML builder algorithm which convert the detected object
using CNN model by object reorganization algorithm to the HTML code using HTML
builder algorithm.

Fig.4.2 System Architecture

System architecture tells us how the system works in the view of the user. It consists of 4
main modules:
• Obtaining the UI Sketch (Uploading of the screenshot of the Sketch)
• In this module used will draw their sketch and upload the image to our
application.
• Region of Interest
• In this module system will extract the region of interest from the given image.
• Element Recognition
• In this module system will apply the RCNN model to recognize the HTML
components.
• Code Generation
• In this module system will generate HTML code for recognized component
using RCNN.
Reference:
[1]. Automatic HTML Code Generation from Mock-up Images Using Machine Learning
Techniques978-1-7281- 1013-4/19/$31.00 © 2019 IEEE
[2]. Convolutional Neural Network (CNN) for Image Detection and Recognition 978-1-5386-
6373-8/18/$31.00 ©2018 IEEE
[3]. Reverse Engineering Mobile Application User Interfaces with REMAUI 978-1-5090-
0025-8/15 $31.00 © 2015 IEEE DOI 10.1109/ASE.2015.32
[4]. pix2code: Generating Code from a Graphical User Interface Screenshot
arXiv:1705.07962v2 [cs. LG] 19 Sep 2017
[5]. Norhidayu binti Abdul Hamid, Nilam Nur Binti Amir Sjarif, “Handwritten Recognition
Using SVM, KNN and Neural Network”, www.arxiv.org/ftp/arxiv/papers/1702/1702.00723
[6]. Mahmoud M. Abu Ghosh; Ashraf Y. Maghari, “A Comparative Study on Handwriting
Digit Recognition Using Neural Networks”, IEEE, 2017
[7]. T. A. Nguyen and C. Csallner, “Reverse Engineering Mobile Application User Interfaces
with REMAUI (T),” in 2015 30th IEEE/ACM International Conference on Automated
Software Engineering (ASE). IEEE, nov 2015, pp. 248–259. [Online]. Available: http:
//ieeexplore.ieee.org/document/7372013/
[8]. S. Natarajan and C. Csallner, “P2A: A Tool for Converting Pixels to Animated Mobile
Application User Interfaces,” Proceedings of the 5th International Conference on Mobile
Software Engineering and Systems - MOBILESoft ’18, pp. 224–235, 2018. [Online].
Available: http://dl.acm.org/citation.cfm?doid=31 97231.3197249.