Webapp With Golang Anti Textbook
Webapp With Golang Anti Textbook
Table of Contents
Introduction
Tools
Go basic knowledge
1.1
2
Hello, Go
2.1
Go foundation
2.2
2.3
struct
2.4
Object-oriented
2.5
interface
2.6
Concurrency
2.7
Summary
2.8
General
Web Programming Basics
Implementation
3
3.1
4
4.1
Database Handling
4.2
Webapp Example
4.3
Form handling
Uploading Files
5
5.1
Templates
User Authentication
Routing
Contributors
10
webapp-with-golang-anti-textbook
Code
The book comes with corresponding code, please use it to understand the book completely,
the book is just plain old theory if you do not check the code out. The code is present in the
code folder in this repository itself.
Contributing
I don't profess to be a God of either Go or webdev or anything in general, and I don't claim
that this is the best book for learning how to build web appplications with Go, but I do believe
that good things happen when people collaborate, so pull requests are not only appreciated,
but they are welcome.
Philosophy
Introduction
webapp-with-golang-anti-textbook
Through this book we want to teach how to develop web applications in Go. We expect
the reader to know the basics of Go but we assume the reader knows nothing about
how to write web applications
The book shall comprise of chapters, if the topic is huge and doesn't fit into one chapter,
then we split into multiple chapters, if possible.
Each chapter should be split into logical parts or sections with a meaningful title which'll
teach the reader something.
Every concept should be accompanied by the Go code (if there is any), for sneak peek
type sections write the Go pseudo code, writing just the necessary parts of the code and
keeping everything else.
The code shouldn't be more than 80 characters wide because in the PDF versions of
the book the code is invisible.
Brevity is the soul of wit, so keep the description as small as possible. But this doesn't
mean that we should just assume that the reader knows the concept and skip it in such
cases do explain the concept.
In the todo list manager which we are creating, we'll strive to implement as much
functionality as possible to give a taste of practical Go programming to the reader, but
we should mention as a note the other way, suppose you re-implement a function like
ParseGlob by listing all html files and using ParseFiles to parse them, we should
License:
Book License: CC BY-SA 3.0 License
Links
Next section: Installation and Tools
Introduction
webapp-with-golang-anti-textbook
Installation
If you know about installation or have installed Go, you can skip to Tools.
This chapter is taken from install page verbatim, except for the changes to be made to adapt
to this book's styling format.
System requirements
Go binary distributions are available for these supported operating systems and
architectures. Please ensure your system meets these requirements before proceeding. If
your OS or architecture is not on the list, you may be able to install from source or use gccgo
instead
Operating system
Architectures
Notes
FreeBSD 8-STABLE
or later
amd64
amd64, 386,
arm
Mac OS X 10.7 or
later
amd64
Windows XP or later
amd64, 386
webapp-with-golang-anti-textbook
Choose the archive file appropriate for your installation. For instance, if you are installing Go
version 1.2.1 for 64-bit x86 on Linux, the archive you want is called go1.2.1.linuxamd64.tar.gz.
webapp-with-golang-anti-textbook
If you chose a directory other than c:\Go , you must set the GOROOT environment variable
to your chosen path.
Add the bin subdirectory of your Go root (for example, c:\Go\bin ) to your PATH
environment variable. Setting environment variables under Windows
Under Windows, you may set environment variables through the "Environment Variables"
button on the "Advanced" tab of the "System" control panel. Some versions of Windows
provide this control panel through the "Advanced System Settings" option inside the
"System" control panel. Test your installation
Check that Go is installed correctly by setting up a workspace and building a simple
program, as follows.
Create a directory to contain your workspace, $HOME/work for example, and set the
GOPATH environment variable to point to that location.
$ export GOPATH=$HOME/work
You should put the above command in your shell startup script ( $HOME/.profile for
example) or, if you use Windows, follow the instructions above to set the GOPATH
environment variable on your system.
Next, make the directories src/github.com/user/hello inside your workspace (if you use
GitHub, substitute your user name for user), and inside the hello directory create a file
named hello.go with the following contents:
package main
import "fmt"
func main() {
fmt.Printf("hello, world\n")
}
The above command will put an executable command named hello (or hello.exe) inside the
bin directory of your workspace. Execute the command to see the greeting:
$ $GOPATH/bin/hello
webapp-with-golang-anti-textbook
hello, world
If you see the "hello, world" message then your Go installation is working.
Before rushing off to write Go code please read the How to Write Go Code document, which
describes some essential concepts about using the Go tools. Uninstalling Go
To remove an existing Go installation from your system delete the go directory. This is
usually /usr/local/go under Linux, Mac OS X, and FreeBSD or c:\Go under Windows.
You should also remove the Go bin directory from your PATH environment variable. Under
Linux and FreeBSD you should edit /etc/profile or $HOME/.profile . If you installed Go
with the Mac OS X package then you should remove the /etc/paths.d/go file. Windows
users should read the section about setting environment variables under Windows. Getting
help
For real-time help, ask the helpful gophers in #go-nuts on the Freenode IRC server.
The official mailing list for discussion of the Go language is Go Nuts.
Report bugs using the Go issue tracker.
Links
Directory
Next section: Tools
webapp-with-golang-anti-textbook
Tools
For html: Brackets, a text editor for the web by Adobe. For Go: Any IDE of your choice which
has a Go language plugin.
gofmt
gofmt is a built in formatting tool. This simple tool formats our gocode according to the
standard format, so it doesn't matter who wrote it because once we want to check in the
code, we can format it, this saves a lot of time. Almost all IDEs provide the gofmt on save
feature, then you can use this gofmt -w Tasks , where Tasks is my folder name. This code
will run gofmt on each go file in my project. If you are in Tasks folder then you can do this
gofmt -w .
We also have a go fmt tool along with gofmt, it runs gofmt -w on all files present in the
current folder.
It is a good practice to run gofmt on our code before committing to version control since it
standardizes formatting of the entire codebase.
godoc
Godoc is a built in documentation tool, when we write gocode, we are expected to give
comments to each exported function and pretty much every important place because the
exported functions are to be used by some random third party, they should have a comment
explaining their use and the syntax. Also godoc scans your $GOPATH and generates
documentation on a server, the golang.org website is godoc running in production. When
you start the godoc server, you'll notice that it gives you a html front end to the
documentation of all your Go code,
The -v flag is for verbose, for some reason godoc doesn't notify the user by printing on the
terminal that "server started on port 8080", so we have to use the -v mode to get to know
when the server started running, because it is rather annoying to keep opening the
localhost:6060 URL ten times when in the fifteenth time the server has started. If you want
to jump to any specific package, go to /pkg/name, for instance if you want to go to the
documentation of net/http the link is localhost:6060/pkg/net/http
Usage: godoc -http=:6060 -v
go test
Tools
webapp-with-golang-anti-textbook
go test is the testing tool for Go. If you are writing the test cases for main.go the file
should be called main_test.go . We'll see more about testing in later chapters where we
start to write Go code.
go build
This is the build tool in Go. When you run it on your project root it'll build the project, the
binary will be named after the name of your root folder, for instance if I build my Tasks
application, the since the folder is Tasks in my src tree, the binary will be named Tasks .
You can instruct go build to create a binary of any name by running go build -o task so it'll
create a binary with the name of task. A go package can have only one main file, but a
project can have multiple go files, you can also give the name of the file you want to build as
go build file.go
If there are no quirks on the libraries, should give you a binary for the respective platforms.
go run
As we have seen, we can use go build to generate a binary, what if we just want to run it
once but don't care for a binary. This is the reason go run was created, it builds your project
and runs it in the /tmp directory so a binary isn't left in your project directory.
go get
This is a way to install packages from various websites like github, bitbucket, launchpad.
What go get does is it clones the repo in your $GOPATH/src folder and runs go install on
the package, which in turn creates a .a file and puts it in the respective $GOPATH/pkg
folder tree
go install
You can install your applications in your $GOPATH/bin using this command. It builds the
project and places a binary directly in the $GOPATH/bin . If you want to use the installed
applications then you need to add the $GOPATH/bin in your path.
Tools
10
webapp-with-golang-anti-textbook
Links
Directory
Previous section: Installation
For those who want to learn Go
Go Programming Basics
If you are comfortable with Go, you can skip the next section which is for those who are new
to Go.
Web Programming basics
Tools
11
webapp-with-golang-anti-textbook
2 Go Programming Basics
Go is a compiled system programming language, and it belongs to the C-family. However, its
compilation speed is much faster than other C-family languages. It has only 25 keywords...
even less than the 26 letters of the English alphabet! Let's take a look at these keywords
before we get started.
break default func interface select
case defer go map struct
chan else goto package switch
const fallthrough if range type
continue for import return var
In this chapter, I'm going to teach you some basic Go knowledge. You will find out how
concise the Go programming language is, and the beautiful design of the language.
Programming can be very fun in Go. After we complete this chapter, you'll be familiar with
the above keywords.
Links
Directory
Previous chapter: Chapter 1 Summary
Next section: "Hello, Go"
Go basic knowledge
12
webapp-with-golang-anti-textbook
or for python should we use 4 spaces or 6 spaces or a tab or two tabs and other user
preferences.
While this might seem to be a shallow problem at the top, but when the codebase grows and
more and more people are working on the same code base, then it is difficult to maintain the
code's "beauty", if you know python then you might be aware of PEP8, which is a set of
guidelines about how to write elegant code. We live in a world where robots can drive a car,
so we shouldn't just write code, we should write elegant code.
For other languages there are many variables when it comes to writing code, every
language is good for its use case, but Go is a little special in that turf because it was
designed at a company which is the very synonym of the Internet (and distributed
computing), typically the flow of writing code goes from Python to Java to C++ for
optimization purposes, but the problem is that almost all languages which are widely in use
right now were written decades ago when 1GB storage costed a lot as compared to now,
where storage and computing has gotten cheap. Computers are getting multiples cores
these days and the "old languages" don't harness concurrency in a way that go does, not
because those languages are bad, but simply because that usecase wasn't relevant when
the languages evolved.
Hello, Go
13
webapp-with-golang-anti-textbook
So to mitigate all the problems that Google faced with the current tools, they wrote a
systems language called Go, which you are about to learn! There are many many
advantages to using golang, and there might be disadvantages too for every coin has both
sides. But significant improvements in places like code formatting, since they desinged the
language in such a way that there won't be wars on how to format code, the gocode written
by anyone in the world (assuming they know and use gofmt ) will look exactly the same, this
won't seem to matter until you work in a team! also when the company uses automated code
review or some other fancy technique then in other languages which don't have strict and
standard formatting rules then the code might get screwed up, but not in go!
Go was designed with concurrency in mind, please note that parallelism != concurrency,
there is an amazing post by Rob Pike on the golang blog, blog.golang.org, you will find it
there, it is worth a read.
Another very important change that go has brought in programming that I personally love is
the concept of GOPATH , gone are the days when you had to create a folder called code and
then create workspaces for eclipse and what not, not you have to keep one folder tree for go
code and it'll be updated by the package manager automatically. Also under the code we are
recommended to create folders with either a custom domain or the github domain, for
example I created a task manager using golang so I created a set of folders
~/go/src/github.com/thewhitetulip/Tasks
Note:
In *nix systems ~ stands for home directory, which is the windows equivalent of
C:\\Users\\username
The ~/go/ is the universe for the go code in your machine, it is just a significant
improvement over other languages so we can store the code efficiently without hassles, it
might seem strange at first, but it does make a lot of sense over the ridiculous package
names some other languages use like reverse domains.
note: along with src there are two folders pkg which is for packages and bin which is for
binary
This GOPATH advantage isn't just restricted to storing code in particular folder, but when you
have created five packages for your project then you don't have to import them like "import
./db" , you can give it import "github.com/thewhitetulip/Tasks/db" , so while doing a go
get on my repo, the go tool will find the package from github.com/... path if it wasn't
Hello, Go
14
webapp-with-golang-anti-textbook
While some complain that go creators have ignored all language research done since the
past 30yrs, well, it might be true, but then again you can' create a product or a language
which everyone will fall in love with, there are always some or the other use cases or
constraints which the creators should consider, and considering all the advantages at least
for web development I do not think any language gets close to the advantages which go
has even if you ignore all that I said above, go is a compiled language which means in
production you'll not setup a JVM or a virtualenv you will have a single static binary! And
like an icing on a cake, all the modern libraries are in the standard library, like the http ,
which is a major advantage, which is the reason you can create webapps in golang without
using a third party web framework
2.1 Hello, Go
Before we start building an application in Go, we need to learn how to write a simple
program. You can't expect to build a building without first knowing how to build its
foundation. Therefore, we are going to learn the basic syntax to run some simple programs
in this section.
Program
According to international practice, before you learn how to program in some languages, you
will want to know how to write a program to print "Hello world".
Are you ready? Let's Go!
package main
import "fmt"
func main() {
fmt.Printf("Hello, world or or or \n")
}
Explanation
Hello, Go
15
webapp-with-golang-anti-textbook
One thing that you should know in the first is that Go programs are composed by package .
package <pkgName> (In this case is package main ) tells us this source file belongs to main
package, and the keyword main tells us this package will be compiled to a program instead
of package files whose extensions are .a .
Every executable program has one and only one main package, and you need an entry
function called main without any arguments or return values in the main package.
In order to print Hello, world , we called a function called Printf . This function is coming
from fmt package, so we import this package in the third line of source code, which is
import "fmt"
The way to think about packages in Go is similar to Python, and there are some advantages:
Modularity (break up your program into many modules) and reusability (every module can be
reused in many programs). We just talked about concepts regarding packages, and we will
make our own packages later.
On the fifth line, we use the keyword func to define the main function. The body of the
function is inside of {} , just like C, C++ and Java.
As you can see, there are no arguments. We will learn how to write functions with arguments
in just a second, and you can also have functions that have no return value or have several
return values.
On the sixth line, we called the function Printf which is from the package fmt . This was
called by the syntax <pkgName>.<funcName> , which is very like Python-style.
As we mentioned in chapter 1, the package's name and the name of the folder that contains
that package can be different. Here the <pkgName> comes from the name in package
<pkgName> , not the folder's name.
You may notice that the example above contains many non-ASCII characters. The purpose
of showing this is to tell you that Go supports UTF-8 by default. You can use any UTF-8
character in your programs.
Each go file is in some package, and that package should be a distint folder in the GOPATH,
but main is a special package which doesn't require a main folder. This is one aspect which
they left out for standardization! But should you choose to make a main folder then you have
to ensure that you run the binary properly. Also one go code can't have more than one
main go file.
~/go/src/github.com/thewhitetulip/Tasks/main $ go build
~/go/src/github.com/thewhitetulip/Tasks $ ./main/main
Hello, Go
16
webapp-with-golang-anti-textbook
the thing here is that when your code is using some static files or something else, then you
ought to run the binary from the root of the application as we see in the second line above, I
am running the main binary outside the main package, sometimes you might wonder why
your application isn't working then this might be one of the possible problems, please keep
this in mind.
One thing you will notice here is that go doesn't see to use semi colons to end a statement,
well, it does, just there is a minor catch, the programmer isn't expected to put semi colons,
the compiler adds semi colons to the gocode when it compiles which is the reason that this
(thankfully!) is a syntax error
func main ()
{
}
because the compiler adds a semi colon at the end of main() which is a syntax error and
as stated above, it helps avoid religious wars, i wish they combine vim and emacs and
create a universal editor which'll help save some more wars! But for now we'll learn Go.
Conclusion
Go uses package (like modules in Python) to organize programs. The function main.main()
(this function must be in the main package) is the entry point of any program. Go
standardizes language and most of the programming methodology, saving time of
developers which they'd have wasted in religious wars. There can be only one main
package and only one main function inside a go main package. Go supports UTF-8
characters because one of the creators of Go is a creator of UTF-8, so Go has supported
multiple languages from the time it was born.
Links
Directory
Previous section: Go basic knowledge
Next section: Go foundation
Hello, Go
17
webapp-with-golang-anti-textbook
2.2 Go foundation
In this section, we are going to teach you how to define constants, variables with elementary
types and some skills in Go programming.
Define variables
There are many forms of syntax that can be used to define variables in Go.
The keyword var is the basic form to define variables, notice that Go puts the variable type
after the variable name.
Do you think that it's too tedious to define variables use the way above? Don't worry,
because the Go team has also found this to be a problem. Therefore if you want to define
variables with initial values, we can just omit the variable type, so the code will look like this
instead:
Go foundation
18
webapp-with-golang-anti-textbook
/*
Define three variables without type "type", and initialize their values.
vname1 is v1vname2 is v2vname3 is v3
*/
var vname1, vname2, vname3 = v1, v2, v3
Well, I know this is still not simple enough for you. Let's see how we fix it.
/*
Define three variables without type "type" and without keyword "var", and initialize their values.
vname1 is v1vname2 is v2vname3 is v3
*/
vname1, vname2, vname3 := v1, v2, v3
Now it looks much better. Use := to replace var and type , this is called a brief
statement. But wait, it has one limitation: this form can only be used inside of functions. You
will get compile errors if you try to use it outside of function bodies. Therefore, we usually
use var to define global variables.
_ (blank) is a special variable name. Any value that is given to it will be ignored. For
example, we give 35 to b , and discard 34 .( This example just show you how it
works. It looks useless here because we often use this symbol when we get function
return values. )
_, b := 34, 35
If you don't use variables that you've defined in your program, the compiler will give you
compilation errors. Try to compile the following code and see what happens.
package main
func main() {
var i int
}
Constants
So-called constants are the values that are determined during compile time and you cannot
change them during runtime. In Go, you can use number, boolean or string as types of
constants.
Go foundation
19
webapp-with-golang-anti-textbook
More examples.
const Pi = 3.1415926
const i = 10000
const MaxThread = 10
const prefix = "astaxie_"
Elementary types
Boolean
In Go, we use bool to define a variable as boolean type, the value can only be true or
false , and false will be the default value. ( You cannot convert variables' type
Numerical types
Integer types include both signed and unsigned integer types. Go has int and uint at the
same time, they have same length, but specific length depends on your operating system.
They use 32-bit in 32-bit operating systems, and 64-bit in 64-bit operating systems. Go also
has types that have specific length including rune , int8 , int16 , int32 , int64 , byte ,
uint8 , uint16 , uint32 , uint64 . Note that rune is alias of int32 and byte is alias of
uint8 .
One important thing you should know that you cannot assign values between these types,
this operation will cause compile errors.
Go foundation
20
webapp-with-golang-anti-textbook
var a int8
var b int32
c := a + b
Although int32 has a longer length than int8, and has the same type as int, you cannot
assign values between them. ( c will be asserted as type int here )
Float types have the float32 and float64 types and no type called float . The latter one
is the default type if using brief statement.
That's all? No! Go supports complex numbers as well. complex128 (with a 64-bit real and
64-bit imaginary part) is the default type, if you need a smaller type, there is one called
complex64 (with a 32-bit real and 32-bit imaginary part). Its form is RE+IMi , where RE is
real part and IM is imaginary part, the last i is the imaginary number. There is a example
of complex number.
var c complex64 = 5+5i
//output: (5+5i)
fmt.Printf("Value is: %v", c)
String
We just talked about how Go uses the UTF-8 character set. Strings are represented by
double quotes "" or backticks `` .
// sample code
var frenchHello string // basic form to define string
var emptyString string = "" // define a string with empty string
func test() {
no, yes, maybe := "no", "yes", "maybe" // brief statement
japaneseHello := "Ohaiou"
frenchHello = "Bonjour" // basic form of assign values
}
It's impossible to change string values by index. You will get errors when you compile the
following code.
var s string = "hello"
s[0] = 'c'
What if I really want to change just one character in a string? Try the following code.
Go foundation
21
webapp-with-golang-anti-textbook
s := "hello"
c := []byte(s) // convert string to []byte type
c[0] = 'c'
s2 := string(c) // convert back to string type
fmt.Printf("%s\n", s2)
and also.
s := "hello"
s = "c" + s[1:] // you cannot change string values by index, but you can get values instead.
fmt.Printf("%s\n", s)
Error types
Go has one error type for purpose of dealing with error messages. There is also a
package called errors to handle errors.
err := errors.New("emit macho dwarf: elf header corrupted")
if err != nil {
fmt.Print(err)
}
Go foundation
22
webapp-with-golang-anti-textbook
Some skills
Define by group
If you want to define multiple constants, variables or import packages, you can use the
group form.
Basic form.
import "fmt"
import "os"
const i = 100
const pi = 3.1415
const prefix = "Go_"
var i int
var pi float32
var prefix string
Group form.
import(
"fmt"
"os"
)
const(
i = 100
pi = 3.1415
prefix = "Go_"
)
var(
i int
pi float32
prefix string
)
Unless you assign the value of constant is iota , the first value of constant in the group
const() will be 0 . If following constants don't assign values explicitly, their values will be
the same as the last one. If the value of last constant is iota , the values of following
constants which are not assigned are iota also.
Go foundation
23
webapp-with-golang-anti-textbook
iota enumerate
Go has one keyword called iota , this keyword is to make enum , it begins with 0 ,
increased by 1 .
const(
x = iota // x == 0
y = iota // y == 1
z = iota // z == 2
w // If there is no expression after the constants name, it uses the last expression,
//so it's saying w = iota implicitly. Therefore w == 3, and y and z both can omit "= iota" as wel
)
const v = iota // once iota meets keyword `const`, it resets to `0`, so v = 0.
const (
e, f, g = iota, iota, iota // e=0,f=0,g=0 values of iota are same in one line.
)
Some rules
The reason that Go is concise because it has some default behaviors.
Any variable that begins with a capital letter means it will be exported, private otherwise.
The same rule applies for functions and constants, no public or private keyword
exists in Go.
in [n]type , n is the length of the array, type is the type of its elements. Like other
languages, we use [] to get or set element values within arrays.
Go foundation
24
webapp-with-golang-anti-textbook
Because length is a part of the array type, [3]int and [4]int are different types, so we
cannot change the length of arrays. When you use arrays as arguments, functions get their
copies instead of references! If you want to use references, you may want to use slice .
We'll talk about later.
It's possible to use := when you define arrays.
a := [3]int{1, 2, 3} // define an int array with 3 elements
b := [10]int{1, 2, 3}
// define a int array with 10 elements, of which the first three are assigned.
//The rest of them use the default value 0.
c := [...]int{4, 5, 6} // use `` to replace the length parameter and Go will calculate it for you.
You may want to use arrays as arrays' elements. Let's see how to do this.
// define a two-dimensional array with 2 elements, and each element has 4 elements.
doubleArray := [2][4]int{[4]int{1, 2, 3, 4}, [4]int{5, 6, 7, 8}}
// The declaration can be written more concisely as follows.
easyArray := [2][4]int{{1, 2, 3, 4}, {5, 6, 7, 8}}
slice
In many situations, the array type is not a good choice -for instance when we don't know
how long the array will be when we define it. Thus, we need a "dynamic array". This is called
slice in Go.
slice is not really a dynamic array . It's a reference type. slice points to an underlying
array whose declaration is similar to array , but doesn't need length.
Go foundation
25
webapp-with-golang-anti-textbook
// just like defining an array, but this time, we exclude the length.
var fslice []int
slice can redefine existing slices or arrays. slice uses array[i:j] to slice, where i is
the start index and j is end index, but notice that array[j] will not be sliced since the
length of the slice is j-i .
// define an array with 10 elements whose types are bytes
var ar = [10]byte {'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'}
// define two slices with type []byte
var a, b []byte
// 'a' points to elements from 3rd to 5th in array ar.
a = ar[2:5]
// now 'a' has elements ar[2],ar[3] and ar[4]
// 'b' is another slice of array ar
b = ar[3:5]
// now 'b' has elements ar[3] and ar[4]
Notice the differences between slice and array when you define them. We use [] to
let Go calculate length but use [] to define slice only.
Their underlying data structure.
The second index will be the length of slice if omitted, ar[n:] equals to
ar[n:len(ar)] .
You can use ar[:] to slice whole array, reasons are explained in first two statements.
More examples pertaining to slice
Go foundation
26
webapp-with-golang-anti-textbook
// define an array
var array = [10]byte{'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j'}
// define two slices
var aSlice, bSlice []byte
// some convenient operations
aSlice = array[:3] // equals to aSlice = array[0:3] aSlice has elements a,b,c
aSlice = array[5:] // equals to aSlice = array[5:10] aSlice has elements f,g,h,i,j
aSlice = array[:] // equals to aSlice = array[0:10] aSlice has all elements
// slice from slice
aSlice = array[3:7] // aSlice has elements d,e,f,glen=4cap=7
bSlice = aSlice[1:3] // bSlice contains aSlice[1], aSlice[2], so it has elements e,f
bSlice = aSlice[:3] // bSlice contains aSlice[0], aSlice[1], aSlice[2], so it has d,e,f
bSlice = aSlice[0:5] // slice could be expanded in range of cap, now bSlice contains d,e,f,g,h
bSlice = aSlice[:] // bSlice has same elements as aSlice does, which are d,e,f,g
slice is a reference type, so any changes will affect other variables pointing to the same
slice or array. For instance, in the case of aSlice and bSlice above, if you change the
value of an element in aSlice , bSlice will be changed as well.
slice is like a struct by definition and it contains 3 parts.
Go foundation
27
webapp-with-golang-anti-textbook
Attention: append will change the array that slice points to, and affect other slices that
point to the same array. Also, if there is not enough length for the slice ( (cap-len) == 0 ),
append returns a new array for this slice. When this happens, other slices pointing to the old
map
map behaves like a dictionary in Python. Use the form map[keyType]valueType to define it.
Let's see some code. The 'set' and 'get' values in map are similar to slice , however the
index in slice can only be of type 'int' while map can use much more than that: for
example int , string , or whatever you want. Also, they are all able to use == and != to
compare values.
// use string as the key type, int as the value type, and `make` initialize it.
var numbers map[string] int
// another way to define map
numbers := make(map[string]int)
numbers["one"] = 1 // assign value by key
numbers["ten"] = 10
numbers["three"] = 3
fmt.Println("The third number is: ", numbers["three"]) // get values
// It prints: The third number is: 3
It's quite easy to change the value through map . Simply use numbers["one"]=11 to
change the value of key one to 11 .
You can use form key:val to initialize map's values, and map has built-in methods to
check if the key exists.
Use delete to delete an element in map .
Go foundation
28
webapp-with-golang-anti-textbook
// Initialize a map
rating := map[string]float32 {"C":5, "Go":4.5, "Python":4.5, "C++":2 }
// map has two return values. For the second return value, if the key doesn't
//exist'ok' returns false. It returns true otherwise.
csharpRating, ok := rating["C#"]
if ok {
fmt.Println("C# is in the map and its rating is ", csharpRating)
} else {
fmt.Println("We have no rating associated with C# in the map")
}
delete(rating, "C") // delete element with key "c"
As I said above, map is a reference type. If two map s point to same underlying data, any
change will affect both of them.
m := make(map[string]string)
m["Hello"] = "Bonjour"
m1 := m
m1["Hello"] = "Salut" // now the value of m["hello"] is Salut
make, new
make does memory allocation for built-in models, such as map , slice , and channel ,
value of type *T . By Go's definition, it returns a pointer which points to type T 's zerovalue.
new returns pointers.
The built-in function make(T, args) has different purposes than new(T) . make can be
used for slice , map , and channel , and returns a type T with an initial value. The reason
for doing this is because the underlying data of these three types must be initialized before
they point to them. For example, a slice contains a pointer that points to the underlying
array , length and capacity. Before these data are initialized, slice is nil , so for slice ,
map and channel , make initializes their underlying data and assigns some suitable values.
make returns non-zero values.
The following picture shows how new and make are different.
Go foundation
29
webapp-with-golang-anti-textbook
Zero-value does not mean empty value. It's the value that variables default to in most cases.
Here is a list of some zero-values.
int 0
int8 0
int32 0
int64 0
uint 0x0
rune 0 // the actual type of rune is int32
byte 0x0 // the actual type of byte is uint8
float32 0 // length is 4 byte
float64 0 //length is 8 byte
bool false
string ""
Links
Directory
Previous section: "Hello, Go"
Next section: Control statements and functions
Go foundation
30
webapp-with-golang-anti-textbook
Control statement
The greatest invention in programming is flow control. Because of them, you are able to use
simple control statements that can be used to represent complex logic. There are three
categories of flow control: conditional, cycle control and unconditional jump.
if
if will most likely be the most common keyword in your programs. If it meets the
if x > 10 {
fmt.Println("x is greater than 10")
} else {
fmt.Println("x is less than or equal to 10")
}
The most useful thing concerning if in Go is that it can have one initialization statement
before the conditional statement. The scope of the variables defined in this initialization
statement are only available inside the block of the defining if .
// initialize x, then check if x greater than
if x := computedValue(); x > 10 {
fmt.Println("x is greater than 10")
} else {
fmt.Println("x is less than 10")
}
// the following code will not compile
fmt.Println(x)
31
webapp-with-golang-anti-textbook
if integer == 3 {
fmt.Println("The integer is equal to 3")
} else if integer < 3 {
fmt.Println("The integer is less than 3")
} else {
fmt.Println("The integer is greater than 3")
}
goto
Go has a goto keyword, but be careful when you use it. goto reroutes the control flow to
a previously defined label within the body of same code block.
func myFunc() {
i := 0
Here: // label ends with ":"
fmt.Println(i)
i++
goto Here // jump to label "Here"
}
for
for is the most powerful control logic in Go. It can read data in loops and iterative
expression1 , expression2 and expression3 are all expressions, where expression1 and
expression3 are variable definitions or return values from functions, and expression2 is a
conditional statement. expression1 will be executed once before looping, and expression3
will be executed after each loop.
Examples are more useful than words.
32
webapp-with-golang-anti-textbook
package main
import "fmt"
func main(){
sum := 0;
for index:=0; index < 10 ; index++ {
sum += index
}
fmt.Println("sum is equal to ", sum)
}
// Printsum is equal to 45
Sometimes we need multiple assignments, but Go doesn't have the , operator, so we use
parallel assignment like i, j = i + 1, j - 1 .
We can omit expression1 and expression3 if they are not necessary.
sum := 1
for ; sum < 1000; {
sum += sum
}
There are two important operations in loops which are break and continue . break jumps
out of the loop, and continue skips the current loop and starts the next one. If you have
nested loops, use break along with labels.
for index := 10; index>0; index-- {
if index == 5{
break // or continue
}
fmt.Println(index)
}
// break prints 109876
// continue prints 1098764321
for can read data from slice and map when it is used together with range .
33
webapp-with-golang-anti-textbook
Because Go supports multi-value returns and gives compile errors when you don't use
values that were defined, you may want to use _ to discard certain return values.
for _, v := range map{
fmt.Println("map's val:", v)
}
switch
Sometimes you may find that you are using too many if-else statements to implement
some logic, which may make it difficult to read and maintain in the future. This is the perfect
time to use the switch statement to solve this problem.
switch sExpr {
case expr1:
some instructions
case expr2:
some other instructions
case expr3:
some other instructions
default:
other code
}
The type of sExpr , expr1 , expr2 , and expr3 must be the same. switch is very flexible.
Conditions don't have to be constants and it executes from top to bottom until it matches
conditions. If there is no statement after the keyword switch , then it matches true .
i := 10
switch i {
case 1:
fmt.Println("i is equal to 1")
case 2, 3, 4:
fmt.Println("i is equal to 2, 3 or 4")
case 10:
fmt.Println("i is equal to 10")
default:
fmt.Println("All I know is that i is an integer")
}
34
webapp-with-golang-anti-textbook
In the fifth line, we put many values in one case , and we don't need to add the break
keyword at the end of case 's body. It will jump out of the switch body once it matched any
case. If you want to continue to matching more cases, you need to use the fallthrough
statement.
integer := 6
switch integer {
case 4:
fmt.Println("integer <= 4")
fallthrough
case 5:
fmt.Println("integer <= 5")
fallthrough
case 6:
fmt.Println("integer <= 6")
fallthrough
case 7:
fmt.Println("integer <= 7")
fallthrough
case 8:
fmt.Println("integer <= 8")
fallthrough
default:
fmt.Println("default case")
}
Functions
Use the func keyword to define a function.
func funcName(input1 type1, input2 type2) (output1 type1, output2 type2) {
// function body
// multi-value return
return value1, value2
}
35
webapp-with-golang-anti-textbook
Functions have zero, one or more than one arguments. The argument type comes after
the argument name and arguments are separated by , .
Functions can return multiple values.
There are two return values named output1 and output2 , you can omit their names
and use their type only.
If there is only one return value and you omitted the name, you don't need brackets for
the return values.
If the function doesn't have return values, you can omit the return parameters
altogether.
If the function has return values, you have to use the return statement somewhere in
the body of the function.
Let's see one practical example. (calculate maximum value)
package main
import "fmt"
// return greater value between a and b
func max(a, b int) int {
if a > b {
return a
}
return b
}
func main() {
x := 3
y := 4
z := 5
max_xy := max(x, y) // call function max(x, y)
max_xz := max(x, z) // call function max(x, z)
fmt.Printf("max(%d, %d) = %d\n", x, y, max_xy)
fmt.Printf("max(%d, %d) = %d\n", x, z, max_xz)
fmt.Printf("max(%d, %d) = %d\n", y, z, max(y,z)) // call function here
}
In the above example, there are two arguments in the function max , their types are both
int so the first type can be omitted. For instance, a, b int instead of a int, b int . The
same rules apply for additional arguments. Notice here that max only has one return value,
so we only need to write the type of its return value -this is the short form of writing it.
Multi-value return
One thing that Go is better at than C is that it supports multi-value returns.
36
webapp-with-golang-anti-textbook
The above example returns two values without names -you have the option of naming them
also. If we named the return values, we would just need to use return to return the values
since they are initialized in the function automatically. Notice that if your functions are going
to be used outside of the package, which means your function names start with a capital
letter, you'd better write complete statements for return ; it makes your code more
readable.
func SumAndProduct(A, B int) (add int, multiplied int) {
add = A+B
multiplied = A*B
return
}
Variadic functions
Go supports functions with a variable number of arguments. These functions are called
"variadic", which means the function allows an uncertain numbers of arguments.
func myfunc(arg ...int) {}
arg int tells Go that this is a function that has variable arguments. Notice that these
arguments are type int . In the body of function, the arg becomes a slice of int .
37
webapp-with-golang-anti-textbook
Can you see that? Even though we called add1 with x , the origin value of x doesn't
change.
The reason is very simple: when we called add1 , we gave a copy of x to it, not the x
itself.
Now you may ask how I can pass the real x to the function.
We need use pointers here. We know variables are stored in memory and they have some
memory addresses. So, if we want to change the value of a variable, we must change its
memory address. Therefore the function add1 has to know the memory address of x in
order to change its value. Here we pass &x to the function, and change the argument's
type to the pointer type *int . Be aware that we pass a copy of the pointer, not copy of
value.
38
webapp-with-golang-anti-textbook
package main
import "fmt"
// simple function to add 1 to a
func add1(a *int) int {
*a = *a+1 // we changed value of a
return *a // return new value of a
}
func main() {
x := 3
fmt.Println("x = ", x) // should print "x = 3"
x1 := add1(&x) // call add1(&x) pass memory address of x
fmt.Println("x+1 = ", x1) // should print "x+1 = 4"
fmt.Println("x = ", x) // should print "x = 4"
}
Now we can change the value of x in the functions. Why do we use pointers? What are the
advantages?
Allows us to use more functions to operate on one variable.
Low cost by passing memory addresses (8 bytes), copy is not an efficient way, both in
terms of time and space, to pass variables.
string , slice and map are reference types, so they use pointers when passing to
functions by default. (Attention: If you need to change the length of slice , you have to
pass pointers explicitly)
defer
Go has a well designed keyword called defer . You can have many defer statements in
one function; they will execute in reverse order when the program executes to the end of
functions. In the case where the program opens some resource files, these files would have
to be closed before the function can return with errors. Let's see some examples.
39
webapp-with-golang-anti-textbook
We saw some code being repeated several times. defer solves this problem very well. It
doesn't only help you to write clean code but also makes your code more readable.
func ReadWrite() bool {
file.Open("file")
defer file.Close()
if failureX {
return false
}
if failureY {
return false
}
return true
}
If there are more than one defer s, they will execute by reverse order. The following
example will print 4 3 2 1 0 .
for i := 0; i < 5; i++ {
defer fmt.Printf("%d ", i)
}
40
webapp-with-golang-anti-textbook
What's the advantage of this feature? The answer is that it allows us to pass functions as
values.
package main
import "fmt"
type testInt func(int) bool // define a function type of variable
func isOdd(integer int) bool {
if integer%2 == 0 {
return false
}
return true
}
func isEven(integer int) bool {
if integer%2 == 0 {
return true
}
return false
}
// pass the function `f` as an argument to another function
func filter(slice []int, f testInt) []int {
var result []int
for _, value := range slice {
if f(value) {
result = append(result, value)
}
}
return result
}
func main(){
slice := []int {1, 2, 3, 4, 5, 7}
fmt.Println("slice = ", slice)
odd := filter(slice, isOdd) // use function as values
fmt.Println("Odd elements of slice are: ", odd)
even := filter(slice, isEven)
fmt.Println("Even elements of slice are: ", even)
}
It's very useful when we use interfaces. As you can see testInt is a variable that has a
function as type and the returned values and arguments of filter are the same as those
of testInt . Therefore, we can have complex logic in our programs, while maintaining
flexibility in our code.
41
webapp-with-golang-anti-textbook
Go doesn't have try-catch structure like Java does. Instead of throwing exceptions, Go
uses panic and recover to deal with errors. However, you shouldn't use panic very
much, although it's powerful.
Panic is a built-in function to break the normal flow of programs and get into panic status.
When a function F calls panic , F will not continue executing but its defer functions will
continue to execute. Then F goes back to the break point which caused the panic status.
The program will not terminate until all of these functions return with panic to the first level of
that goroutine . panic can be produced by calling panic in the program, and some errors
also cause panic like array access out of bounds errors.
Recover is a built-in function to recover goroutine s from panic status. Calling recover in
defer functions is useful because normal functions will not be executed when the program
is in the panic status. It catches panic values if the program is in the panic status, and it
gets nil if the program is not in panic status.
The following example shows how to use panic .
var user = os.Getenv("USER")
func init() {
if user == "" {
panic("no value for $USER")
}
}
42
webapp-with-golang-anti-textbook
Go programs will call init() and main() automatically, so you don't need to call them by
yourself. For every package, the init function is optional, but package main has one and
only one main function.
Programs initialize and begin execution from the main package. If the main package
imports other packages, they will be imported in the compile time. If one package is imported
many times, it will be only compiled once. After importing packages, programs will initialize
the constants and variables within the imported packages, then execute the init function if
it exists, and so on. After all the other packages are initialized, programs will initialize
constants and variables in the main package, then execute the init function inside the
package if it exists. The following figure shows the process.
import
We use import very often in Go programs as follows.
import(
"fmt"
)
43
webapp-with-golang-anti-textbook
The dot operator means you can omit the package name when you call functions inside
of that package. Now fmt.Printf("Hello world") becomes to Printf("Hello world") .
2. Alias operation. It changes the name of the package that we imported when we call
functions that belong to that package.
import(
f "fmt"
)
explaining it to you.
import (
"database/sql"
_ "github.com/ziutek/mymysql/godrv"
)
The _ operator actually means we just want to import that package and execute its
init function, and we are not sure if want to use the functions belonging to that
package.
Links
Directory
Previous section: Go foundation
Next section: struct
44
webapp-with-golang-anti-textbook
2.4 struct
struct
We can define new types of containers of other properties or fields in Go just like in other
programming languages. For example, we can create a type called person to represent a
person, with fields name and age. We call this kind of type a struct .
type person struct {
name string
age int
}
struct
45
webapp-with-golang-anti-textbook
P := person{age:24, name:"Bob"}
struct
46
webapp-with-golang-anti-textbook
47
webapp-with-golang-anti-textbook
We see that we can access the age and name fields in Student just like we can in Human.
This is how embedded fields work. It's very cool, isn't it? Hold on, there's something cooler!
You can even use Student to access Human in this embedded field!
mark.Human = Human{"Marcus", 55, 220}
mark.Human.age -= 1
In the above example, we can see that all types can be embedded fields and we can use
functions to operate on them.
struct
48
webapp-with-golang-anti-textbook
There is one more problem however. If Human has a field called phone and Student has a
field with same name, what should we do?
Go use a very simple way to solve it. The outer fields get upper access levels, which means
when you access student.phone , we will get the field called phone in student, not the one in
the Human struct. This feature can be simply seen as field overload ing.
package main
import "fmt"
type Human struct {
name string
age int
phone string // Human has phone field
}
type Employee struct {
Human // embedded field Human
specialty string
phone string // phone in employee
}
func main() {
Bob := Employee{Human{"Bob", 34, "777-444-XXXX"}, "Designer", "333-222"}
fmt.Println("Bob's work phone is:", Bob.phone)
// access phone field in Human
fmt.Println("Bob's personal phone is:", Bob.Human.phone)
}
Links
Directory
Previous section: Control statements and functions
Next section: Object-oriented
struct
49
webapp-with-golang-anti-textbook
Object-oriented
We talked about functions and structs in the last two sections, but did you ever consider
using functions as fields of a struct? In this section, I will introduce you to another form of
function that has a receiver, which is called method .
method
Suppose you define a "rectangle" struct and you want to calculate its area. We'd typically
use the following code to achieve this goal.
package main
import "fmt"
type Rectangle struct {
width, height float64
}
func area(r Rectangle) float64 {
return r.width*r.height
}
func main() {
r1 := Rectangle{12, 2}
r2 := Rectangle{9, 4}
fmt.Println("Area of r1 is: ", area(r1))
fmt.Println("Area of r2 is: ", area(r2))
}
The above example can calculate a rectangle's area. We use the function called area , but
it's not a method of the rectangle struct (like class methods in classic object-oriented
languages). The function and struct are two independent things as you may notice.
It's not a problem so far. However, if you also have to calculate the area of a circle, square,
pentagon, or any other kind of shape, you are going to need to add additional functions with
very similar names.
Object-oriented
50
webapp-with-golang-anti-textbook
For those reasons, we have the method concept. method is affiliated with type. It has the
same syntax as functions do except for an additional parameter after the func keyword
called the receiver , which is the main body of that method.
Using the same example, Rectangle.area() belongs directly to rectangle, instead of as a
peripheral function. More specifically, length , width and area() all belong to rectangle.
As Rob Pike said.
"A method is a function with an implicit first argument, called a receiver."
Syntax of method.
func (r ReceiverType) funcName(parameters) (results)
Object-oriented
51
webapp-with-golang-anti-textbook
package main
import (
"fmt"
"math"
)
type Rectangle struct {
width, height float64
}
type Circle struct {
radius float64
}
func (r Rectangle) area() float64 {
return r.width*r.height
}
func (c Circle) area() float64 {
return c.radius * c.radius * math.Pi
}
func main() {
r1 := Rectangle{12, 2}
r2 := Rectangle{9, 4}
c1 := Circle{10}
c2 := Circle{25}
fmt.Println("Area of r1 is: ", r1.area())
fmt.Println("Area of r2 is: ", r2.area())
fmt.Println("Area of c1 is: ", c1.area())
fmt.Println("Area of c2 is: ", c2.area())
}
Object-oriented
52
webapp-with-golang-anti-textbook
One thing that's worth noting is that the method with a dotted line means the receiver is
passed by value, not by reference. The difference between them is that a method can
change its receiver's values when the receiver is passed by reference, and it gets a copy of
the receiver when the receiver is passed by value.
Can the receiver only be a struct? Of course not. Any type can be the receiver of a method.
You may be confused about customized types. Struct is a special kind of customized type there are more customized types.
Use the following format to define a customized type.
type typeName typeLiteral
I hope that you know how to use customized types now. Similar to typedef in C, we use
ages to substitute int in the above example.
Object-oriented
53
webapp-with-golang-anti-textbook
Object-oriented
54
webapp-with-golang-anti-textbook
in this method. Thus, if we don't use a pointer, it will only change the value inside a copy of
Box.
If we see that a receiver is the first argument of a method, it's not hard to understand how it
works.
You might be asking why we aren't using (*b).Color=c instead of b.Color=c in the
SetColor() method. Either one is OK here because Go knows how to interpret the
assignment. Do you think Go is more fascinating now?
You may also be asking whether we should use (&bl[i]).SetColor(BLACK) in PaintItBlack
because we pass a pointer to SetColor . Again, either one is OK because Go knows how to
interpret it!
Inheritance of method
Object-oriented
55
webapp-with-golang-anti-textbook
We learned about inheritance of fields in the last section. Similarly, we also have method
inheritance in Go. If an anonymous field has methods, then the struct that contains the field
will have all the methods from it as well.
package main
import "fmt"
type Human struct {
name string
age int
phone string
}
type Student struct {
Human // anonymous field
school string
}
type Employee struct {
Human
company string
}
// define a method in Human
func (h *Human) SayHi() {
fmt.Printf("Hi, I am %s you can call me on %s\n", h.name, h.phone)
}
func main() {
mark := Student{Human{"Mark", 25, "222-222-YYYY"}, "MIT"}
sam := Employee{Human{"Sam", 45, "111-888-XXXX"}, "Golang Inc"}
mark.SayHi()
sam.SayHi()
}
Method overload
If we want Employee to have its own method SayHi , we can define a method that has the
same name in Employee, and it will hide SayHi in Human when we call it.
Object-oriented
56
webapp-with-golang-anti-textbook
package main
import "fmt"
type Human struct {
name string
age int
phone string
}
type Student struct {
Human
school string
}
type Employee struct {
Human
company string
}
func (h *Human) SayHi() {
fmt.Printf("Hi, I am %s you can call me on %s\n", h.name, h.phone)
}
func (e *Employee) SayHi() {
fmt.Printf("Hi, I am %s, I work at %s. Call me on %s\n", e.name,
e.company, e.phone) //Yes you can split into 2 lines here.
}
func main() {
mark := Student{Human{"Mark", 25, "222-222-YYYY"}, "MIT"}
sam := Employee{Human{"Sam", 45, "111-888-XXXX"}, "Golang Inc"}
mark.SayHi()
sam.SayHi()
}
You are able to write an Object-oriented program now, and methods use rule of capital letter
to decide whether public or private as well.
Links
Directory
Previous section: struct
Next section: interface
Object-oriented
57
webapp-with-golang-anti-textbook
2.6 Interface
Interface
One of the subtlest design features in Go are interfaces. After reading this section, you will
likely be impressed by their implementation.
What is an interface
In short, an interface is a set of methods that we use to define a set of actions.
Like the examples in previous sections, both Student and Employee can SayHi() , but they
don't do the same thing.
Let's do some more work. We'll add one more method Sing() to them, along with the
BorrowMoney() method to Student and the SpendSalary() method to Employee.
Now, Student has three methods called SayHi() , Sing() and BorrowMoney() , and
Employee has SayHi() , Sing() and SpendSalary() .
This combination of methods is called an interface and is implemented by both Student and
Employee. So, Student and Employee implement the interface: SayHi() and Sing() . At
the same time, Employee doesn't implement the interface: SayHi() , Sing() ,
BorrowMoney() , and Student doesn't implement the interface: SayHi() , Sing() ,
SpendSalary() . This is because Employee doesn't have the method BorrowMoney() and
Type of Interface
An interface defines a set of methods, so if a type implements all the methods we say that it
implements the interface.
type Human struct {
name string
age int
phone string
}
type Student struct {
Human
school string
loan float32
interface
58
webapp-with-golang-anti-textbook
}
type Employee struct {
Human
company string
money float32
}
func (h *Human) SayHi() {
fmt.Printf("Hi, I am %s you can call me on %s\n", h.name, h.phone)
}
func (h *Human) Sing(lyrics string) {
fmt.Println("La la, la la la, la la la la la...", lyrics)
}
func (h *Human) Guzzle(beerStein string) {
fmt.Println("Guzzle Guzzle Guzzle...", beerStein)
}
// Employee overloads Sayhi
func (e *Employee) SayHi() {
fmt.Printf("Hi, I am %s, I work at %s. Call me on %s\n", e.name,
e.company, e.phone) //Yes you can split into 2 lines here.
}
func (s *Student) BorrowMoney(amount float32) {
s.loan += amount // (again and again and...)
}
func (e *Employee) SpendSalary(amount float32) {
e.money -= amount // More vodka please!!! Get me through the day!
}
// define interface
type Men interface {
SayHi()
Sing(lyrics string)
Guzzle(beerStein string)
}
type YoungChap interface {
SayHi()
Sing(song string)
BorrowMoney(amount float32)
}
type ElderlyGent interface {
SayHi()
Sing(song string)
SpendSalary(amount float32)
}
interface
59
webapp-with-golang-anti-textbook
We know that an interface can be implemented by any type, and one type can implement
many interfaces simultaneously.
Note that any type implements the empty interface interface{} because it doesn't have
any methods and all types have zero methods by default.
Value of interface
So what kind of values can be put in the interface? If we define a variable as a type
interface, any type that implements the interface can assigned to this variable.
Like the above example, if we define a variable "m" as interface Men, then any one of
Student, Human or Employee can be assigned to "m". So we could have a slice of Men, and
any type that implements interface Men can assign to this slice. Be aware however that the
slice of interface doesn't have the same behavior as a slice of other types.
package main
import "fmt"
type Human struct {
name string
age int
phone string
}
type Student struct {
Human
school string
loan float32
}
type Employee struct {
Human
company string
money float32
}
func (h Human) SayHi() {
fmt.Printf("Hi, I am %s you can call me on %s\n", h.name, h.phone)
}
func (h Human) Sing(lyrics string) {
fmt.Println("La la la la...", lyrics)
}
func (e Employee) SayHi() {
fmt.Printf("Hi, I am %s, I work at %s. Call me on %s\n", e.name,
e.company, e.phone) //Yes you can split into 2 lines here.
interface
60
webapp-with-golang-anti-textbook
}
// Interface Men implemented by Human, Student and Employee
type Men interface {
SayHi()
Sing(lyrics string)
}
func main() {
mike := Student{Human{"Mike", 25, "222-222-XXX"}, "MIT", 0.00}
paul := Student{Human{"Paul", 26, "111-222-XXX"}, "Harvard", 100}
sam := Employee{Human{"Sam", 36, "444-222-XXX"}, "Golang Inc.", 1000}
tom := Employee{Human{"Sam", 36, "444-222-XXX"}, "Things Ltd.", 5000}
// define interface i
var i Men
//i can store Student
i = mike
fmt.Println("This is Mike, a Student:")
i.SayHi()
i.Sing("November rain")
//i can store Employee
i = tom
fmt.Println("This is Tom, an Employee:")
i.SayHi()
i.Sing("Born to be wild")
// slice of Men
fmt.Println("Let's use a slice of Men and see what happens")
x := make([]Men, 3)
// these three elements are different types but they all implemented interface Men
x[0], x[1], x[2] = paul, sam, mike
for _, value := range x {
value.SayHi()
}
}
Empty interface
An empty interface is an interface that doesn't contain any methods, so all types implement
an empty interface. This fact is very useful when we want to store all types at some point,
and is similar to void* in C.
interface
61
webapp-with-golang-anti-textbook
If a function uses an empty interface as its argument type, it can accept any type; if a
function uses empty interface as its return value type, it can return any type.
This means any type that implements interface Stringer can be passed to fmt.Println as an
argument. Let's prove it.
interface
62
webapp-with-golang-anti-textbook
package main
import (
"fmt"
"strconv"
)
type Human struct {
name string
age int
phone string
}
// Human implemented fmt.Stringer
func (h Human) String() string {
return "Name:" + h.name + ", Age:" + strconv.Itoa(h.age) + " years, Contact:" + h.phone
}
func main() {
Bob := Human{"Bob", 39, "000-7777-XXX"}
fmt.Println("This Human is : ", Bob)
}
Looking back to the example of Box, you will find that Color implements interface Stringer as
well, so we are able to customize the print format. If we don't implement this interface,
fmt.Println prints the type with its default format.
fmt.Println("The biggest one is", boxes.BiggestsColor().String())
fmt.Println("The biggest one is", boxes.BiggestsColor())
Attention: If the type implemented the interface error , fmt will call error() , so you don't
have to implement Stringer at this point.
63
webapp-with-golang-anti-textbook
It's quite easy to use this pattern, but if we have many types to test, we'd better use switch .
switch test
Let's use switch to rewrite the above example.
interface
64
webapp-with-golang-anti-textbook
package main
import (
"fmt"
"strconv"
)
type Element interface{}
type List []Element
type Person struct {
name string
age int
}
func (p Person) String() string {
return "(name: " + p.name + " - age: " + strconv.Itoa(p.age) + " years)"
}
func main() {
list := make(List, 3)
list[0] = 1 //an int
list[1] = "Hello" //a string
list[2] = Person{"Dennis", 70}
for index, element := range list {
switch value := element.(type) {
case int:
fmt.Printf("list[%d] is an int and its value is %d\n", index, value)
case string:
fmt.Printf("list[%d] is a string and its value is %s\n", index, value)
case Person:
fmt.Printf("list[%d] is a Person and its value is %s\n", index, value)
default:
fmt.Println("list[%d] is of a different type", index)
}
}
}
One thing you should remember is that element.(type) cannot be used outside of the
switch body, which means in that case you have to use the comma-ok pattern .
Embedded interfaces
The most beautiful thing is that Go has a lot of built-in logic syntax, such as anonymous
fields in struct. Not suprisingly, we can use interfaces as anonymous fields as well, but we
call them Embedded interfaces . Here, we follow the same rules as anonymous fields. More
interface
65
webapp-with-golang-anti-textbook
specifically, if an interface has another interface embedded within it, it will behave as if it has
all the methods that the embedded interface has.
We can see that the source file in container/heap has the following definition:
type Interface interface {
sort.Interface // embedded sort.Interface
Push(x interface{}) //a Push method to push elements into the heap
Pop() interface{} //a Pop method that pops elements from the heap
}
We see that sort.Interface is an embedded interface, so the above Interface has the three
methods contained within the sort.Interface implicitly.
type Interface interface {
// Len is the number of elements in the collection.
Len() int
// Less returns whether the element with index i should sort
// before the element with index j.
Less(i, j int) bool
// Swap swaps the elements with indexes i and j.
Swap(i, j int)
}
Reflection
Reflection in Go is used for determining information at runtime. We use the reflect
package, and this official article explains how reflect works in Go.
There are three steps involved when using reflect. First, we need to convert an interface to
reflect types (reflect.Type or reflect.Value, this depends on the situation).
t := reflect.TypeOf(i) // get meta-data in type i, and use t to get all elements
v := reflect.ValueOf(i) // get actual value in type i, and use v to change its value
After that, we can convert the reflected types to get the values that we need.
interface
66
webapp-with-golang-anti-textbook
Finally, if we want to change the values of the reflected types, we need to make it modifiable.
As discussed earlier, there is a difference between pass by value and pass by reference.
The following code will not compile.
var x float64 = 3.4
v := reflect.ValueOf(x)
v.SetFloat(7.1)
Instead, we must use the following code to change the values from reflect types.
var x float64 = 3.4
p := reflect.ValueOf(&x)
v := p.Elem()
v.SetFloat(7.1)
We have just discussed the basics of reflection, however you must practice more in order to
understand more.
Links
Directory
Previous section: Object-oriented
Next section: Concurrency
interface
67
webapp-with-golang-anti-textbook
Concurrency
It is said that Go is the C language of the 21st century. I think there are two reasons: first, Go
is a simple language; second, concurrency is a hot topic in today's world, and Go supports
this feature at the language level.
goroutine
goroutines and concurrency are built into the core design of Go. They're similar to threads
but work differently. More than a dozen goroutines maybe only have 5 or 6 underlying
threads. Go also gives you full support to sharing memory in your goroutines. One goroutine
usually uses 4~5 KB of stack memory. Therefore, it's not hard to run thousands of
goroutines on a single computer. A goroutine is more lightweight, more efficient and more
convenient than system threads.
goroutines run on the thread manager at runtime in Go. We use the go keyword to create a
new goroutine, which is a function at the underlying level ( main() is a goroutine ).
go hello(a, b, c)
Output
Concurrency
68
webapp-with-golang-anti-textbook
hello
world
hello
world
hello
world
hello
world
hello
We see that it's very easy to use concurrency in Go by using the keyword go . In the above
example, these two goroutines share some memory, but we would better off following the
design recipe: Don't use shared data to communicate, use communication to share data.
runtime.Gosched() means let the CPU execute other goroutines, and come back at some
point.
The scheduler only uses one thread to run all goroutines, which means it only implements
concurrency. If you want to use more CPU cores in order to take advantage of parallel
processing, you have to call runtime.GOMAXPROCS(n) to set the number of cores you want
to use. If n<1 , it changes nothing. This function may be removed in the future, see more
details about parallel processing and concurrency in this article.
channels
goroutines run in the same memory address space, so you have to maintain synchronization
when you want to access shared memory. How do you communicate between different
goroutines? Go uses a very good communication mechanism called channel . channel is
like a two-way pipeline in Unix shells: use channel to send or receive data. The only data
type that can be used in channels is the type channel and the keyword chan . Be aware
that you have to use make to create a new channel .
ci := make(chan int)
cs := make(chan string)
cf := make(chan interface{})
Concurrency
69
webapp-with-golang-anti-textbook
package main
import "fmt"
func sum(a []int, c chan int) {
total := 0
for _, v := range a {
total += v
}
c <- total // send total to c
}
func main() {
a := []int{7, 2, 8, -9, 4, 0}
c := make(chan int)
go sum(a[:len(a)/2], c)
go sum(a[len(a)/2:], c)
x, y := <-c, <-c // receive from c
fmt.Println(x, y, x + y)
}
Sending and receiving data in channels blocks by default, so it's much easier to use
synchronous goroutines. What I mean by block is that a goroutine will not continue when
receiving data from an empty channel, i.e ( value := <-ch ), until other goroutines send data
to this channel. On the other hand, the goroutine will not continue until the data it sends to a
channel, i.e ( ch<-5 ), is received.
Buffered channels
I introduced non-buffered channels above. Go also has buffered channels that can store
more than a single element. For example, ch := make(chan bool, 4) , here we create a
channel that can store 4 boolean elements. So in this channel, we are able to send 4
elements into it without blocking, but the goroutine will be blocked when you try to send a
fifth element and no goroutine receives it.
ch := make(chan type, n)
n == 0 ! non-bufferblock
n > 0 ! buffernon-block until n elements in the channel
You can try the following code on your computer and change some values.
Concurrency
70
webapp-with-golang-anti-textbook
package main
import "fmt"
func main() {
c := make(chan int, 2) // change 2 to 1 will have runtime error, but 3 is fine
c <- 1
c <- 2
fmt.Println(<-c)
fmt.Println(<-c)
}
for i := range c will not stop reading data from channel until the channel is closed. We
use the keyword close to close the channel in above example. It's impossible to send or
receive data on a closed channel; you can use v, ok := <-ch to test if a channel is closed.
If ok returns false, it means the there is no data in that channel and it was closed.
Remember to always close channels in producers and not in consumers, or it's very easy to
get into panic status.
Concurrency
71
webapp-with-golang-anti-textbook
Another thing you need to remember is that channels are not like files. You don't have to
close them frequently unless you are sure the channel is completely useless, or you want to
exit range loops.
Select
In the above examples, we only use one channel, but how can we deal with more than one
channel? Go has a keyword called select to listen to many channels.
select is blocking by default and it continues to execute only when one of channels has
data to send or receive. If several channels are ready to use at the same time, select
chooses which to execute randomly.
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 1, 1
for {
select {
case c <- x:
x, y = y, x + y
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit)
}
select has a default case as well, just like switch . When all the channels are not ready
for use, it executes the default case (it doesn't wait for the channel anymore).
Concurrency
72
webapp-with-golang-anti-textbook
select {
case i := <-c:
// use i
default:
// executes here when c is blocked
}
Timeout
Sometimes a goroutine becomes blocked. How can we avoid this to prevent the whole
program from blocking? It's simple, we can set a timeout in the select.
func main() {
c := make(chan int)
o := make(chan bool)
go func() {
for {
select {
case v := <- c:
println(v)
case <- time.After(5 * time.Second):
println("timeout")
o <- true
break
}
}
}()
<- o
}
Runtime goroutine
The package runtime has some functions for dealing with goroutines.
runtime.Goexit()
Exits the current goroutine, but defered functions will be executed as usual.
runtime.Gosched()
Lets the scheduler execute other goroutines and comes back at some point.
runtime.NumCPU() int
Concurrency
73
webapp-with-golang-anti-textbook
runtime.NumGoroutine() int
Links
Directory
Previous section: interface
Next section: Summary
Concurrency
74
webapp-with-golang-anti-textbook
2.8 Summary
In this chapter, we mainly introduced the 25 Go keywords. Let's review what they are and
what they do.
break default func interface select
case defer go map struct
chan else goto package switch
const fallthrough if range type
continue for import return var
If you understand how to use these 25 keywords, you've learned a lot of Go already.
Links
Directory
Previous section: Concurrency
Next chapter: Web Programming basics
Summary
75
webapp-with-golang-anti-textbook
Go Programming Basics
Go is a statically typed language and it is an easy to learn language. But that doesn't mean
that programming in Go is really simple, becuase some times it can cause some really
subtle bugs which tend to be impossible to find, so we need to be careful in general while
programming. Go also has many restrictions like you can't have an unused variable. Go
doesn't require you to add semi colons at the end of each line, but the compiler does add
semi colons at the end of each line, this is the reason braces are used for formatting. Go is a
complied language, which means when we compile the code, it'll generate a statically linked
binary. This means that after building your project (for your machine) you can take that
binary and run it on machine with the same OS. If your project involves the use of non Go
files then you have to carry them along with the binary file since things don't scale well if the
binary file has the static files.
Workspace
The standard way of writing Go code is to make a directory and assign it to the environment
variable $GOPATH . In most unix systems it is done with export GOPATH=/usr/home/suraj/Go .
The idea is that all your Go code should reside in a directory tree so the code isn't lying
around in random places. There is another environment variable $GOROOT which holds the
installation of Go language itself and is not to be confused with $GOPATH .
My $GOPATH is /usr/home/suraj/Go , it has the following structure
pkg: The libraries which'll be imported in our code, there is a .a file created at the
respective path.
bin: The binary file of our project will be installed here on calling Go install in the
project folder
src: Will hold the actual code, if you have a github account, then you can create a folder
tree inside like
Go
src
github.com
thewhitetulip
wshare
picsort
golang.net
sourcegraph.com
General
76
webapp-with-golang-anti-textbook
bin (binaries)
wshare
picsort
pkg
This makes it easy for organizing code because all my Go code lies in
$GOPATH/src/github.com/thewhitetulip , and the rest of the code lies two levels up in
$GOPATH/src . It also makes life easy for packages, as we'll see in the next sections,
Packages
Packages are a very integral part of the Go language since it was designed for building
complex software for large teams. Unlike other languages one doesn't need to do anything
out of the ordinary for creating packages in Go, all you are required to do is create a folder of
the name of the package and then use the statement package name in each file which
resides in the folder which you created. Please note that in each such folder/package there
can be only one file with the main package declared, that is because when we build the Go
code, the complier will get confused which file to build. Using packages is as easy as
creating them.
Don't use the MVC pattern just for the sake of using it, write a small web application entirely
in the main.go file and later try to add a new feature to the application, then you'll realize
that having one file doesn't scale well, yes that is general knowledge but avoid doing things
without understanding them well first.
I didn't make the models & views initially, all handlers were moved into a views.go file
located in a views folder, this became the views package.
The database related methods went into a db package.
The package name is db , but the way to name packages is to give their path from the
$GOPATH/src directory, the reason being when the package is resolved by the Go compiler,
then it first looks in the standard library and then starts looking for the full name in the
$GOPATH/src folder. Hence it is imperative to use the full name, else the code won't compile,
break the flow of people who fork your project, for instance when you clone
General
77
webapp-with-golang-anti-textbook
github.com/thewhitetulip/Tasks/ it'll import all the required packages too, and if the
packages were created just in the src directory, then it'll break the application, since if my db
package isn't github.com/thewhitetulip/Tasks/db but just db then the go get command
will be unable to get the db package.
The downside of this is that when someone wants to fork your project, they have to rename
all the package declarations.
But if you Go to main folder and build and run then it won't run
[Tasks/main] $ go build main.go
[Tasks/main] $ ./main
This won't run because as per our instruction, the binary will expect all the static files and the
rest of the packages in the current directory, which is the directory called main, but all our
other files are present one level up, so we have to ensure that we call the binary from one
level up.
Internal deployment
We'll code in our $GOPATH/src/github.com folder, but we also need to keep a deployment
version on your machine. Since I use Tasks on a regular basis, I have made a folder
~/Tasks and here I keep the deployment version of Tasks, which contains the binary
Running a server
Usually we run a server on some random 8000+ port, so it doesn't affect some other
application which might be using ports, godoc uses the port 6060 . While running a server,
we have to bind the server to some IP address, we can use a public IP address, which
would mean that anyone can visit your web app over the Internet, or by using a private IP.
General
78
webapp-with-golang-anti-textbook
But if you are on a private network, your machine is assigned an IP address, which can start
with 192.0.0.0 or 10.0.0.0 . If you run a server on that IP address, then your web server is
accessible from over your LAN, so anyone in your LAN can use the web application.
So there are security implications while running your server, if you do
log.Fatal(http.ListenAndServe(":8080", nil))
Then it'll bind the server to the IP address your machine is on the LAN. If you want your web
application to be visible from only your machine then
log.Fatal(http.ListenAndServe("127.0.0.1:8080", nil))
Note: 127.0.0.1
127.0.0.1 is generally called localhost, which is a special IP address which is given to
every machine to refer to itself. it doesn't matter if you are connected to the network or not,
your machine will always have this IP address, so if you are wanting privacy for your web
application bind it to 127.0.0.1 because no other machine in the world other than your own
can see this IP address, if some other machine goes to localhost or 127.0.0.1 then it'll
refer to itself.
Where as if you do want your web application to be accessible then bind it to
0.0.0.0.0:8080 or just :8080 , when you give just the port number, the Go language
Links
Directory
Previous section: Tools
Next section: Web Programming Basics
General
79
webapp-with-golang-anti-textbook
GET vs POST
GET was created to get a resource over HTTP, for instance if you visit any website, then the
browser sends a GET request with the URL / to the server. POST was created to send data
from the client to the server.
80
webapp-with-golang-anti-textbook
In GET the data transferred is via the URL in POST the data isn't hidden or encrypted by
default, but it isn't sent via the URL, it is easily accessible to anyone who knows how to read
a HTTP request, which isn't rocket science really. Security isn't something you add at the
end of the application like a cherry on the cake, it is something you build your application
around. In short there isn't much difference between GET and POST when we consider
security, both transfer data in plain text the thing is GET is relatively a little less secure
since the URL is logged by a proxy server/firewall/browser history and that GET requests
can be done by the browser on behalf of the user without confirmation. Bots are common
over the internet, bots can visit randomly to every link present in your application, but they
don't send random data to any Form you have.
If you want to deploy your application into the wild, you have to use HTTPS, but that too
can't help if the user is naive and responds to a Man in the middle attack, where an attacker
sets up a similar looking website and tricks the user into revealing their username and
password. Long story short, security isn't absolute but it is upto the developer to have a
relatively less insecure website, which means it should check use the HTTP methods
correctly, sanitize all data coming from the user and stop a malicious user in all the normal
ways to exploit a web application.
// Create a new category.
// PUT /categories
// Handler function returns the ID of the new category.
r.HandleFunc("/categories", views.PutCategoryFunc).
Methods("PUT")
// Update an existing category.
// POST /categories/12
r.HandleFunc("/categories/{id}", views.PostCategoryFunc).
Methods("POST")
// View the details of a category.
// GET /categories/12
r.HandleFunc("/categories/{id}", views.GetCategoryFunc).
Methods("GET")
// Delete an existing category.
// DELETE /categories/12
r.HandleFunc("/categories/{id}", views.DeleteCategoryFunc).
Methods("DELETE")
With these routes, think of "/categories" as a directory containing JSON or XML files. Each
file has a name like "12" that contains the details of a category. To create a new category
you PUT a new file into the directory. To view the category you GET the category at
"/categories/12", and to delete the category you DELETE the file at "/categories/12".
81
webapp-with-golang-anti-textbook
Thus when we want to delete a task, then rather than sending GET /delete/1234, we should
send a DELETE /tasks/1234.
Here routers come into picture and the architecture of the application is significantly reduced
because we aren't trying to use GET for everything, but we are using the same URL in a
different context with the different HTTP method.
Example
Suppose we run a blog, the smallest component of which is a post, a post has tags, it is
written by some author, at some time and has some primary key to uniquely identify it in our
database and it has a slug which means the URL.
Earlier URLs used to be ugly, but now they are beautiful, like our fictional blog has this post,
surajblog.com/posts/welcome-the-new-year , the slug is the welcome-the-new-year .
When the server gets a HTTP GET request of /posts/welcome-the-new-year , it'll search for
URL handlers starting with the list of URL handlers we have given, then it'll find the closest
match, in our case it'll be /post/ , then it'll call the handler of this URL. The library will go
from the top to bottom of the HandleFunc listing, hence our / root URL should be at the
very bottom of our list.
http.HandleFunc("/post/", ShowPostBySlug)
http.HandleFunc("/", ShowAllPosts)
What this handler will do is that it'll go to the database, fetch the (id, title, content, time) from
the database for that particular post then it'll read a template and populate data into the
template, which is ultimately the HTML document which is sent back to our browser.
What is a template?
Templates are a way to present data to the user. When the server responds with an HTTP
Response, it has to send an HTML page to the browser. They are a set of parameterized
html pages which has a mechanism to load data.
We put in variables in the template which are populated as per the data sent while the
template is executed. So rather than have an individual page for each response, we club all
pages into types, suppose a profile page, a timeline page and an edit page. We then pass
the username and the details of the profile to the profile page, so one template can show the
profile information to multiple users with different data. The same goes with every other
example.
82
webapp-with-golang-anti-textbook
So writing a web application essentially means, we have to decide upon the URLs, the
structure of the database and write functions in Go to handle each supported pattern of URL,
fetch data from the database, write templates to populate data in templates corresponding to
each URL, the handlers which we write populate data in the templates, a method generally
known as rendering.
Not abusing templates
Templates support a lot of things like template variables, but one has to make sure that there
is no business logic inside the template. We separate the business logic from the
presentation logic, since templates are a way to represent data and not a way to manipulate
data. Web apps which follow these are easily maintainable since the change in business
logic doesn't mean that we have to change everything, but with one app divided into
modules, this triggers a change in that particular part of the module.
Suppose we write a todo list manager which supports multiple users, so we have to ensure
that we show tasks of that particular user rather than sending all the tasks to the template
and filtering there tasks which are of that user on every page. A better approach will surely
be to pass the tasks of only that user while rendering the template, so in the future if we
want to show tasks of some user and his friend, then we won't have to change the template,
we'll just modify the function which populates the Tasks object.
Below is such part of such method which is used to handle the backend functionality of our
EditTask URL which is /edit/<id> located at
file db/tasks.go
func GetTaskByID(id int) types.Context {
//Code to populate the tasks variable
context := types.Context{Tasks: tasks}
return context
}
If you notice, we aren't rendering the template here, because this is a database package
whose job is to return the appropriate data to the view, which will render the template from
the data sent by the database function. Thus we are separating the data fetching logic from
the data representation logic. The separation is going to come into significant light when we
add comments to our application. Thus rather than sending all data to our template and have
the template render data conditionally, we fix the Context object for the template and
render data according to that.
file views/addViews.go
83
webapp-with-golang-anti-textbook
Thus we can split an application into views, database, templates and the main file. This is
the standard way of doing things in this world, not because it is the status quo, but because
it is easier to maintain the web applications that way and since maintainability is the holy
grail for software you are recommended to follow this. You should however try not using this
model once while learning, so you understand the exact merits of this process.
Static Files
Static files play a major role in our web application, they are all the CSS/JS/Images which
we load into our html pages we serve them over the URL pattern /static/ , the way this is
implemented is that whenever a request like /static/<filepath> , we go to the public
directory of our application and then look for the path which we got, if we get a file of that
path then we serve the file, othewise it automatically sends a 404 error.
The project structure should include a public which'll contain all your static files. We will
have a templates folder on the same folder where the public is present. The internal
structure can be anything you like for you are going to reference the path names in your
code.
The reason templates is a separate folder is that we do not want it to be accessible form
inside the http.Dir function of the /static/ URL.
public
| |-- static
| | |-- css
| | | `-- styles.css
..and more
| | `-- js
| | |-- bootstrap.min.js
| | .... and more
templates
| |-- completed.html
| | ...and more
Note Output
The above output is of the tree program.
84
webapp-with-golang-anti-textbook
If you are to implement the static file serving handler yourself, make sure you sanitize the
input, because an attacker might send some malicious request like GET ../../ then we must
not provide a HTTP response with the listing of the directory two levels up. This is very
critical, we should always send an HTTP 404 response to anything but the static files, now
one might say how does the attacker know the file name or the folder structure. In network
security, an attacker who has decided he has to get inside will get inside, with infinite
attempts and infinite time, so we have to strive for making the time taken for breaking into
our servers as big as possible.
Links
Directory
Previous section: Go Programming Basics
Next section: Basics webapp
85
webapp-with-golang-anti-textbook
file main/main.go
package main
import (
"log"
"net/http"
)
func main() {
PORT := ":8080"
log.Fatal(http.ListenAndServe(PORT, nil))
}
After you run the main.go file, you will notice that the program does nothing, it doesn't print
anything because we told it only to listen on the port, this is according to the logic of Go that
no message is good and that errors have to be explicit. But there is no need to adhere to
that philosophy 100% because some times it is good to notify what is happening to the
application, of course it doesn't mean that we print each and every thing happening in the
execution, that is for the -v verbose mode.
Make this change to the main file
code example file: 3.1basicServer.go
Implementation
86
webapp-with-golang-anti-textbook
package main
import (
"log"
"net/http"
)
func main() {
PORT := ":8080"
log.Print("Running server on "+ PORT)
log.Fatal(http.ListenAndServe(PORT, nil))
}
Now run this like we mentioned above and we see this message on the terminal
2016/01/01 22:00:36 Running server on :8080
Open your browser and open localhost:8080 , you'll get the message "404 page not found"
This is because as of now we have just started a server to listen on the port 8080, but we
haven't written anything to it yet. When we go to localhost:8080 , we are sending a HTTP
GET request with the URL as / , and since it can't find any handler handling the / URL,
it'll display the standard HTTP404 error, which means page not found.
Handling URLs
http.HandleFunc("/complete/", CompleteTaskFunc)
//ShowCompleteTasksFunc is used to populate the "/completed/" URL
func ShowCompleteTasksFunc(w http.ResponseWriter, r *http.Request) {
}
We use the HandleFunc function in the net/http package to handle the URLs. In this
function we pass the first argument as the URL to be handled and the second parameter as
either the full fledged function definition or a function object while it is possible to define a
function in the parameter itself, it isn't the best way to write a web application and should be
avoided.
The handler function requires two arguments, the object of the ResponseWriter and that of
the Request, if you see the documentation then you'll notice that the Request object
contains all the things which came in with the request, r.Method will hold if the method is
GET or POST or any other method for that matter.
Implementation
87
webapp-with-golang-anti-textbook
While the request object will store all the information related to the HTTP request that we are
going to serve, the ResponseWriter object will store the output which our server is going to
give for that request, we'll write data to it, either directly using the Write method or by
executing a template.
Simple Handler
We will wrap this chapter up with an example of writing the URL name on the page we are
rendering, so if the user goes to /golang/1234 then will write /golang/1234 on the webpage
which she gets to see on localhost:8080/golang/1234 .
package main
import (
"log"
"net/http"
)
func main() {
PORT := ":8080"
log.Print("Running server on "+ PORT)
http.HandleFunc("/",CompleteTaskFunc)
log.Fatal(http.ListenAndServe(PORT, nil))
}
func CompleteTaskFunc(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(r.URL.Path))
}
Parameterized routing
The above example works fine, as long as it is simple enough, we need to make
modifications for handling parameterized routing.
//DeleteTaskFunc is used to delete a task,
//trash = move to recycle bin, delete = permanent delete
func DeleteTaskFunc(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" {
id := r.URL.Path[len("/delete/"):]
w.Write([]byte("delete the "+id+ " task"))
}
Implementation
88
webapp-with-golang-anti-textbook
Here we are checking the method type in each view, we can totally use a router like
httprouter or any other suitable one, but we want to see how things work in pure Go, thus we
aren't using a mux/router.
Note DELETE HTTP verb
Here you will notice that we aren't using the DELETE / request, this is because we'll use it
later in the book when we AJAXIFY the application, for now, we'll do it like GET /delete/1234,
just to get a hang of developing a fairly sophisticated application.
Since in a previous example we showed how to get hold of the URL in a view handler it is
only intuitive that when we process the handler we take the URL, then take a sub string of
the URL and remove the /delete/ part and we have the ID of the task which we'll now
delete.
This example makes use of the slicing concept. It is simple, if Path is our string variable
then Path[1:] is the substring which includes everything from the first character, index zero.
For static file serving, we use the FileServer method present in the http package which
serves the files which of the folder which is given as an argument of the function.
The FileServer is a built in method in net/http which is used to serve a file over HTTP.
Static files are equally as important for our webapplication as our other files are, because for
the matter static files are the files which are going to support the front end logic, like CSS/JS.
Homework
No book can just be read, you have to take some additional efforts on your side too
Read the documentation of net/http & log and get to know of the methods/functions
in the packages [1]
Find out how many alternatives are there to ListenAndServe.
Write a handler to serve static files, if file is available on that path then it should serve
the file, otherwise it must return an HTTP404 error.
Write a command line application to share files/folder over HTTP, the syntax should be
like this ./wshare -f file.pdf file on link = 192.168.2.1:8080.
Footnotes
Implementation
89
webapp-with-golang-anti-textbook
[1]: one amazing aspect of Go is its small library, at any point of time it is possible to know all
the functions of one package, this is not possible with other languages who have blown up to
the state where you can't program without a manual, but that doesn't mean that reading
manuals is bad, on contrary it is going to help if we read more documentation, which detail
the API.
Links
Directory
Previous section: Web Programming Basics
Next section: Designing our web app
Implementation
90
webapp-with-golang-anti-textbook
The Design
There is an urge for getting to program as soon as possible, but in Software Engineering,
actual programming constitutes around 5% of all the effort, for programming is just
translating ideas into computer executable code. Hence the time given to design is very
important for any application.
We decide on our API, for instance how exactly is our app going to behave?
91
webapp-with-golang-anti-textbook
file ~/main/main.go
package main
import (
"log"
"net/http"
)
func main() {
http.HandleFunc("/", ShowAllTasksFunc)
http.HandleFunc("/complete/", CompleteTaskFunc)
http.HandleFunc("/delete/", DeleteTaskFunc)
http.HandleFunc("/deleted/", ShowTrashTaskFunc)
http.HandleFunc("/trash/", TrashTaskFunc)
http.HandleFunc("/edit/", EditTaskFunc)
http.HandleFunc("/completed/", ShowCompleteTasksFunc)
http.HandleFunc("/restore/", RestoreTaskFunc)
http.HandleFunc("/add/", AddTaskFunc)
http.HandleFunc("/update/", UpdateTaskFunc)
http.HandleFunc("/search/", SearchTaskFunc)
http.HandleFunc("/login", GetLogin)
http.HandleFunc("/register", PostRegister)
http.HandleFunc("/admin", HandleAdmin)
http.HandleFunc("/add_user", PostAddUser)
http.HandleFunc("/change", PostChange)
http.HandleFunc("/logout", HandleLogout)
http.Handle("/static/", http.FileServer(http.Dir("public")))
log.Print("running on port 8080")
log.Fatal(http.ListenAndServe(":8080", nil))
}
92
webapp-with-golang-anti-textbook
In this file itself, create all functions we mentioned above and make the necessary changes
as per one definition that we show below
func ShowAllTasksFunc(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" {
message := "all pending tasks GET"
} else {
message := "all pending tasks POST"
}
w.Write([]byte(message))
}
In your browser type localhost:8080 and type all these URLs and see what message you
get.
Router
Currently we aren't using a router because we want to implement as much functionality as
possible ourselves, this, by no means says that routers are a waste. We do not use AJAX
right now, just plain HTTP so routers won't matter much. When we add in AJAX handlers,
then we need to route our requests to respective handlers so we'd make one AJAX handler
and one non AJAX handler.
This is because we follow the MVC architecture in our application, which is model-viewcontroller. Our routing logic will be done in our controller, which will direct our requests to our
handlers, the handlers will talk to our database and render templates. Views populate data
into our templates.
Homework
Check the documentation for http.ResponseWriter and http.Request objects and get to
know all the variables/functions/constants for http package and these two which we
mentioned
Links
Directory
93
webapp-with-golang-anti-textbook
94
webapp-with-golang-anti-textbook
Using databases in Go
Go doesn't provide out of the box support for any database, but it provides an interface,
which can be used by database library creators to keep all the database libraries compatible
with each other.
We will use the sqlite database for this book.
Use the following insert statements to enter data in our table, so we'll begin reading data in
our ShowAllTasks function which we wrote in the previous chapter
Database Handling
95
webapp-with-golang-anti-textbook
go get "github.com/mattn/go-sqlite3"
Accessing database in go
We import the library as import _ "github.com/mattn/go-sqlite3"
We are importing our driver anonymously, when we import any package default it gets
imported as per its name, for instance when we import net/http the exported
variables/functions be available in the program as http for accessing functions and
variables of the package, but since go-sqlite3 is a database package which uses the
database/sql interface, we don't want all our database code to be specific to go-sqlite3,
hence we give an empty operator as an alias so we can use the functions without
mentioning explicitly the package name & the none of exported fields are available. This is
where the interoperability of databases comes into the picture.
Under the hood, the driver registers itself to the database/sql package.
Database Handling
96
webapp-with-golang-anti-textbook
The _ operator has a unique role in go, it is used to denote an empty variable, used when
we want to ignore data.
nums is an array, while we loop through arrays using range it returns a (key,value) pair. here
Also note that we can give a package alias instead of the underscore character should we
want an alias for our package.
Every database has a connection mechanism, file for sqlite and IP address for
MySQL/Postgres. We define it as a global variable since we want it to be accessible in the
package to keep our database access logic all in one package rather than sprinkling it
around recklessly.
var database *sql.DB
database, err = sql.Open("sqlite3", "./tasks.db")
if err != nil {
log.Fatal(err)
}
own pool of idle connections. Thus, the Open function should be called just once. It is rarely
necessary to close a DB.
Database Handling
97
webapp-with-golang-anti-textbook
err = db.Ping()
if err != nil {
//do something about it
}
The sql.DB object shoudn't be opened and closed frequently, it should be closed only when
we no longer need to access the database.
Please refer to the documentation of database/sql for more details.
Note exported and unexported variables
In Go when a variable/function starts with a capital letter, it is public which means accessible
from packages which import the package. It is good practice to not keep any exported
variables which are not required.
We use the Query method to query the database when we expect some result from the
database.
Database Handling
98
webapp-with-golang-anti-textbook
This guarantees us that the file will be closed before the main function returns, both in case
of normal/abnormal termination. Take note of the behaviour of defer before using it in your
code. Becuase if it is misused it causes very difficult to find bugs.
file ~/main/main.go
Find and fix the bug:
Database Handling
99
webapp-with-golang-anti-textbook
package main
import (
_ "github.com/mattn/go-sqlite3"
"fmt"
)
var database *sql.DB
func init() {
defer database.Close()
database, err = sql.Open("sqlite3", "./tasks.db")
if err != nil {
fmt.Println(err)
}
}
//intentional bug exists, fix it
func main() {
getTasksql = "select id, title, content, created_date from task
where finish_date is null and is_deleted='N' order by created_date asc"
rows, err := database.Query(getTasksql)
if err != nil {
fmt.Println(err)
}
defer rows.Close()
for rows.Next() {
err := rows.Scan(&TaskID, &TaskTitle, &TaskContent, &TaskCreated)
TaskContent = strings.Replace(TaskContent, "\n", "<br>", -1)
if err != nil {
fmt.Println(err)
}
fmt.Println(TaskID, TaskTitle, TaskContent, TaskCreated)
}
err = rows.Err()
if err != nil {
log.Fatal(err)
}
}
Always defer rows.Close() , that way we free the database connection in the pool for so
long as the rows contains the result set, the database connection is in use and not
available in the connection pool. When the rows.Next() function returns EOF (End of File),
which means that it has reached the end of records, it'll call rows.Close() for you, the
Close() can be called multiple times without side effects.
Database Handling
100
webapp-with-golang-anti-textbook
Single-Row Queries
If a query returns at most one row, you can use a shortcut:
var taskDescription string
query:="select taskDescription from task where id = ?"
err = db.QueryRow(query, 1).Scan(&taskDescription)
if err != nil {
log.Fatal(err)
}
fmt.Println(taskDescription)
Errors from the query are deferred until Scan() is called, and then are returned from that.
You can also call QueryRow() on a prepared statement:
query :="select taskDescription from task where id = ?"
stmt, err := db.Prepare(query, 1).Scan(&taskDescription)
if err != nil {
log.Fatal(err)
}
var taskDescription string
err = stmt.QueryRow(1).Scan(&taskDescription)
if err != nil {
log.Fatal(err)
}
fmt.Println(taskDescription)
Database Handling
101
webapp-with-golang-anti-textbook
We always use forms to get values that we have to write to the database, we get the title,
content, priority and other required things and use the above function like functions to insert
data into the database.
Note Prepare (from the go documentation)
Prepare creates a prepared statement for later queries or executions. Multiple queries or
executions may be run concurrently from the returned statement. The caller must call the
statement's Close method when the statement is no longer needed.
Note Error handling
Go takes a different approach to exception handling than popular languages which use the
try catch statements, although this results in one list in the if err != nil land, but this
makes the language small and simpler to handle exceptions because we don't need to
remember those 1010101 different classes the language/library takes care to identify the
exception type for us and returns a error string, which we check for nil and then take some
action on it.
An excellent resource about Go and databases can be found at http://go-database-sql.org
The fault in our code:
Fixing the intentional bug in the above code:
Database Handling
102
webapp-with-golang-anti-textbook
func init() {
defer database.Close()
database, err = sql.Open("sqlite3", "./tasks.db")
if err != nil {
fmt.Println(err)
}
}
The defer keyword calls the function when the calling function exits, in our case, init is a
special function, so basically we are opening the database and closing it immediately. Since
init is the first function to be executed.
You might want to use call the database.Close() method from a wrapper in the views
package through the main package. In this way, if the main function which will be the
controller of our application is terminated, we'll free up the database resource.
Homework
See the /code/chapter-4/4.5database in our code repository and modify the file to insert
data from the 4.3formsupload folder. We have two working code set, one of printing form
values on the console and one of fetching db values and rendering a template. What you
have to do is based on this chapter, write methods to insert values from the form to the
database.
Links
Directory
Previous section: Basic Functionality
Next section: Basic Example
Database Handling
103
webapp-with-golang-anti-textbook
An Example
Expected output:
open your task.db file in sqlite3 like this
[Tasks] $ sqlite3 task.db
sqlite> select title from task limit 1;
Publish on github
Now this output should match with the one we see at localhost:8080
After running the file, go to localhost:8080 and localhost:8080/add
file ~/main/main.go
package main
import (
"database/sql"
"fmt"
_ "github.com/mattn/go-sqlite3"
"log"
"net/http"
"time"
)
var database *sql.DB
var err error
//Task is the struct used to identify tasks
type Task struct {
Id int
Title string
Content string
Created string
}
//Context is the struct passed to templates
type Context struct {
Tasks []Task
Navigation string
Search string
Message string
}
func init() {
Webapp Example
104
webapp-with-golang-anti-textbook
Webapp Example
105
webapp-with-golang-anti-textbook
Homework
The homework is to split the code into packages and get it to work, the type definition goes
into the types/types.go file, the handler definition goes into the views/views.go , the
database read and write methods go into the db/db.go . Make sure that after you refactor
the code, that the code runs.
Links
Directory
Previous section: Database setup
Next section: Closing remarks
Webapp Example
106
webapp-with-golang-anti-textbook
When we are populating this HTML page, we can also populate it dynamically, without
getting bogged down by syntax, ignore templating for the while We have a variable called
Navigation and one called Categories, we loop through Categories and if the Navigation is
equal to that category then the checked value is true.
Form handling
107
webapp-with-golang-anti-textbook
Category:
<select name="category" class="dropdown">
<option>---</option>
{{$navigation := .Navigation}} {{$categories := .Categories}}
{{range $cat := $categories}}
Here we have used a drop down box, you can use radio buttons like below
<input type="radio" name="gender" value="1">Female
<input type="radio" name="gender" value="2">Male
<input type="radio" name="gender" value="3">Other
Or checkboxes
<input type="checkbox" name="gender" value="female">Female
<input type="checkbox" name="gender" value="male">Male
<input type="checkbox" name="gender" value="other">Other
We get the value of a drop down box/radio button/checkbox on the server side by using the
name field like below:
value := r.Form.Get("gender")
value := r.FormValue("gender")
As we saw earlier, our webservers basically take a HTTP Request object and return an
HTTP Response Object, below is an example of a sample HTTP Request object.
The host is the IP address sending the req, User Agent: fingerprint of the machine, the
Accept- fields define various parts like the language, encoding Referer is which IP made the
call, Cookie is the value of the cookie stored on the system and Connection is the type of
connection.
In this request snapshot, we also have a file which we upload, for file upload, the content
type is multipart/form-data
Request Header
Form handling
108
webapp-with-golang-anti-textbook
Host: 127.0.0.1:8081
User-Agent: ...
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://127.0.0.1:8081/
Cookie: csrftoken=abcd
Connection: keep-alive
Request Body
Content-Type: multipart/form-data;
boundary=---------------------------6299264802312704731507948053
Content-Length: 15031
-----------------------------6299264802312704731507948053
Content-Disposition: form-data; name="title"
working with forms
-----------------------------6299264802312704731507948053
Content-Disposition: form-data; name="CSRFToken"
abcd
-----------------------------6299264802312704731507948053
Content-Disposition: form-data; name="content"
finish the chapter working with forms
-----------------------------6299264802312704731507948053
Content-Disposition: form-data; name="uploadfile"; filename="2.4workingwithform.md"
Content-Type: text/x-markdown
--file content------------------------------6299264802312704731507948053
Content-Disposition: form-data; name="priority"
3
-----------------------------6299264802312704731507948053--
If you had wondered how Google's home page shows a pop up when you visit google.com
on IE of Firefox, it checks your User-Agent. The thing with HTTP request is that they can be
modified to any extent, the Chrome developer tools gives you quite sophisticated tools to
modify your requests, even User Agent spoofing is a default feature avaiable, this feature is
available so we can test our webapps in one window simulating many internet browsers at
one go.
The basic part of working with forms is to identify which user that particular form belongs to,
there are ways to attain that, we can either have a stateful or a stateless web server.
Form handling
109
webapp-with-golang-anti-textbook
A stateless server doesn't store sessions, it requires an authentication key for each request
while a stateful server stores sessions. For storing sessions a cookie is used, which is a file
which is stored in the private memory of the web browser which we use. Only the website
which created the cookie can access the cookie, no third party websites can access the
cookies, but the OS user can read/edit/delete cookies using the web browser.
CSRF
CSRF stands for Cross Request Site Forgery. Any website can send a POST request to
your web server, who sent the request can be found in the Referer field of your HTTP
response. It is a form of confused deputy attack in which the deputy is your web browser. A
malicious user doesn't have direct access to your website, so it makes use of your browser
to send a malicious request. Typically cookies enable your browser to authenticate itself to a
webserver, so what these malicious websites do is, they send in a HTTP request on behalf
of your browser.
We can thwart this attack by restricting the referer to your own domain, but it is quite easy to
manipulate the misspelt referer field of a HTTP request.
Another way is to use tokens. While rendering our form, we send in a hidden field with crypto
generated string of 256 characters, so when we process the POST request, we first check if
the token is valid or not and then decide if the data came from a genuine source or from a
malicious source. It doesn't have to be malicious actually, even if a legitimate user tried to
trick your webserver into accepting data, we shouldn't entertain it.
To check the csrf token, we serve the token to the form and store it in a cookie, when we get
the POST request, we check if both are equal or not. This is because a malicious person
might trick a user to click on a form but they can't set cookies for your webapplication.
A point to note here is that never trust user data. Always clean/sanitize data which you get
from the end user.
Note Javascript
If you are serious about web development, you ought to learn Javascript in detail. While
building a web app, there will be times when you would want to improve the UI of your
application, which would mean a change in the html page. Using JS is inevitable while
building beautiful webapps, while adding some new html feature, open the "web inspector"
present in the developer tools and dynamically add the html code. The web inspector allows
us to manipulate the CSS and HTML part. Now open the javascript console, that enables
you to test the JS feature which you are willing to add. For instance, in the tasks application,
there was no provision to expand/contract the size of the task, so I added a button in the
web inspector,
Form handling
110
webapp-with-golang-anti-textbook
<button class="toggle"></button>
This proved that it works, so now go to your template and actually add the code. Make sure
the html is correct because while running, the html files are parsed once, so for any html
change, you have to run the web app for each HTML change. So once the html is set up, if
you change the JS/CSS then you just have to refresh the page, because the html page gets
the JS/CSS each time the page is loaded.
As we saw in the above paragraph, for preventing CSRF, we need to generate a token and
send as a hidden field in the form and store it in a cookie, when we get the POST request
from the
Forms in Go
In the below function we set the cookie, we first generate a CSRF token, which'll be unique
for each HTTP request which we get and store it in a cookie.
//ShowAllTasksFunc is used to handle the "/" URL which is the default ons
func ShowAllTasksFunc(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" {
context := db.GetTasks("pending") //true when you want non deleted notes
if message != "" {
context.Message = message
}
context.CSRFToken = "abcd"
message = ""
expiration := time.Now().Add(365 * 24 * time.Hour)
cookie := http.Cookie{Name: "csrftoken",Value:"abcd",Expires:expiration}
http.SetCookie(w, &cookie)
homeTemplate.Execute(w, context)
} else {
message = "Method not allowed"
http.Redirect(w, r, "/", http.StatusFound)
}
}
The below handler handles the POST request sent by our form, it fetches the value of the
csrftoken cookie and gets the value of the hidden CSRFToken field of the add task form. If the
value of the cookie is equal to the value fetched by the form, then we allow it to go to the
database.
Form handling
111
webapp-with-golang-anti-textbook
The call to ParseForm will parse the contents of the form into Gettable fields which we can
fetch using the Get function. This call is compulsory.
//AddTaskFunc is used to handle the addition of new task, "/add" URL
func AddTaskFunc(w http.ResponseWriter, r *http.Request) {
if r.Method == "POST" {
r.ParseForm()
file, handler, err := r.FormFile("uploadfile")
if err != nil {
log.Println(err)
}
taskPriority, priorityErr := strconv.Atoi(r.FormValue("priority"))
if priorityErr != nil {
log.Print("unable to convert priority to integer")
}
priorityList := []int{1, 2, 3}
for _, priority := range priorityList {
if taskPriority != priority {
log.Println("incorrect priority sent")
//might want to log as security incident
taskPriority=1 //this defaults the priority to low
}
}
title := template.HTMLEscapeString(r.Form.Get("title"))
content := template.HTMLEscapeString(r.Form.Get("content"))
formToken := template.HTMLEscapeString(r.Form.Get("CSRFToken"))
cookie, _ := r.Cookie("csrftoken")
if formToken == cookie.Value {
if handler != nil {
r.ParseMultipartForm(32 << 20) //defined maximum size of file
defer file.Close()
f, err := os.OpenFile("./files/"+handler.Filename, os.O_WRONLY|os.O_CREATE, 0666)
if err != nil {
log.Println(err)
return
}
defer f.Close()
io.Copy(f, file)
filelink :=
"<br> <a href=/https/www.scribd.com/files/" + handler.Filename + ">" + handler.Filename + "</a>"
content = content + filelink
}
truth := db.AddTask(title, content, taskPriority)
if truth != nil {
message = "Error adding task"
log.Println("error adding task to db")
} else {
message = "Task added"
log.Println("added task to db")
Form handling
112
webapp-with-golang-anti-textbook
}
http.Redirect(w, r, "/", http.StatusFound)
} else {
log.Fatal("CSRF mismatch")
}
} else {
message = "Method not allowed"
http.Redirect(w, r, "/", http.StatusFound)
}
}
Note Cookies
Cookie is a way to store data on the browser, HTTP is a stateless protocol, it wasn't built for
sessions, basically the Internet itself wasn't built considering security in mind since initially it
was just a way to share documents online, hence HTTP is stateless, when the web server
receives requests, it can't distinguish between two consequitive requests, hence the concept
of cookies were added, thus while starting a session, we generate a session ID and store it
on the database in memory or on the database and we store the same ID on a cookie on the
web browser and we validate both of them to authenticate them.
We have to note that, if we set an expiry date for a cookie, then it is stored on the filesystem
otherwise it is stored in memory in the browser. In the incognito mode, this is the case, all
the cookies are stored in memory and not in the filesystem.
HTTP Request
Host: localhost:8080
User-Agent: .....
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://localhost:8080/
Cookie: csrftoken=abcd
Connection: keep-alive
Cache-Control: max-age=0
The browser, while sending a response appends all the cookie data stored in its memory or
file along with the other aspects of the HTTP request so we can access the cookie as
r.Cookie , it'll contain every cookie for that particular domain, we'd then loop through it to
fetch the data which we want. This is important for security reasons, suppose I set a cookie
on my imaginedomain.com and if it contains a csrf token and if that cookie is accessible to
some other webapp then it is horrible since they can masquerade as any legitmate user. But
Form handling
113
webapp-with-golang-anti-textbook
this ain't possible, since a website can only access the cookie stored for its own domain,
plus we have the feature to set HTTP only value of a cookie as true, which says that even
javascript can't access the cookies.
HTTP Response
Content-Type: text/html; charset=utf-8
Date: Tue, 12 Jan 2016 16:43:53 GMT
Set-Cookie: csrftoken=abcd; Expires=Wed, 11 Jan 2017 16:43:53 GMT
Transfer-Encoding: chunked
When we set cookies, we write then to the HTTP response which we send to the client and
the browser reads the Cookie information and stores them in the memory or the filesystem
depending on the Expires field of the response.
From the go documentation:
type Cookie struct {
Name string
Value string
Path string // optional
Domain string // optional
Expires time.Time // optional
RawExpires string // for reading cookies only
// MaxAge=0 means no 'Max-Age' attribute specified.
// MaxAge<0 means delete cookie now, equivalently 'Max-Age: 0'
// MaxAge>0 means Max-Age attribute present and given in seconds
MaxAge int
Secure bool
HttpOnly bool
Raw string
Unparsed []string // Raw text of unparsed attribute-value pairs
}
Input Validation
The basic aspect of web applications is that no data can be trusted, even if the user isn't
malicious herself, there are many ways to trick the browser into sending HTTP requests and
fooling the web server to respond to what seems like a legitimate request. Hence we have to
verify everything that comes from the user.
Form handling
114
webapp-with-golang-anti-textbook
One might argue here that we do all sorts of validation using javascript, but there are ways
to evade JS validations, the simplest way is to disable JS and more sophisticated ways are
to manipulate the HTTP request before the browser sends it, it is literally trivial when we use
just the web developer tools that these days browsers provide.
if formToken == cookie.Value and title != nil and content!=nil
The title and content of the task is mandatory, hence we added the not nil part.
We do input validation when we don't want junk data to go into our database, suppose the
user hit the submit button twice, or some other scenario. But we also have to consider the
case where the user wants to run some script on our website, which is dangerous, so we
use the template.HTMLEscapeString method to escape what might run in memory of the
current browser session. Even the data which comes from your drop down list should be
validated.
Everything in a web form is sent via a Request as we saw in the above example, we use a
drop down list when we have are expecting a particular set of inputs but for someone who
knows what firefox dev tools are, can easily modify anything in the request, for example we
have the priority as drop down list we might think that since we have only three entries in the
drop down list, should we keep a check in our view handler to not accept anything but the
three values which are present in our template. We ought to keep a check like below:
priorityList := []int{1, 2, 3}
for _, priority := range priorityList {
if taskPriority != priority {
log.Println("incorrect priority sent")
//might want to log as security incident
}
}
All a malicious user has to do is change this value and resend the request, they can literally
insert any value here, just think what if they send rm -fr *.* and accidentally enough this
command is executed on our server. This also brings in another aspect of security, never run
your machine in root mode, always keep the root mode for admin tasks and use a non root
mode. Even if that isn't the case, a less dangerous example will be sending a huge number
Form handling
115
webapp-with-golang-anti-textbook
with the request, assuming that we have used integer as our variable, the program might
crash if it has to handle a number beyond its storage capacity. This might be termed as a
denial of service attack.
Links
Directory
Previous section: Basic Example
Next section: Uploading files
Form handling
116
webapp-with-golang-anti-textbook
Uploading files
Uploading files is the next step in form processing, in case of files, we send the entire file
data in the HTTP header, so we have to set the form encoding to enctype="multipart/formdata" . This will inform our server that we are going to get a file from the form along with the
We first provide the maximum size of the file which is 32 ^ 20, which is gigantic for our
webapp, not everyone has the Internet infrastructure to upload that big a file, but we want to
be flexible.
We basically get a file from the form request, in the form handler we open another file with
the same/different name and then read the file form the request and write it on the server.
We need to handle the scene where we name the file differently so we'd need to store the
old file name -> new file name relation somewhere, it can be a database table.
It is recommended to NOT keep the same name as the user, because user data can't be
trusted is the holy grail of web app development, so we need to escape the scrub the file for
malicious content/name.
We now want to randomize the file name of the files which the users upload. In your
AddTaskFunc add the following lines
Uploading Files
117
webapp-with-golang-anti-textbook
if handler != nil {
r.ParseMultipartForm(32 << 20) //defined maximum size of file
defer file.Close()
randomFileName := md5.New()
io.WriteString(randomFileName, strconv.FormatInt(time.Now().Unix(), 10))
io.WriteString(randomFileName, handler.Filename)
token := fmt.Sprintf("%x", randomFileName.Sum(nil))
f, err := os.OpenFile("./files/"+token, os.O_WRONLY|os.O_CREATE, 0666)
if err != nil {
log.Println(err)
return
}
defer f.Close()
io.Copy(f, file)
filelink := "<br> <a href=/https/www.scribd.com/files/" + token + ">" + handler.Filename + "</a>"
content = content + filelink
fileTruth := db.AddFile(handler.Filename, token)
if fileTruth != nil {
message = "Error adding filename in db"
log.Println("error adding task to db")
}
}
~/Tasks/db/db.go
// AddFile is used to add the md5 of a file name which is uploaded to our application
// this will enable us to randomize the URL without worrying about the file names
func AddFile(fileName, token string) error {
SQL, err := database.Prepare("insert into files values(?,?)")
if err != nil {
log.Println(err)
}
tx, err := database.Begin()
if err != nil {
log.Println(err)
}
_, err = tx.Stmt(SQL).Exec(fileName, token)
if err != nil {
log.Println(err)
tx.Rollback()
} else {
log.Println(tx.Commit())
}
return err
}
Uploading Files
118
webapp-with-golang-anti-textbook
table structure
CREATE TABLE files(name varchar(1000) not null, autoName varchar(255) not null);
Links
Directory
Previous section: Working with forms
Next section: Templating
Uploading Files
119
webapp-with-golang-anti-textbook
Templates
package: text/template
In the first chapter we had a cursory glance over the concept of templates. This chapter is
dedicated entirely to templates. As we said previously, a web application responds to certain
URLs and gives an html page to the browser which the browser then interprets and shows to
the end user. This html page which is sent to the browser is what is called templates in the
backend for we have a template which stores some variables, and in real time data is
provided into the template which makes it a complete html page.
Let's take a practical example. Suppose we are building a micro blogging site. We would
start with creating the front end in html. Our microblogging site will show Hi User on the
right corner.
In our static html we write this <p>Hi User</p>
But if we serve this page on our webserver it'll not change anything, it'll show Hi User , the
name of the user won't come magically, we have to put it into the page somehow, here we
use a variable so in Go we'd approach this by using a variable named {{.Name}} so our
html now will be <p>Hi {{.Name}}</p> . The . is mandatory.
Now, this is the logic we apply to all our html pages, keeping such {{}} variable expansion
parameters and serving the value of the parameter while executing the template. If you are
wondering how we'd do that, this is the syntax
homeTemplate.Execute(w, context)
//Context is the struct passed to templates
type Context struct {
Tasks []Task
Name string
Search string
Message string
}
These are the three parts of using templates, first you need to create types like we have
created the Context type, then we need to read the template, then we need to use the
components of that type in our template file. So what remains now is passing an object of
that type during template execution.
Every template requires the context object because that is what defines the data to be
populated in the template.
Templates
120
webapp-with-golang-anti-textbook
Reading template:
templates, err = template.Must(template.ParseFiles(allFiles...))
Note: template.Must
Must is a helper that wraps a call to a function returning (*Template, error) and panics if the
error is non-nil Where allFiles is populated as below:
var allFiles []string
templatesDir := "./public/templates/"
files, err := ioutil.ReadDir(templatesDir)
if err != nil {
fmt.Println("Error reading template dir")
}
for _, file := range files {
filename := file.Name()
if strings.HasSuffix(filename, ".html") {
allFiles = append(allFiles, templatesDir+filename)
}
}
For the sake of demonstration of how to parse multiple files we have used the ParseFiles
method to parse all the .html files, you can use the ParseGlob method which is available
in the standard library.
template.Must(template.ParseGlob(templatesDir + "*.html"))
... operator : allFiles is a string slice and allFiles... passes the function a parameter
as a string.
2. ParseFiles performance:
There is one point to note here about performance in parsing files, typically a template
file won't change until there is some major change to the codebase so we should only
parse the files once, rather than keep this code in each view handler and doing a
Templates
121
webapp-with-golang-anti-textbook
template.ParseFiles("home.html")
template.Execute(w, context)
This block of code will unnecessarily read the html page each time while serving the
response of a request, which means if ten people are using our blogging site then for
each page they visit the code will read the html page, but there is no need to do things
this way, we can read the html files once at the start and then use the Lookup method
of the template class,
homeTemplate = templates.Lookup("home.html")
homeTemplate.Execute(w, context)
Sub templating
We learnt how to pass data to templates and display it in the html page, it so happens that a
lot of code is used in all templates suppose the navigation bar or the header, then we need
not write the same chunk everywhere, we can create a template to store that, and use sub
templating {{template "_head.html" .}}
Here, we have identified a chunk of code which we want to replicate and put it in
_head.html . Then we put the above statement in each template file where we wish to have
our header. This way templates become a lot smaller and don't contain replicated code
everywhere. Do note that the . before the first } is intentional and it means that all the
variables which were passed to the current template are passed to the sub template.
The sub templates which we create depends on our requirement, but the basic point behind
it is that if we are going to repeat a block of HTML code then we should form it as a
template.
Note Using Sub Templates
The main point to note over here is that when we are going to use our templates or sub
templates, all those html files need to be parsed. The basic point in templating is that we
have a variable which stores all templates even if we aren't going to refer to that directly in
our code using the Lookup method on the template variable. The lookup method takes the
name of the template. When the {{template _head.html .}} is evaluated, it goes to our
template variable and tries to find out the template parsed with the exact name, if it is not
present then it doesn't complain by default, we should use Must method if we want it to
complain.
Templates
122
webapp-with-golang-anti-textbook
Example
file views/views.go
package views
import (
"io/ioutil"
"net/http"
"os"
"strconv"
"strings"
"text/template"
)
var (
homeTemplate *template.Template
deletedTemplate *template.Template
completedTemplate *template.Template
loginTemplate *template.Template
editTemplate *template.Template
searchTemplate *template.Template
templates *template.Template
message string
//message will store the message to be shown as notification
err error
)
//PopulateTemplates is used to parse all templates present in
//the templates folder
func PopulateTemplates() {
var allFiles []string
templatesDir := "./public/templates/"
files, err := ioutil.ReadDir(templatesDir)
if err != nil {
fmt.Println("Error reading template dir")
}
for _, file := range files {
filename := file.Name()
if strings.HasSuffix(filename, ".html") {
allFiles = append(allFiles, templatesDir+filename)
}
}
if err != nil {
fmt.Println(err)
os.Exit(1)
}
templates, err = template.Must(template.ParseFiles(allFiles...))
// templates, err := template.Must(template.ParseGlob(templatesDir+".html"))
if err != nil {
Templates
123
webapp-with-golang-anti-textbook
fmt.Println(err)
os.Exit(1)
}
homeTemplate = templates.Lookup("home.html")
deletedTemplate = templates.Lookup("deleted.html")
editTemplate = templates.Lookup("edit.html")
searchTemplate = templates.Lookup("search.html")
completedTemplate = templates.Lookup("completed.html")
loginTemplate = templates.Lookup("login.html")
}
//ShowAllTasksFunc is used to handle the "/" URL
//TODO add http404 error
func ShowAllTasksFunc(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" {
context := db.GetTasks("pending")
//true when you want non deleted notes
if message != "" {
context.Message = message
}
homeTemplate.Execute(w, context)
message = ""
} else {
message = "Method not allowed"
http.Redirect(w, r, "/", http.StatusFound)
}
}
Templates
124
webapp-with-golang-anti-textbook
The {{ if .Tasks}} block checks if the array is empty or not, if it is not then it'll go to the {{
.range .Tasks }} which will loop through the array, then the {{.Title}} will access the title
and {{.Content}} will access the Content of that particular Task instance and we'll see all
the tasks as a list.
Template variables
We have this scenario, we have a range of categories in the navigation drawer and if we visit
the /category/study , then our category name should be highlighted in the navigation
drawer. We do this by storing the value of the .Navigation field - which tells if it is a Edit
page/Trash page/Category page
{{$nav := .Navigation}}
{{ range $index, $cat := .Categories }}
<li class="sidebar-item">
<a href="/category/{{$cat.Name}}" {{ if eq $cat.Name $nav }} class="active" {{end}}>
Creating variables
{{$url := ""}} will create a blank variable, one has to not that all variables are practically
strings once they are rendered. {{$url := .Navigation}} will create a new variable and
initialize it with the value of .Navigation
For understanding why template variables are required, we need to go into the above block
of code, when we are using the range operator, we are parsing the array and the range
block gets the elements of the block by default.
My Context type is
type Context struct {
Tasks []Task
Navigation string
Search string
Message string
CSRFToken string
Categories []CategoryCount
Referer string
}
Templates
125
webapp-with-golang-anti-textbook
This will mark only that particular category as active and not all of them.
In templating logic the operator is first and then the two operands.
eq: equal, le: less than equal, ge: greater than equal
You can also use if else clause like below:
{{$url := ""}}
<div class="navbar-header">
{{$url := ""}} {{else if eq .Navigation "edit"}} {{$url := ""}} {{else}} {{$url := "/category"}}{
<p class="navbar-brand" href="{{$url}}/{{.Navigation}}">
Here we had some complicated stuff, if our page is a search one, we had to show Results
for : <query> , pending, deleted,edit, completed for respective and /category/ if we are in
the category. So we defined an empty URL and assigned the URL values according to the
complicated if else structure.
Homework
1. Take the html pages from http://github.com/thewhitetulip/omninotesweb and modify
them to suit our purposes We would need to create one template each for the ones we
mentioned in the above variable declaration, use templating as far as possible and later
Templates
126
webapp-with-golang-anti-textbook
Links
Directory
Previous section: Uploading Files
Next section: Working with forms
Templates
127
webapp-with-golang-anti-textbook
Authentication
Authentication is used to verify if the users have access to that particular part of your web
application. For understanding how to implement authentication we need to understand what
happens behind the scenes of a browser. Suppose we run a bank webapplication. We want
only our legitimate users to access our webapplication. We set up a login page and provide
our users with their username and password which they can use to validate their claim to our
webapp.
When we submit the login form, the browser takes our username, password and sends a
POST request to the webserver, which again responds with a HTTP redirect response and
we are returned to our bank dashboard.
The HTTP protocol is stateless, which means every request is unique. There is no way for
identifying automatically if a request is related to another request. This brings about the
problem of authentication, how then can we validate if the users have access to our
webapp?
We can send the username along with each HTTP request, either in the URL via a GET
request or in the POST request. But this is inefficient since for each request, the webserver
would need to hit the database to validate the username, also this would mean weak
security since if I know your username, I can impersonate you pretty easily and the
webserver is helpless to identify this impersonation.
To solve this problems Sessions were invented, sessions need to use cookies on the
browser to function. The basic idea is that the server generates a sessionID and stores it in
a cookie. With subsequent requests, the browser will send the sessionID along with the
request, the webserver will then come to know from that sessionID if the request is a fake
one or not. Also we get to know who the user is from that.
Cookies
Cookies, as we saw in a previous chapter can be used to store a key,value pair. We used a
cookie to store the CSRF token, the cookie had the name as CSRF and value as the token.
Please don't confuse sessions with cookies, because sessions aren't a key,value pair.
Sessions are a way of working with cookies on the server side. There is a gap of the entire
Internet between sessions and cookies.
User Authentication
128
webapp-with-golang-anti-textbook
Cookies are stored in our browsers, for security reasons we need to enable the
"isHTTPOnly" field of our cookies, so only our webapplication can read the cookie.
Otherwise anyone javascript application can easily read our cookie defeating its purpose, we
might as well not keep an authentication mechanism for our webapp.
From the go documentation type Cookie struct { Name string Value string
Path string // optional
Domain string // optional
Expires time.Time // optional
RawExpires string // for reading cookies only
// MaxAge=0 means no 'Max-Age' attribute specified.
// MaxAge<0 means delete cookie now, equivalently 'Max-Age: 0'
// MaxAge>0 means Max-Age attribute present and given in seconds
MaxAge int
Secure bool
HttpOnly bool
Raw string
Unparsed []string // Raw text of unparsed attribute-value pairs
}
The domain of our cookie enables a restricted access to our cookie. A visitor goes to our
fictional bank website, sbank.com and enters a username and password, a cookie is stored
in the browser which only the sbank.com domain can access since we are security aware
and we have set the HttpOnly field to true while setting the cookie. This means some
malicious website which is set up by an attacker to intentionally target our bank isn't able to
access the cookie using javascript.
One has to remember that cookie is nothing but a file stored in a user's browser, if can be
accessed over HTTP by our webserver, a client browser does allow access to it through
browser settings or custom javascript. The browser, after all is a platform, and we have APIs
to that platform.
Sessions
A session is a series of actions performed between you and the webapp you are acting,
enabled by the browser and the Internet you are using. While generating a new session, we
need to check if a sessions is already active, if so we return the same session ID rather than
create a new one, if not, we generate a new session ID. Session IDs need to be sufficiently
random. Of course we can't generate something totally random, but we have to ensure to
generate something that nobody else can replicate, unless they have access to our private
key which we use to generate our random number.
User Authentication
129
webapp-with-golang-anti-textbook
login/logout reset password sessions token based authentication store session in database
not memory
Links
Directory
Previous section: Templates
Next section: Working with files
User Authentication
130
webapp-with-golang-anti-textbook
Files
JSON and XML are two of the most common ways to transmit data between web
applications. We'll use JSON for our configuration file.
For our webapplication we have a set of configuration values like the server port where our
application will run. Suppose you are developing your application in $GOPATH and also
using it somewhere else, then you can't run them in parallel because both sources use the
same port number. Naturally we want a way to parameterize that port number. The
parameter or configuration value list may contain more things like database connection
information. As of now we will use a config.json file and read the serverPort variable and
bind our server on that port.
Our configuration file uses a fixed structure, hence it is simple enough to UnMarshal it to a
struct type, we can use some advance concepts to accomodate unstructured JSON files,
because that is the whole point of JSON, we can have data in an unstructured format.
NoSQL has been famous lately, they are basically JSON document stores. We have projects
like boltdb which store data in a key value pair, ultimately in flat files or in memory.
file: $GOPATH/src/github.com/thewhitetulip/Tasks/config/config.go
131
webapp-with-golang-anti-textbook
package config
import (
"encoding/json"
"io/ioutil"
"log"
)
// Stores the main configuration for the application
// define a struct object containing
type Configuration struct {
ServerPort string
}
var err error
var config Configuration
// ReadConfig will read the config.json file to read the parameters
// which will be passed in the config object
func ReadConfig(fileName string) Configuration {
configFile, err := ioutil.ReadFile(fileName)
if err != nil {
log.Fatal("Unable to read log file")
}
//log.Print(configFile)
err = json.Unmarshal(configFile, &config)
if err != nil {
log.Print(err)
}
return config
}
file: $GOPATH/src/github.com/thewhitetulip/Tasks/config.json
{
"ServerPort": ":8081"
}
file: $GOPATH/src/github.com/thewhitetulip/Tasks/main.go
values := config.ReadConfig("config.json")
// values is the object now, we can use the
// below statement to access the port name
values.ServerPort
132
webapp-with-golang-anti-textbook
We use the json.Unmarshal to read the JSON file into our structure object. This is a very
simple and basic example of parsing JSON files, you can have nested structures of many
levels inside the main config object, but that is the features of Go, so long as it can be
represented as a JSON document you can use the Unmarshal method to translate the file
into an object which you can use in your program.
Homework
Alter the config.json file to take the name of the sqlite database as a configuration
parameter.
Read about the JSON library in godoc
Links
Directory
Previous section: Working with forms
Next section: Routing
133
webapp-with-golang-anti-textbook
Routing
Till now we used routing directly inside of our handlers. For a large application though, it'd be
better to have a router in place. We can either use a third party one with countless features
or the standard mux.
As our application matures, routing plays a big role into it. We intentionally avoided routing
till now because as a web developer, we must understand what happens in the background
of our application. Web frameworks allow us to build applications quickly, but the downside
of them is that they provide us with a framework as the name suggests, which means you
are totally restricted by the API which the framework will provide. Thus we need to know how
to implement bare bone stuff, so in future we might want to modify the framework we need,
or rather create our own one.
First of all we'd need to install the httprouter, do a go get -u
github.com/julienschmidt/httprouter
Routing
134
webapp-with-golang-anti-textbook
package main
import (
"fmt"
"github.com/julienschmidt/httprouter"
"net/http"
"log"
)
func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
fmt.Fprint(w, "Welcome!\n")
}
func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}
func main() {
router := httprouter.New() // creates a new router
router.GET("/", Index) // will direct the GET / request to the Index function
router.GET("/hello/:name", Hello) // will redirect the GET /name to Hello, stores the name of the
// in the a variable of httprouter.Params
log.Fatal(http.ListenAndServe(":8080", router))
}
Routing
135
webapp-with-golang-anti-textbook
rather than that we can have 3 different handlers one for when request comes via AJAX, one
for normal GET and one for normal POST. This way we do not have one function handler
checking the type of request and then writing three separate logic inside one function.
But that doesn't mean we have to use httprouter, if in an app there aren't much sophisticated
routing required then we can use the default Mux since it is good enough, but if we have
complicated routing then httprouter is the best.
Homework
Read the httprouter documentation and source code to get a deeper understanding of
routers, since routers are an integral part of any web application. Then rewrite our
application using the httprouter.
Links
Directory
Previous section: Working with files
Next section: Advance topics
Routing
136
webapp-with-golang-anti-textbook
1.
Contributors
137