jail(8)

Hacking UNIX with FreeBSD jail(8), Secure Virtual Servers

Presentation for DefCon 14, by Isaac Levy, (.ike)
 .ike Context
I have used jails extensively for web
application servers and software
development purposes

the methodology Im presenting here is

attempting to be stock UNIX (no ike-
specific magic formulas)

I am not a jail author, no commit bit...

Warranty / Announcement
Ill be out and about later if anyone has
more complex questions or strategies
they want to discuss

Im *trying* to stick to classic UNIX

process and ideas, and stock
methodology (no ike-specific magic)

Im assuming you all know your way

around various *NIX Operating Systems
scale, patterns, complexity
(a big picture exercise)
 http://www.powersof10.com/
Film: Powers of Ten, 1977, Charles and Ray Eames http://www.youtube.com/watch?v=4i6B7HzijSo
 http://www.powersof10.com/
Film: Powers of Ten, 1977, Charles and Ray Eames http://www.youtube.com/watch?v=4i6B7HzijSo
 http://www.powersof10.com/
Film: Powers of Ten, 1977, Charles and Ray Eames http://www.youtube.com/watch?v=4i6B7HzijSo
 1
da
ed0
hd2 da0 ed1
tmp
em
m
var mnt em
km
dev
null
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
e

use
m
ho

in usr r
sb us
bin loc er
al
jails
in
sb

src ports
bin

BSD UNIX

Internet universe, (according to ike, today.)

Open Systems Interconnection (OSI) Reference Model
Upper Layers Lower Layers
Application Presentation Session Layer Transport Layer Network Layer Data Link Layer Physical Layer
Layer (7) Layer (6) (5) (4) (3) (2) (1)

e-mail POP/SMTP POP/25

RS-X, CAT 1
Internet
Transmission
Protocol
Newsgroups Usenet 532 Control SLIP, PPP
Version 6 ISDN
Protocol (TCP)
(ipv6)
Web
HTTP 80
Applications ADSL

File Transfer FTP 20/21

ATM

Host Sessions Telnet 23

Directory 802.11 SNAP

DNS 53 FDDI
Services
Internet
User Datagram Protocol
Network Mgmt. SNMP 161/162 CAT 1-5
Protocol (UDP) Version 4
(ipv4)
RPC
File Services NFS Ethernnet II Coaxial Cables
Portmapper
 1
da
ed0
hd2 da0 ed1
tmp
em
m
var mnt em
km
dev
null
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
e

use
m
ho

in usr r
sb us
bin loc er
al
jails
in
sb

src ports
bin

BSD UNIX

Internet universe, (according to ike, today.)

yadda yadda
 1
da
ed0
hd2 da0 ed1
tmp
em
m
var mnt em
km
dev
null
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
e

use
m
ho

in usr r
sb us
bin loc er
al
jails
in
sb

src ports
bin

BSD UNIX

Internet universe, (according to ike, today.)

 1
da
UNIX hd2 da0
ed0
ed1
tmp
em
m
var mnt m
e
km
dev
null

de
v
boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin

e
use

m
ho
in usr r
sb us
n
bi loc er
al
in
sb

src ports
n
bi
UNIX

devices

de
v
kernel

userland
 1
da
UNIX hd2 da0
ed0
ed1
tmp
em
m
var mnt m
e
km
dev
null

de
v
boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin

e
use

m
ho
in usr r
sb us
n
bi loc er
al
in
sb

src ports
n
bi
 UNIX Spiral Galaxy
1
da
ed0
hd2 da0 ed1
tmp
em
m
var mnt em
km
dev
null
de
rc.d boot
v
etc
FreeBSD proc
lib kernel
sbin root
bin
e

use
m
ho

in usr r
sb us
n
bi loc er
al
in
sb

src ports
n
bi

Spiral Galaxy NGC 1232

Our world is complex

(thx Dan Geer & ShmooCon).
UNIX Helium Atom

devices
de
v
kernel

userland

Our world is simple too...

 1
da
UNIX hd2 da0
ed0
ed1
tmp
em
m
var mnt m
e
km
dev
null

de
v
boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin

e
use

m
ho
in usr r
sb us
n
bi loc er
al
in
sb

src ports
n
bi
Mandelbrot Fractal - Julia set
 1
da
ed0
hd2 da0 ed1
tmp
em
m
var mnt
mem
k
dev

de
v
boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin

e
use
m
in usr ho r
sb us
n
bi loc e r
al
in
sb

src ports
jails
virtual
n
bi

UNIXs
 d
ed0
hd 2 d a0 ed1
tmp
em
m
var mnt
mem
k
dev

de
v
boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin

e
use

m
ho
in u sr r
sb us
n
bi loc er
al
in
sb

jails src ports

n
bi
 var mnt m
m e
k
dev

de
v
boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin

e
use

m
ho
in us r r
sb
n us
bi loc e r
al
in
sb

jails src ports

n
bi
 de
v
boot
etc
FreeBSD proc
kernel
in root
in

e
use

m
ho
n usr r
n us
bi loc er
al
in
sb

jails src ports

n
bi
 kernel
root

e
use

m
ho
usr r
us
loc e r
al

jails src ports

 root

e
use

m
ho
us r r
us
er

src ports
 us
er

ort s
 us
er

ts
 us
er

ts
virtual
UNIXs
 You get the idea-

So what real-world contexts

warrant virtualizing the
ENTIRE operating system?
external
security
threats

development
messes
Mutually Untrusted Users
Mutually Untrusted Users
Mutually Untrusted Users
Mutually Untrusted Users

telnet forever!
 Mutually Untrusted Users

login:admin su
pass:love 24/7 ?
Mutually Untrusted Users
Mutually Untrusted Users
Mutually Untrusted Users

You run
*WHAT* as
CGI?
Mutually Untrusted Users

programs are users too...

Mutually Untrusted Users

muscle memory kills!

Harmony.
Once upon a time,
wasnt UNIX *fun*?

http://mckusick.com/beastie/
maintaining old junk?
Rack full of stuff Example:

3 webservers
1 local-use dns cache
fileserver (for 2 people)
2 dev servers
 jail(8)!
Rack full of stuff ,
becomes 1u server!
host:/path/to/jaildir/

ed0
hd2 da0 ed1
tmp
em
m
var mnt em
km
dev
null
de
v

boot
rc.d
etc
/dev/null proc
kernel

Jailing Server 192.168.1.10

lib
sbin root
bin
e

user
m
ho

in usr
sb us
bin loca er
l
in
sb

src ports
bin

1
da
ed0
hd2 da0 ed1
tmp
m
me
var mnt em
km

Jail 1 192.168.1.11
dev
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
me

use
ho

n usr r
sbi use
bin loca r
l
in
sb

jails src ports

bin

1
da
ed0
hd2 da0 ed1
tmp
m
me
var mnt em

Jail 2 192.168.1.12
3 webservers
km
dev
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
me

use
ho

n usr r
sbi use
bin loca r
l
in
sb

jails src ports

bin

1
da
ed0
hd2 da0 ed1
tmp
m
me

Jail 3 192.168.1.13
var mnt em
km
dev
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel

1 local-use dns cache

sbin root
bin
me

use
ho

n usr r
sbi use
bin loca r
l
in
sb

jails src ports

bin

1
da
ed0
hd2 da0 ed1
tmp

Jail 4 192.168.1.14
m
me
var mnt em
km
dev
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
me

use
ho

n usr r
sbi use
bin loca r
l
in

fileserver (for 2 people)

sb

jails src ports

bin

1
da
ed0
hd2 da0 ed1
tmp
m
me

Jail 5 192.168.1.15
var mnt em
km
dev
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
me

use
ho

n usr r
sbi use
bin loca r
l
in
sb

jails src ports

bin

2 dev servers
1
da
ed0
hd2 da0 ed1
tmp
m
me

Jail 6 192.168.1.16
var mnt em
km
dev
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
me

use
ho

n usr r
sbi use
bin loca r
l
in
sb

jails src ports

bin

1
da
ed0
hd2 da0 ed1
tmp
m
me
var mnt em

Jail 7 192.168.1.17
km
dev
de
v

boot
rc.d
etc
FreeBSD proc
lib kernel
sbin root
bin
me

use
ho

n usr r
sbi use
bin loca r
l
in
sb

jails src ports

bin
jail(8)
 Definitions
what is a jail(8):
a user space utility, like ifconfig(8)
produces a virtual system image
process tree based
what is jail(2):
a system call to imprison a process
it calls chroot and attaches to IP
a very few lines of source code!
 Definitions

what jail is not:

it is not a classical machine emulator
it is not chroot (jail vocabulary is
commonly misused with other *NIX
cultures)
 Great Uses for jail(8)
hardware resource sharing, an entire OS can
be dedicated to a given service

securely separate untrusted users/processes

learning/development/testing/hacking
insane high availability possibilities
honeypots
highly vulnerable network services
 Poor Uses for jail(8)
kernel access (you dont get a kernel)
limited network interface access
limited device driver access
when chroot(8) will simply do the job
some applications require particular low-level
system calls:

Notably, PostgreSQL doesnt run (securely)

in jails based on SysV IPC
 How To jail(8)
DEFINITIVE instructions in jail man pages,
1. compile a FreeBSD userland from source
somewhere on host machine, minor
tweaks.
2. create an IP alias on a network interface
3. run the jail(8) call with the IP, and userland,
to boot the jail, (so to speak).
 Practical Comparison

1 host:/path/to/jaildir/
da
ed0
da0 ed 0
hd2 ed 1 da0
tmp
tmp hd2 ed 1
em
var
m em
mnt em var
m
km mnt em
km
dev
null dev
de

null

de
v

boot
rc.d

v
boot
etc rc.d
FreeBSD proc etc
/dev/null proc
lib kernel kernel
lib
sbin root root
sbin
bin bin
e

use
m

e
use

m
ho

in usr r

ho
sb in usr r
n us sb us
bi loc er
bi
n
loc er
a l a l
in

in
sb

src ports
sb
src ports
n

n
bi

bi
making a jail
Host Machine
 preflight (simple)
1. get source to build with (cvsup is great)

2. make somewhere for the jails to live

(partitions, disk mounts, etc...)

3.make somewhere for jail-related start/mgmt

scripts to live
(starting jails from /etc/rc.d/jail can
thrash violently in most contexts! Bad!)
preflight- (man, definitive)
preflight- (build from src)
preflight- (build from src)
preflight- (build from src)
b kernel
sbin
bin

e
m
ho
in usr
sb u
i n
b loc
al
in
sb

jails src ports

n
bi
preflight- (build from src)
 cal
preflight
in

ja ils s rc p
n
bi

$D
preflight- (build from src)

compile!
preflight- (build from src)

compile!
preflight host:/path/to/jaildir/

lib

ports
preflight host:/path/to/jaildir/

tmp

var mnt

etc

lib
sbin
bin

e
m
ho
in usr
sb
n
bi loc
al

ports
preflight- (mount /dev)
preflight- (mount /dev)
preflight- (null kernel)
preflight- (null kernel)
preflight host:/path/to/jaildir/

tmp

var mnt

etc

lib
sbin
bin

e
m
ho
in usr
sb
n
bi loc
al

ports
preflight host:/path/to/jaildir/

1 ed0
da da0 ed1
tmp
em
m
var mnt m
e
km
dev
null

de
v
etc

lib
sbin
bin

e
m
ho
in usr
sb
n
bi loc
al
preflight Common Question:
Why isnt there an
automated build
system for this stage?

- Take care with the

build procedure, its
better to automate
things later, once you
have basics setup.

(network, users,
packages, time, etc.)
preflight- (config host)

jailinghost:/etc/rc.conf (stock)
preflight- (config host)

jailinghost:/etc/rc.conf
preflight- (master system)

jailinghost:/etc/ssh/sshd_conf
 1
da
ed0
hd2 da0 ed1
tmp
em
m
var mnt m
e
km
dev

de
v
boot
rc.d
etc

lib
sbin
FreeBSD
kernel
proc

root
?
bin

e
use

m
ho
in usr r
sb us
n
bi loc e r
al
in
sb

jails src ports

n
bi
configure - call jailed sh
(analagous to booting a machine in su mode)

Text
configure - call jailed sh
(analagous to booting a machine in su mode)

Text
configure - call jailed sh
(analagous to booting a machine in su mode)

Text
configure - call jailed sh
configure the jail, inside the jail

Text
configure - call jailed sh
configure the jail, inside the jail

Text
configure - call jailed sh
sysctl, whee!

Text
configure - call jailed sh
root pw

Text
configure - call jailed sh
root pw

Text
configure - call jailed sh
add users

Text
configure - call jailed sh
add users

Text
configure - call jailed sh
add users

Text
configure - call jailed sh
set timezone

Text
configure - call jailed sh
set timezone

Text
configure - call jailed sh
set timezone

Text
configure - call jailed sh
set timezone

Text
configure - call jailed sh
set timezone

Text
configure - call jailed sh
network options...

Text
configure - call jailed sh
run ssh, important

Text
configure - call jailed sh
check rc.conf in jail

Text
configure - call jailed sh
check rc.conf in jail

Text
configure - call jailed sh
jail-specific stuff (just use common sense)

Text
configure host:/path/to/jaildir/

1 ed0
da da0 ed1
tmp
em
m
var mnt m
e
km
dev
null

de
v
etc
/dev/null
lib kernel
sbin
bin

e
m
ho
in usr
sb
n
bi loc
al
configure host:/path/to/jaildir/

re0 1 ed 0
tmp da da0 ed 1

em
m
var mnt
mem
k
dev
null

de
v
rc.d
etc
/dev/null
lib kernel
sbin root
bin

e
use

m
ho
in usr r
sb us
n
bi loc er
al
in
sb

n
bi
configure - call jailed sh
were finished configuring jailed system!

Text
configure - assign ip alias
(use ifconfig)

Text
configure - assign ip alias
(ip for the jail)

Text
configure - assign ip alias
(original ip for the host machine)

Text
configure - call jailed sh
(analagous to booting a machine in su mode)

Text
configure - call jailed sh
(analagous to booting a machine in su mode)

Text
preflight host:/path/to/jaildir/

re0 1 ed 0
tmp da da0 ed 1

em
m
var mnt
mem
k
dev
null

de
v
rc.d
etc
/dev/null
lib kernel
sbin root
bin

e
use

m
ho
in usr r
sb us
n
bi loc er
al
in
sb

n
bi
 preflight host:/path/to/jaildir/

192.168.1.200
1 ed 0
tmp da da0 ed 1

em
m
var mnt
mem
k
re0 dev
null

de
v
rc.d
etc
/dev/null proc
192.168.1.2 lib kernel
sbin root
192.168.1.200
192.168.1.x bin

e
use

m
ho
in usr r
sb us
n
bi loc er
al
in
sb

n
bi
 start tangent! (script),
remember how I said rc.d is usually a bad idea?
 start!
were gonna start the jail manually here....
 start!
were gonna start the jail manually here....
 start!
type some random junk to seed entropy,
 start!
jail finished starting
 running
jls(8) lists running jails, gives a jail ID
 using the jail
ssh into the jail, treat it like a server.
 using the jail
ssh into the jail, treat it like a server.
 using the jail
ssh into the jail, treat it like a server.
inside the jail
just like any new server
inside the jail
just like any new server
inside the jail
just like any new server
inside the jail
you have root!
 inside the jail
how do you know you are inside a jail?

http://www.freebsd.org/cgi/query-pr.cgi?pr=95977 - will explain this url later...

stop and start jail
exit the jail, (ssh)
 stop and start jail
look at jailed processes (man page goodies)
 stop and start jail
look at jailed processes (man page goodies)
stop and start jail
use killall with -j flag
 stop and start jail
watch out for stacking mount points!
 stop and start jail
watch out for stacking mount points!
stop and start jail
restarting with the script this time,
stop and start jail
restarting with the script this time,
 stop and start jail
now the jid has incrimented once, to 6
 running processes
jexec to check processes (bad idea, in practice)
 Practical Comparison

1 host:/path/to/jaildir/
da
ed0
d a0 ed 0
hd2 ed1 da0
tmp
tmp hd2 ed 1
em
var
m em
mnt em var
m
km mnt em
km
dev
dev
de

null

de
v

boot
rc.d

v
boot
etc rc.d
FreeBSD proc etc
/dev/null proc
lib kernel kernel
lib
sbin root
sbin root
bin bin
e

use
m

e
use

m
ho

in usr r

ho
sb in usr r
n us sb us
bi loc e r bi
n
loc er
al a l
n

in
i
sb

src ports

sb
jails src ports
n

n
bi

bi

host jail
 Process Tree:
JailingServer
\_init
\_daemon/process etc...
\_daemon/process etc...
host \_daemon/process etc...
\_daemon/process etc...
jail
d1 \_jail (Jail 1)
\_daemon/process etc... host:/path/to/jaildir/

em \_daemon/process etc... e d0

m \_daemon/process etc...
tmp hd2 da0 e d1

em

em
m
var mnt em

m \_jail (Jail 2) km

k dev
null

de
\_daemon/process etc...

v
boot
rc.d
etc
/dev/null proc
\_daemon/process etc... lib kernel
root
sbin
\_daemon/process etc... bin

e
use

m
ho
in usr r
sb
\_jail (Jail 3) bi
n
loc
al
us
er

\_daemon/process etc...

in
sb
src ports

n
proc

bi
\_daemon/process etc...
\_daemon/process etc...
root \_jail (Jail 4)
\_daemon/process etc...
\_daemon/process etc...
use \_daemon/process etc...
r
us
er
d1

em
m
mem
k

host:/path/to/jaildir/

e d0
proc tmp hd2 da0 e d1

em
m
var mnt em
km
dev
null

de
root

v
boot
rc.d
etc
/dev/null proc
lib kernel
sbin root
bin

e
use

m
ho
in r
use
usr
sb us
n
bi loc er

r al

host jail
n
i
sb

src ports
n
bi

us
er
d1

em
m
mem
k

host:/path/to/jaildir/

e d0
proc tmp hd2 da0 e d1

em
m
var mnt em
km
dev
null

de
root

v
boot
rc.d
etc
/dev/null proc
lib kernel
sbin root
bin

e
use

m
ho
in r
use
usr
sb us
n
bi loc er

r al

host jail
n
i
sb

src ports
n
bi

us
er
d1

em
m
mem
k

host:/path/to/jaildir/

e d0
proc tmp hd2 da0 e d1

em
m
var mnt em
km
dev
null

de
root

v
boot
rc.d
etc
/dev/null proc
lib kernel
sbin root
bin

e
use

m
ho
in r
use
usr
sb us
n
bi loc er

r al

host jail
n
i
sb

src ports
n
bi

us
er
jail(8) best practices

diagrams from A City is Not A Tree , essay by urban designer Christopher Alexander
and opportunities...
break out of jail?
Poul-Henning Kamp (PHK) wrote the
jail feature for R&D Associates http://
www.rndassociates.com/ who
contributed it to FreeBSD around 1998.

To my knowledge, nobody has broken

out of a jail directly, ever. It is however
assumed that nobody has tried that hard
yet, as it is still considered esoteric.

If someone breaks jail, PHK wrote that

he would love to know about it.
 best practices
ssh into jails to manage their
processes!!!!

You always can see the jailed

filesystem/userland from host
server, be careful.

Design your jailing system

carefully, be creative with core
UNIX utilities.

Use your highest secure

practices for host server...
 great utilities
4.x, jps, jkill, jtop
5.x, 6.x, onward builtin ps, kill
!plus jls(8), jexec(8) jattach
(2), sysctl features for jailing

Design your jailing system

carefully, be creative (note
about nullfs, devfs)

additionally, handy: pstree,

xtail, disk images via mdconfig
common weak points
lost jail?
[hostname lockdown]
resource attacks
disks full
[partitions, disk images]
fork bombs, memory hogs
[securelevels, login.conf]
process control
direct driver access
[flags to mount devfs, procfs]
common weak points
lost jail?
[hostname lockdown]
resource attacks
disks full
[partitions, disk images]
fork bombs, memory hogs
[securelevels, login.conf]
process control
direct driver access
[flags to mount devfs, procfs]
 Comments on Isolation

1 host:/path/to/jaildir/
da
ed0
d a0 ed 0
hd2 ed1 da0
tmp
tmp hd2 ed 1
em
var
m em
mnt em var
m
km mnt em
km
dev
dev
de

null

de
v

boot
rc.d

v
boot
etc rc.d
FreeBSD proc etc
/dev/null proc
lib kernel kernel
lib
sbin root
sbin root
bin bin
e

use
m

e
use

m
ho

in usr r

ho
sb in usr r
n us sb us
bi loc e r bi
n
loc er
al a l
n

in
i
sb

src ports
sb
jails src ports
n

n
bi

bi
 memory/process attacks

http://www.samag.com/documents/s=1151/sam0105d/0105d.htm
http://www.freebsd.org/doc/en_US.ISO8859-1/books/arch-handbook/jail.html

OpenRoot Project, fork-bombs, FreeBSD

SecureLevels/maxproc, reality, and process control
memory/process attacks
(check the Defcon 14 CD)
# hog.c, a small utility to hog system memory
# written by Brian Redman (BER) sometime around 1986
# Basic Instructions, Compile this code to a binary:

cc hog.c -o hog

# then run something like:

hog 10

# and the hog will do just that- sit and hog 10mb of ram.
# To run a hog stampede, (a fork bomb):

while (1)
hog 99m&
end
memory/process attacks
(check the Defcon 14 CD)
# STEP 1)
# jailed /etc/login.conf file, example of restricted values:
:maxproc=30:\
:memoryuse=25M:\

# STEP 2)
# Set immutable flags on jailed /etc/login.conf, example:
chflags schg $D/etc/login.conf

# STEP 3)
# Set a higher securelevel on a per-jail basis
# (5.x onward, 4.x jailing only securlevels for entire host)
# add the following line to the jailed /etc/sysctl.conf:

kern.securelevel=2
# securelevel 1 is minimum, read the man page for securelevel
 honeypot?
compile and give the jail a kernel, fix sysctl:

http://www.freebsd.org/cgi/query-pr.cgi?pr=95977
 disk resource control

Put at least your jailed systems on a seperate

partition, or perhaps each jail (rigid in practice)

File-Backed Disk Images (mdconfig, in

handbook)- insanely flexible, but take extra
memory (usually negligible)
file-backed disks (.dmg)

WOW, theyre convienent.

watch out for device numbering (or things get
lost), heres where Jailing strategies from 4.x
come in handy... unless someone has a better
way of manging device nodes
speed is getting excellent for file-backed
memorry disks, but will always introduce some
overhead in file I/O
 file-backed disks (.dmg)
FreeBSD handbook has tons more information!

# writing 1gb blank file, (analagous to creating an

unformatted harddrive)
dd if=/dev/zero of=1gb.img bs=1k count=1024k

# attaching the file (analagous to attaching a harddrive)...

mdconfig -a -t vnode -f 1gb.img -u 1101

# formating the disk...

disklabel -r -w md1101 auto

# detaching the disk (analagous to ejecting a harddrive)...

mdconfig -d -u 1101
 file-backed disks (.dmg)
mount disks when starting jails,
<snip - jail start script>

mdconfig -a -t vnode -f /path/to/jaildisk_file.dmg -u 200

mount /dev/md200c /path/to/jail_userland_mount_dir

# regarding '-u 200' above, it can be handy to use some

# variant of a jail's respective IP address for it's disk
# image devide node id, so it's easy to track down on host
# system with many jailed servers.

# later in script,
jail /path/to/jail_userland_mount_dir \
hostname.fqdn.com \
10.0.1.200 \
/bin/sh /etc/rc

</snip>
 automation

Tarball packaging is your friend.

clean, simple, reliable.
be aware of dev/proc mounts
be aware of symlinks
use FreeBSD Ports Mechanism!
(not for the ports collection, thats insanely
presumptuous, [borderline irresponsible]
CVS/SVN anyone?
upgrading jailed systems

Simply use buildworld, (FROM HOST SYSTEM),

toss buildworld DESTDIR flag, with a jails
userland path

follow the handbook: http://www.freebsd.org/doc/

en_US.ISO8859-1/books/handbook/
makeworld.html
/etc/sysctl.conf (host)
(check the Defcon 14 CD)
# $FreeBSD: src/etc/sysctl.conf,v 1.8 2003/03/13 18:43:50 mux Exp $
#
# This file is read when going to multi-user and its contents piped thru
# ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details.
#

# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0

# ikenote jailing additives

security.jail.set_hostname_allowed=0 # default = 1 # jailed resetting hostname.
security.jail.enforce_statfs=2 # default = 2 # mount point info.
security.jail.allow_raw_sockets=0 # default = 0 # for ping, etc...
security.jail.socket_unixiproute_only=1 # default = 1 # access to routing sockets.
security.jail.sysvipc_allowed=0 # default = 0 # SysV shareed mem? Ha!
security.jail.chflags_allowed=0 # default = 0 # root less than root...
sysctl (stock values)
(check the Defcon 14 CD)

$ sysctl -a | grep jail

security.jail.set_hostname_allowed: 0
security.jail.socket_unixiproute_only: 1
security.jail.sysvipc_allowed: 0
security.jail.enforce_statfs: 2
security.jail.allow_raw_sockets: 0
security.jail.chflags_allowed: 0
security.jail.jailed: 0
firewalls (quick comment)
context:
why jail in the first place again?
threats affect an entire host server
firewall at a higher level (mental shift to treat
the host like a network gateway!)

global system firewalling, throttling

different boxes? different rules?
Start Script w/ Disk Image
(check the Defcon 14 CD)
#!/bin/sh

# simple, complete script to start a jail.

# define the absolute path to the jail,

J=/usr/local/jails/jailed.userland.directory

# define the ip address for the jail,

I=10.0.1.192

# define a hostname,
H=fqdn.com

ifconfig en0 inet alias $I/32

mount -t procfs proc $J/proc

mount_devfs devfs $J/dev
## add additonal flags to mount_devfs, to hide unnecessary devices!!!
## check the man page for mount_devfs

jail $J $H $I /bin/sh /etc/rc

jail crontab misc...
(check the Defcon 14 CD)

# comment out the following, just to keep syslog quiet for irrelevant items.

# Save some entropy so that /dev/random can re-seed on boot.

# */11 * * * * operator /usr/libexec/save-entropy

# Adjust the time zone if the CMOS clock keeps local time, as opposed to
# UTC time. See adjkerntz(8) for details.
# 1,31 0-5 * * * root adjkerntz -a
future directions...

important fun:

CARP, from PF/OpenBSD

GEOM
NFS Improvements
more NAS/SAN support (GEOM, ggated)
FreeBSD 4.x, 5.x, 6.x, (7x!)
sick possibilities...
GEOM Gate, CARP, fun with failover jails...
net net

switch1 switch2

carp1 carp2 carp3 carp4

application server application server

application server application server

application server application server

application server application server

switchA switchB

NAS safe storage

 misc

Compile md(4) into the kernel for File-Backed

Disks, for better performance

GOTCHA: rm a jail directory? chflags -R noschg

jaildir
 Stillborn.
Suggestions?
 Special Thanks:
wintermute (of iMeme), taught me to jail(8).
Hes here somewhere- buy him a drink.

reality schooled me more BSD than he knows...

Poul-Henning Kamp wrote the jail feature for R&D

Associates http://www.rndassociates.com/ who
contributed it to FreeBSD around 1998.

Robert Watson wrote the extended documentation,

found a few bugs, added a few new features, and
cleaned up the userland jail environment.

ike is proud to be a part of the New York City *BSD Users Group,
and the Lower East Side Mac Unix Users Group
 [email protected]
ike is proud to be a part of the New York City *BSD Users Group,
and the Lower East Side Mac Unix Users Group