Sensor Network Protocol

Design and Implementation

Philip Levis
UC Berkeley

Sensor Network Constraints

Distibuted, wireless networks with limited resources
Energy, energy, energy.

Communication is expensive.
Idle listening is the principal energy cost.
Radio hardware transition times can be important.
Low transmission rates can lower cost of idle listening.

Nodes cannot maintain a lot of state.

RAM is at a premium.

CS 268, Spring 2005

Constraints, continued
Uncontrolled environments that drive execution
Variation over time and space
The uncommon is common
Unforseen corner cases and aberrations

CS 268, Spring 2005

Sensor Network Behavior

CS 268, Spring 2005

Design Considerations
Uncontrolled environment: simplicity is critical.
The world will find your edge conditions for you.
Simplicity and fault tolerance can be more important
than raw performance.

Wireless channel: cheap broadcast primitive.

Protocols can take advantage of spatial redundancy.

Redundancy requires idempotency

But we have limited state.

CS 268, Spring 2005

A Spectrum of Protocol Classes

Dissemination: One to N
Collection Routing: N to One
Landmark Routing: N to changing one
Aggregation Routing: N to One
Any-to-Any

CS 268, Spring 2005

A Spectrum of Protocol Classes

Dissemination: One to N
Collection Routing: N to One
Landmark Routing: N to changing one
Aggregation Routing: N to One
Any-to-Any

CS 268, Spring 2005

A Spectrum of Protocol Classes

Dissemination: One to N
Collection Routing: N to One
Landmark Routing: N to changing one
Aggregation Routing: N to One
Any-to-Any

CS 268, Spring 2005

Dissemination
Fundamental networking protocol
Reconfiguration
Reprogramming
Management

Dissemination: reliably deliver a datum to every

node in a network.

CS 268, Spring 2005

To Every Node in a Network

Network membership is not static
Loss
Transient disconnection
Repopulation

Limited resources prevent storing complete

network population information
To ensure dissemination to every node, we must
periodically maintain that every node has the
data.
CS 268, Spring 2005

10

The Real Cost

Propagation is costly
Virtual programs (Mat, TinyDB): 20-400 bytes
Parameters, predicates: 8-20 bytes
To every node in a large, multihop network

But maintenance is more so

For example, one maintenance transmission every minute
Maintenance for 15 minutes costs more than 400B of data
For 8-20B of data, two minutes are more costly!

Maintaining that everyone has the data costs more than

propagating the data itself.
CS 268, Spring 2005

11

Three Needed Properties

Low maintenance overhead
Minimize communication when everyone is up to date

Rapid propagation
When new data appears, it should propagate quickly

Scalability
Protocol must operate in a wide range of densities
Cannot require a priori density information

CS 268, Spring 2005

12

Existing Algorithms Are

Insufficient
Epidemic algorithms
End to end, single destination communication, IP overlays

Probabilistic broadcasts
Discrete effort (terminate): does not handle disconnection

Scalable Reliable Multicast

Multicast over a wired network, latency-based suppression

SPIN (Heinzelman et al.)

Propagation protocol, does not address maintenance cost

CS 268, Spring 2005

13

Solution: Trickle

CS 268, Spring 2005

14

Solution: Trickle
Every once in a while, broadcast what data you
have, unless youve heard some other nodes
broadcast the same thing recently.

CS 268, Spring 2005

15

Solution: Trickle
Every once in a while, broadcast what data you
have, unless youve heard some other nodes
broadcast the same thing recently.
Behavior (simulation and deployment):
Maintenance: a few sends per hour
Propagation: less than a minute
Scalability: thousand-fold density changes

CS 268, Spring 2005

16

Solution: Trickle
Every once in a while, broadcast what data you
have, unless youve heard some other nodes
broadcast the same thing recently.
Behavior (simulation and deployment):
Maintenance: a few sends per hour
Propagation: less than a minute
Scalability: thousand-fold density changes

Instead of flooding a network, establish a trickle of

packets, just enough to stay up to date.
CS 268, Spring 2005

17

Trickle Assumptions
Broadcast medium
Concise, comparable metadata
Given A and B, know if one needs an update

Metadata exchange (maintenance) is the

significant cost

CS 268, Spring 2005

18

Detecting That a Node Needs

an Update
As long as each node communicates with others,
inconsistencies will be found
Either reception or transmission is sufficient
Define a desired detection latency,
Choose a redundancy constant k
k = (receptions + transmissions)
In an interval of length

Trickle keeps the rate as close to k/ as possible

CS 268, Spring 2005

19

Trickle Algorithm
Time interval of length
Redundancy constant k (e.g., 1, 2)
Maintain a counter c
Pick a time t from [0, ]
At time t, transmit metadata if c < k
Increment c when you hear identical metadata to your own
Transmit updates when you hear older metadata
At end of , pick a new t

CS 268, Spring 2005

20

Example Trickle Execution

k=1

c
A

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
21

Example Trickle Execution

k=1

c
A

tA1

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
22

Example Trickle Execution

k=1

c
A

tA1

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
23

Example Trickle Execution

k=1

c
A

tA1

tC1

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
24

Example Trickle Execution

k=1

c
A

tA1

tC1

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
25

Example Trickle Execution

k=1

c
A

tA1
tB1

tC1

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
26

Example Trickle Execution

k=1

c
A

tA1
tB1

tC1

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
27

Example Trickle Execution

k=1

c
A

tA1
tB1

tB2

tC1

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
28

Example Trickle Execution

k=1

c
A

tA1
tB1

tB2

tC1

tC2

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
29

Example Trickle Execution

k=1

c
A

tA1

tA2
tB1

tB2

tC1

tC2

time
transmission
CS 268, Spring 2005

suppressed transmission

reception
30

Experimental Methodology
High-level, algorithmic simulator
Single-hop network with a uniform loss rate

TOSSIM, simulates TinyOS implementations

Multi-hop networks with empirically derived loss rates

Real world deployment in an indoor setting

In experiments (unless said otherwise), k =1

CS 268, Spring 2005

31

Maintenance Evaluation
Start with idealized assumptions, relax each
Lossless cell
Perfect interval synchronization
Single hop network

Ideal: Lossless, synchronized single hop network

k transmissions per interval
First k nodes to transmit suppress all others
Communication rate is independent of density

First step: introducing loss

CS 268, Spring 2005

32

Loss

(algorithmic simulator)

Transmissions/Interval

12
10
8

60%
40%
20%
0%

6
4
2
0
1

16

32

64

128

256

Motes
CS 268, Spring 2005

33

Logarithmic Behavior of Loss

Transmission increase is due to the probability that
one node has not heard n transmissions
Example: 10% loss
1 in 10 nodes will not hear one transmission
1 in 100 nodes will not hear two transmissions
1 in 1000 nodes will not hear three, etc.

Fundamental bound to maintaining a per-node

communication rate

CS 268, Spring 2005

34

Synchronization
(algorithmic simulator)

14

Transmissions/Interval

12
10
8

Not Synchronized

Synchronized

4
2
0
1

16

32

63

128

256

Motes
CS 268, Spring 2005

35

Short Listen Effect

Lack of synchronization leads to the short listen
effect
For example, B transmits three times:

A
B
C
D
Time
CS 268, Spring 2005

36

Short Listen Effect Prevention

Add a listening period: t from [0.5, ]

Listen-only period

CS 268, Spring 2005

37

Effect of Listen Period

(algorithmic simulator)

14

Transmissions/Interval

12
10

Not Synchronized
Synchronized
Listening

8
6
4
2
0
1

16

32

63

128

256

Motes
CS 268, Spring 2005

38

Multihop Network
(TOSSIM)

(transmissions + receptions)
-k
intervals

Redundancy:

Nodes uniformly distributed in 50x50 area

Logarithmic scaling holds
Redundancy over Density in TOSSIM
4

Redundancy

3.5
3
2.5

No Collisions
Collisions

2
1.5
1
0.5
0

16

32

64

128

256

512

1024

Motes

CS 268, Spring 2005

39

Empirical Validation
(TOSSIM and deployment)
1-64 motes on a table, low transmit power

CS 268, Spring 2005

40

Maintenance Overview
Trickle maintains a per-node communication rate
Scales logarithmically with density, to meet the
per-node rate for the worst case node
Communication rate is really a number of
transmissions over space

CS 268, Spring 2005

41

Interval Size Tradeoff

Large interval
Lower transmission rate (lower maintenance cost)
Higher latency to discovery (slower propagation)

Small interval
Higher transmission rate (higher maintenace cost)
Lower latency to discovery (faster propagation)

Examples (k=1)
At = 10 seconds: 6 transmits/min, discovery of 5 sec/hop
At = 1 hour: 1 transmit/hour, discovery of 30 min/hop
CS 268, Spring 2005

42

Speeding Propagation
Adjust : l, h
When expires, double up to h
When you hear newer metadata, set to l
When you hear newer data, set to l
When you hear older metadata, send data

CS 268, Spring 2005

43

Simulated Propagation
New data (20 bytes)
at lower left corner

Time To Reprogram, Tau, 10 Foot Spacing

(seconds)

16 hop network
Time to reception in
seconds

18-20
16-18

Set l = 1 sec

14-16
12-14
10-12

Set h = 1 min

8-10
6-8
4-6
2-4
0-2

20s for 16 hops

Wave of activity
CS 268, Spring 2005

Time

44

Empirical Propagation
Deployed 19 nodes in office setting
Instrumented nodes for accurate installation times
40 test runs

CS 268, Spring 2005

45

Empirical Propagation
Deployed 19 nodes in office setting
Instrumented nodes for accurate installation times
40 test runs

CS 268, Spring 2005

46

Network Layout
(about 4 hops)

CS 268, Spring 2005

47

Empirical Results
k=1, l=1 second, h=20 minutes
Mote Distribution, h=20m, k=1
30%
25%
20%
15%
10%
5%
0%
0

10

15

20

25

30

35

40

45+

Time (seconds)

CS 268, Spring 2005

48

Network Layout
(about 4 hops)

CS 268, Spring 2005

49

Empirical Results
k=1, l=1 second, h=20 minutes
Mote Distribution, h=20m, k=1
30%
25%
20%
15%
10%
5%
0%
0

10

15

20

25

30

35

40

45+

Time (seconds)

Sparse networks can have a few stragglers.

CS 268, Spring 2005

50

Dissemination
Trickle scales logarithmically with density
Can obtain rapid propagation with low
maintenance
In example deployment, maintenance of a few
sends/hour, propagation of 30 seconds

Controls a transmission rate over space

Coupling between network and the physical world

Trickle is a nameless protocol

Uses wireless connectivity as an implicit naming scheme
No name management, neighbor lists
Stateless operation (well, eleven bytes)
CS 268, Spring 2005

51

A Spectrum of Protocol Classes

Dissemination: One to N
Collection Routing: N to One
Landmark Routing: N to changing one
Aggregation Routing: N to One
Any-to-Any

CS 268, Spring 2005

52

Aggregation Routing
Collect data aggregates from a network
How many nodes are there?
What is the mean temperature?
What is the median temperature?

Deliver aggregate to a central point (root)

Two examples: TinyDB and Synopsis Diffusion

CS 268, Spring 2005

53

TinyDB: Tiny Aggregation (TAG)

Proposed by Sam Madden et al.
Compute aggregates over a collection tree

CS 268, Spring 2005

54

Query Propagation Via TreeBased Routing

Tree-based routing

Used in:
Query delivery
Data collection

Topology selection is important;

e.g.
Krishnamachari, DEBS 2002,
Intanagonwiwat, ICDCS 2002,
Heidemann, SOSP 2001
LEACH/SPIN, Heinzelman et al.
MOBICOM 99
SIGMOD 2003

Q:SELECT

R:{}

R:{}

Q
Q

D
R:{}Q

R:{}Q

E
CS 268, Spring 2005

Continuous process
Mitigates failures

R:{}

Q
Q

F
Q
55

Basic Aggregation

In each epoch:

Each node samples local sensors once

Generates partial state record (PSR)

local readings
readings from children

Outputs PSR during assigned comm. interval

At end of epoch, PSR for whole network

output at root

New result on each successive epoch

Extras:

Predicate-based partitioning via GROUP BY

3
4
5

CS 268, Spring 2005

56

Illustration: Aggregation
SELECT COUNT(*)
FROM sensors

Interval 4

Sensor #
1
1

Interval #

Epoch

5
1

3
2
1
4

CS 268, Spring 2005

4
1
5

57

Illustration: Aggregation
SELECT COUNT(*)
FROM sensors

Interval 3

Sensor #
1
1

Interval #

4
3

5
1

2
1

CS 268, Spring 2005

58

Illustration: Aggregation
SELECT COUNT(*)
FROM sensors

Interval 2

Sensor #
1
1

Interval #

3
2

2
1

CS 268, Spring 2005

59

Illustration: Aggregation
SELECT COUNT(*)
FROM sensors

Sensor #

Interval 1

1
1

Interval #

2
1

1
5

CS 268, Spring 2005

60

Illustration: Aggregation
SELECT COUNT(*)
FROM sensors

Interval 4

Sensor #
1
1

Interval #

2
1

CS 268, Spring 2005

1
5

61

Interval Assignment: An Approach

4 intervals / epoch

SELECT
COUNT(*)

Interval # = Level

CSMA for collision

T
2 avoidance
Z
Z L
Z
Z
Z

Time intervals for

3 power conservation
Many variations
4 Time Sync
CS 268, Spring 2005

5 4

Level = 1
Z

Comm Interval
Z

2 1
L T

3
Z

5
Z

Epoch
2

L T

L T

L T

Pipelining: Increase
throughput by delaying
5
result arrival until a later epoch

62

Aggregation Framework
Support any aggregation function conforming to:
Aggn={finit, fmerge, fevaluate}
Finit {a0}

<a0>

Partial State Record (PSR)

Fmerge {<a1>,<a2>} <a12>

Fevaluate {<a1>}

aggregate value

Example: Average
AVGinit

{v}

<v,1>

AVGmerge {<S1, C1>, <S2, C2>}

< S 1 + S2 , C 1 + C 2 >

AVGevaluate{<S, C>}

S/C

CS 268, Spring 2005

63

Types of Aggregates
SQL supports MIN, MAX, SUM, COUNT, AVERAGE
Any function over a set can be computed via TAG
In network benefit for many operations
E.g. Standard deviation, top/bottom N, spatial
union/intersection, histograms, etc.
Compactness of PSR

CS 268, Spring 2005

64

TAG/TinyDB Observations
Complex: requires a collection tree as well as
pretty good time synchronization
Fragile: single lost result can greatly perturb result
In practice, really hard to get working.
Sonoma data yield < 50%
Intel TASK project (based on TinyDB) has had many
deployment troubles/setbacks (GDI 2004)

CS 268, Spring 2005

65

Another Take on Aggregation

Sketches proposed by two groups
BU: Jeffrey Considine, Feifei Li, George Kollios and John
Byers (ICDE 2004)
CMU: Suman Nath, Phillip B. Gibbons, Srinivasan Seshan,
and Zachary Anderson (SenSys 2004)

Based on work probabilistic counting algorithm

work by Flajolet and Martin (1985)

CS 268, Spring 2005

66

Basic Observation
Fragility comes from duplicate and order sensitivity
A PSR included twice will perturb the result
Computational model bound to communication model

Decoupling routing from computation is desirable

Compute aggregates over arbitrary network layers, can
take advantage of multipath propagation, etc.
Decoupling requires aggregates to be resistant to a
single result being included multiple times, ordering

CS 268, Spring 2005

67

Synopsis Diffusion
Order and duplicate insensitive (ODI) aggregates
Every node generates a sketch of its value
Aggregation combines sketches in an ODI way
E.g., take a boolean OR

Compute aggregate result from combined sketch

CS 268, Spring 2005

68

More Specifically
Three functions:
Generate initial sketch
(produce a bit field)
Merge sketches (ODI)
Compute aggregate
from complete sketch

3
4
5

CS 268, Spring 2005

69

Example
Three functions:
Generate initial sketch
(produce a bit field)
Merge sketches (ODI)
Compute aggregate
from complete sketch

3
4
5

CS 268, Spring 2005

70

Example
Three functions:
Generate initial sketch
(produce a bit field)
Merge sketches (ODI)
Compute aggregate
from complete sketch

3
4

4
5

CS 268, Spring 2005

71

Example
Three functions:
Generate initial sketch
(produce a bit field)
Merge sketches (ODI)
Compute aggregate
from complete sketch

123

2 4

3
45

4
5

CS 268, Spring 2005

345

72

Example
Three functions:
Generate initial sketch
(produce a bit field)
Merge sketches (ODI)
Compute aggregate
from complete sketch

12345

2 45

3
45

4
5

CS 268, Spring 2005

345

73

Example
Three functions:
Generate initial sketch
(produce a bit field)
Merge sketches (ODI)
Compute aggregate
from complete sketch

12345

2 45

3
45

4
5

CS 268, Spring 2005

345

74

Example ODI: Count

How many nodes are there in the network?
Generating initial sketch
Generate a random number
Sketch is the least significant 0 bit in the number
E.g.: 0110101011101011 -> 0000000000000100

Merging sketches: binary OR of two sketches

Resolving aggregate:
Final value is the least significant 0 in the sketch/1.546
0000000101111111 -> 0000000010000000/1.546
CS 268, Spring 2005

75

Count Example
Three functions:
Generate initial sketch
Merge sketches (ODI)
00001
Compute aggregate
from complete sketch

00100

2
00010

3
4
00001

CS 268, Spring 2005

00001

76

Count Example
Three functions:
Generate initial sketch
Merge sketches (ODI)
00011
Compute aggregate
from complete sketch

00101

2
00011

3
4
00001

CS 268, Spring 2005

00011

77

Count Example
Three functions:
Generate initial sketch
Merge sketches (ODI)
00011
Compute aggregate
from complete sketch

00111

2
00011

3
4
00001

CS 268, Spring 2005

00011

78

Count Example
Three functions:
Generate initial sketch
Merge sketches (ODI)
00011
Compute aggregate
from complete sketch

00111

2
00011

3
4
00001

CS 268, Spring 2005

00011

79

Count Example
Three functions:
Generate initial sketch
Merge sketches (ODI)
00011
Compute aggregate
from complete sketch

01000

2
00011

3
4
00001

CS 268, Spring 2005

00011

80

Count Example
Three functions:
Generate initial sketch
Merge sketches (ODI)
00011
Compute aggregate
from complete sketch
8/1.556 = 5.14

01000

2
00011

3
4
00001

CS 268, Spring 2005

00011

81

ODI Issues
Sketches are robust, but are inaccurate estimates.
Standard deviation of error is 1.13 bits

Many sketches are needed for an accurate result

Compute many (e.g., 12) in parallel
Generate a stream of sketches

CS 268, Spring 2005

82

A Stream of Sketches
ODI rings
Only merge
in lower rings

CS 268, Spring 2005

83

A Stream of Sketches
ODI rings
Only merge
in lower rings
Example: hop count

CS 268, Spring 2005

84

Arbitrary Network Layers

ODI aggregates can be computed with Trickle
Two trickles: ODI sequence number, ODI sketches
When you hear a new sequence number, generate a
new local sketch, reset both trickles
Delivers aggregate to every node in the network

ODI aggregates require a gradient

Gradient determines what values can be merged
Prevents a sketch persisting forever
Gradient through time (seq no,), or through space (hops)

CS 268, Spring 2005

85

TAG vs. ODI

TAG: computes exact value, bound to a specific
routing layer that is vulnerable to loss and requires
complex synchronization
If it works right once, you get the precise answer.
Really hard to get to work right.

ODI: computes estimate, decoupled from network

layer, multipath makes it more resistant to loss,
requires simple synchronization
Simple implementations can accurately compute
estimate, many estimates needed for a precise answer.
CS 268, Spring 2005

86

Implementation Experience
TAG: implemented in TinyDB system
Two months of work to get TinyDB to work in deployment.
Very low data yield, no-one has been able to get it to
work again (TASK project).

ODI: a count query is 30 lines of code

A few tricks: coarse time synchronization needed.
Hasnt been tested to the same degree as TAG.

CS 268, Spring 2005

87

Design Considerations
Uncontrolled environment: simplicity is critical.
The world will find your edge conditions for you.
Simplicity and fault tolerance can be more important
than raw performance.

Wireless channel: cheap broadcast primitive.

Protocols can take advantage of spatial redundancy.

Redundancy requires idempotency

But we have limited state.

CS 268, Spring 2005

88

Questions

CS 268, Spring 2005

89