PSUBSCRIBE

Subscribes to channels that match a given pattern

PUNSUBSCRIBE
Unsubscribes from channels that match a given pattern
With this knowledge, it is trivial to implement chat and notification systems, either for
end-users or to stream messages between logical parts of applications. Pub/sub can
even be used as a building block of a robust queueing system. Let’s look at our simple
implementation of an instant messaging chat system.
On the server side, Node and Socket.IO will take care of the network layer, and Redis
will act as a straightforward implementation of pub/sub that delivers messages between
clients. On the client side, we’ll use a hint of jQuery to process messages, and send data
to the server.

Discussion
For this recipe, we’ll assume that you have a recent installation of Node.js, as well as
npm in order to install the necessary node libraries to support the chat system
(Socket.IO and Redis). We’ll start by looking at how we install the necessary software
to build the chat solution, and then go through the code for the server and client sides
of the software.

Installing the necessary software

Let’s start off by installing the necessary node libraries using npm:

npm install socket.io

npm install redis

Implementing the server side code

On the server side, we’ll be running Redis and creating a Javascript file that we’ll run
with Node.js. This piece of code will take care of setting up a connection to Redis and
listening on a given port for connecting clients (either using websockets or flash—this
choice will be handled transparently by Socket.IO). Let’s go through our necessary
JavaScript code. Create a chat.js file containing the following code:

var http = require('http'),

io = require('socket.io'),
redis = require('redis'),
rc = redis.createClient();

These lines require the libraries we installed and create the variables we’ll use to access
Redis and Socket.IO. We’ll access Redis with the “redis” variable, and “io” will let us

Using Redis’s Pub/Sub Functionality to Create a Chat System | 27

www.it-ebooks.info
access all the sockets that are connected to our server (web clients, who visit our chat
page).
The next thing we must do in our code is to set up an HTTP server system on top of
which Socket.io will do its websocket magic. Here are the lines to do that:

server = http.createServer(function(req, res){

// we may want to redirect a client that hits this page
// to the chat URL instead
res.writeHead(200, {'Content-Type': 'text/html'});
res.end('<h1>Hello world</h1>');
});

// Set up our server to listen on 8000 and serve socket.io

server.listen(8000);
var socketio = io.listen(server);

If you have some experience with Node.js or Socket.IO, this code is pretty straightfor-
ward. What we’re basically doing is setting up an HTTP server, specifying how it will
reply to requests, making it listen on a port (in this case, we’re going to listen on port
8000), and attaching Socket.IO to it so that it can automatically serve the Socket.IO
JavaScript files and set up the websocket functionality.
Now we set up the small bits of Redis code to support our functionality. The Redis
client we set up with Node.js must subscribe to a specific chat channel, and deal with
messages on that channel when they arrive. So that’s what we do next:

// if the Redis server emits a connect event, it means we're ready to work,
// which in turn means we should subscribe to our channels. Which we will.
rc.on("connect", function() {
rc.subscribe("chat");
// we could subscribe to more channels here
});

// When we get a message in one of the channels we're subscribed to,

// we send it over to all connected clients.
rc.on("message", function (channel, message) {
console.log("Sending: " + message);
socketio.sockets.emit('message', message);
})

As you can see, our actual Redis code is extremely simple. All we do is listen for messages
on a specific channel, and when they arrive, we broadcast them to all clients that are
connected to us.

28 | Chapter 3: Leveraging Redis

www.it-ebooks.info
Implementing the client side code
With the server side part completed, all we have to do is create a small page that con-
nects to Node.js, sets up Socket.IO on the client side, and then deals with incoming
and outgoing messages. So let’s create a page like that now. Here’s the main trunk for
a very simple HTML5 page:

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Chat with Redis</title>
</head>
<body>
<ul id="messages">
<!-- chat messages go here -->
</ul>
</body>
</html>

Now we need to include the two main pieces we need to get the functionality working:
jQuery and Socket.IO. We’ll grab the first from Google’s CDN, and the second from
our Node.js server (Socket.IO takes care of setting this up for you automatically). Insert
these two lines in the head section of your page:

<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js"></script>
<script src="http://localhost:8000/socket.io/socket.io.js"></script>

We’re now ready to connect to Node.js from this page and start listening to and pro-
cessing messages. Add the following code to your head section:

<script>
var socket = io.connect('localhost', { port: 8000 });

socket.on('message', function(data){
var li = new Element('li').insert(data);
$('messages').insert({top: li});
}
</script>

This piece of JavaScript makes the client-side Socket.IO connect to our Node.js instance
on port 8000 and start listening to message events. When a message arrives, it creates
a new list element and adds it to the unordered list we had already added to our code.

Using Redis’s Pub/Sub Functionality to Create a Chat System | 29

www.it-ebooks.info
Remember, this is very simple code and the resulting chat page will look ugly by default,
but it is also trivial to update it to look better.
All we’re missing at this point is a form and a way to send messages from one client to
the server so that they can be broadcast to everyone else. This is done with Socket.IO’s
emit function, which we already used on the server side as well. Write something like:

<form id="chatform" action="">

<input id="chattext" type="text" value="" />
<input type="submit" value="Send" />
</form>

<script>
$('#chatform').submit(function() {
socket.emit('message', $('chattext').val());
$('chattext').val(""); // cleanup the field
return false;
});
</script>

When a client fills the form and clicks Send, jQuery will use our socket variable to emit
a message event to the server, which will then broadcast the message to everyone else.
The return false statement in the last script tag keeps the form from actually being
submitted. Our submission code is handled by Socket.IO.

Further improvements
In the previous sections, we built the main pieces of a chat system using Node.js,
Socket.IO, and Redis. There are many ways we could improve our code. Instead of
sending and receiving regular strings, we could create small JSON snippets that include,
with the message, a bit of metadata like a username or avatar. We could also improve
our server-side code to include multiple channels, or allow subscribing to several chan-
nels using a pattern. The possibilities are endless, and Redis’s pub/sub implementation
makes it trivial to implement robust solutions for chat or notifications.

Implementing an Inverted-Index Text Search with Redis

Problem
An inverted index is an index data structure that stores mappings of words (or other
content) to their locations in a file, document, database, etc. This is generally used to
implement full text search, but it requires previous indexing of the documents to be
searched.

30 | Chapter 3: Leveraging Redis

www.it-ebooks.info
In this recipe, we’ll use Redis as the storage backend for a full-text search implemen-
tation.

Solution
Our implementation will use one set per word, containing document IDs. In order to
allow fast searches, we’ll index all the documents beforehand. Search itself is performed
by splitting the query into words and intersecting the matching sets. This will return
the IDs of the documents containing all the words we search for.*

Discussion
Indexing
Let’s say we have a hundred documents or web pages that we want to allows searches
on. The first step is indexing these. In order to do so, we should split the text into its
separate words and perhaps exclude stop words and words under three characters in
length. We’ll use a Ruby script to do this:
def id_for_document(filename)
doc_id = $redis.hget("documents", filename)
if doc_id.nil?
doc_id = $redis.incr("next_document_id")
$redis.hset("documents", filename, doc_id)
$redis.hset("filenames", doc_id, filename)
end
doc_id
end

STOP_WORDS = ["the", "of", "to", "and", "a", "in", "is", "it", "you", "that"]
f = File.open(filename)
doc_id = id_for_document(filename)
f.each_line do |l|
l.strip.split(/ |,|\)|\(|\;|\./).each do |word|
continue if word.size <= 3 || STOP_WORDS.include?(word)
add_word(word, doc_id)
end
end

So, we’ve filtered the words that will be added to the index and generated unique IDs
for our documents. We still need the indexing function:
def add_word(word, doc_id)
$redis.sadd("word:#{word}", doc_id)
end

So, for each each word that we find in our documents, we have created a new set
containing the IDs of the documents where that word can be found.

* This recipe is based on an example by Salvatore Sanfilippo released under the BSD license.

Implementing an Inverted-Index Text Search with Redis | 31

www.it-ebooks.info