Mon, 09 Jun 2008

Everything Old Is New Again (Server Push & no-parse headers)

In 1995, the rage on the Internet was Netscape. They had just released Netscape 1.1, and it shipped with a new feature called "server-push". The guys at Netscape applied a concept from email (MIME type: multipart/mixed from mail attachments) and created multipart/mixed-replace. The idea was that sometimes you don't want people to download your web page, and have it remain on the screen-- sometimes you want the content to be refreshed, when and if the server chooses, and replace what was pushed a moment before.

I recall opening up Photoshop, creating several layers, and exporting GIFs that would get refreshed, and writing some CGI programs that would use this new mime type to display, what were in effect, server-driven animated GIFs. On the server side, you had to convince httpd (now apache) not to server-parse the response to the client... Since you, the programmer, had to decide what content to push to the client, having the server compute the mime type (is it a JPEG? A GIF? A HTML document?) or the content length (is this GIF larger or smaller than the one before it?) by parsing the content wouldn't work. So, a convention was born to precede any such scripts with "nph-", meaning, "let me tell the client what kind of file and how big it is..."

I just finished a project that incorporates this technology, but didn't realize that I'd be brushing up on this technique I first learned 13 years ago.

With a baby on the way, I wanted a baby-monitor I could check even when I was back at work. Most off-the-shelf baby monitors use limited range 900Mhz wireless spectrum with no Internet capabilities. Instead, I got a couple of Panasonic pan/tilt/zoom webcams, and to save $80/unit, I also got some Panasonic powerline adapters so I didn't have to buy the more expensive 802.11g versions. The adapters allow me to avoid inconvenient wiring, and keep the video IP traffic off my wireless network.

The cameras have a live video feature, which I didn't really think much about how it worked... It worked just fine in Safari and Firefox, and that's all that mattered. But live video is limited-- you can't see what activity has taken place throughout the day. Well it turns out the cameras have security features, allowing me to upload JPG still captures from the cameras when their motion detectors are tripped.

But the problem was, each time the sensor is tripped several dozen images would get uploaded to a remote directory, and I'd have to manually go through them one at a time.

It was neat, however, to be out and about, and to be able to pull up my iPhone and see who's been at my front door, or if my cats have been jumping into the crib...

The first incarnation of this project was simply an apache directory-index. Click the file, see the still, click back, click another file... Repeat ad nauseam.

So I thought, hmm, perhaps I can build a file manager and use some software to seam the images into an mpeg movie, or motion-jpeg. Group the images by hour of day, and provide a "delete" function so that I don't have thousands of images piling up...

Then the thought hit me to use server-push (by now you probably saw this coming), and as I did a little tcpdump (network packet sniffing) analysis, I realized that this is the exact same thing that the camera's firmware is doing for any browsers monitoring the "video" feed...

After a little web coding (1455 bytes to be exact-- I'll post it if anyone is interested), I wrote a simple script that lets me browse all the events that my camera has captured and uploaded to the Internet, view them in equivalent video quality as the camera's built-in server, and when I'm done, delete the batch of files representing that video.

But best of all, (the browser on the iPhone) can handle multipart/mixed-replace content!

Khan Klatt

Khan Klatt's photo