Real time web: Making web real time with push technology

Annoying son and the angry father

A boy tells his father “Hey dad! I want to go the zoo right now!” Then his father takes him to the car and start driving to the zoo. The boy is so excited about the tour and eager to get to the zoo immediately. So, while they are travelling boy repeatedly asks his father “Are we there yet?” Then father replied “No”

Boy: Are we there yet?
Dad: No

Boy: Are we there yet?
Dad: No

Boy: Are we there yet?
Dad: No

Boy is too annoying and angry father tells to the boy “Will you stop asking the same question? Otherwise I’ll get back you to home!”

World Wide Web in the good old days

If you carefully read the above story, you can find a similarity between that story and the traditional web. This is exactly analogous to a case like RSS reader repeatedly polling a web server for new content. Annoying boy is RSS reader and the angry father is web server in that case.

This is the nature of the web from its beginning. Since web is based on the client-server architecture, every time client has to initiate the request and server has to respond. Client exactly doesn’t know when new content arrives from the server so that client has to poll the server periodically. That’s why your email client repeatedly polls the mail server for new email. That’s why your news reader repeatedly polls the news server for latest news.

This “Pull model” makes so many problems when it comes to the bandwidth consumption. Continuous polling makes higher traffic on the networks and it causes the server hardware overloaded. Because of these issues, several attempts have came up during the last few years.

What is real-time web?

“The Real-Time Web is a paradigm based on pushing information to users as soon as it’s available – instead of requiring that they or their software check a source periodically for updates. It can be enabled in many different ways and can require a different technical architecture. It’s being implemented in social networking, search, news and elsewhere – making those experiences more like Instant Messaging and facilitating unpredictable innovations. Early benefits include increased user engagement (“flow”) and decreased server loads, but these are early days. Real-time information delivery will likely become ubiquitous, a requirement for almost any website or service.” – Read Write Web, 2009

Actually speaking, social networks put the first steps of making web real time. If you are Facebook or Twitter fanatic, you may have experienced their real time news feeds and status updates. They are more likely to Instant Messaging. Once your friend tags you in a photograph, you will be immediately notified by Facebook. Whenever a new Tweet arrives, you will be notified by Twitter. Likewise social networks offer their users a real time experiences day by day.

How to make web real time?

As I mentioned earlier there are several attempts to make the traditional web real time. Some of them are very straightforward in their nature while some of them are technically challenging. Below I’m going to describe several approaches to make web real time.

1. Web hooks

The concept of a WebHook is simple. A WebHook is an HTTP callback: an HTTP POST that occurs when something happens; a simple event-notification via HTTP POST.

A web application implementing WebHooks will POST a message to a URL when certain things happen. When a web application enables users to register their own URLs, the users can then extend, customize, and integrate that application with their own custom extensions or even with other applications around the web. For the user, WebHooks are a way to receive valuable information when it happens, rather than continually polling for that data and receiving nothing valuable most of the time.

Read more about web hooks here

2. HTTP server push (HTTP Streaming)

HTTP streaming is yet another elegant way of getting content as soon as they are published. Here, web server sends or pushes data to the web browser which is opposite to the traditional client-server architecture.

In this mechanism, web server doesn’t terminate a connection after response data has been served to a client. The web server leaves the connection open such that if an event is received, it can immediately be sent to one or multiple clients. Otherwise the data would have to be queued until the client’s next request is received. Several web servers offer this functionality CGI (Common Gateway Interface)

Read more about HTTP server push here


XMPP is a protocol that can be used to send instant messages. Its underlying technology is XML stanzas. By using XMPP, server can push new content to its client including browsers, desktop applications and mobile devices as well.

Read more about XMPP here

4. Comet

Comet is a common term that describes a web application model in which a long-held HTTP request allows a web server to push data to a browser, without the browser explicitly requesting it. Comet has various methods to achieve this web model.

Hidden iFrames, Ajax with long polling and XMLHttpRequest are some of the methodologies that are adhere to this Comet web application model.

Read more about Comet here

5. pubsubhubbub

pubsubhubbub is a simple, open, server-to-server web-hook-based pubsub (publish/subscribe) protocol as an extension to Atom and RSS.

Currently this is the most rapidly adopting technology to push new content from server side to the client side. Parties (servers) speaking the Pubsubhubbub protocol can get near-instant notifications (via web hook callbacks) when a topic (feed URL) they’re interested in is updated.

This architecture composed of a publisher, subscriber and a hub. Publisher delegates the responsibility of distributing new content to the hub. Subscribers initially require subscribing to this hub, in order to get new content delivered. Whenever the new content is added, publisher notifies the hub about the changes so that hub “pushes” the content to its subscriber’s in real time.

Read more about pubsubhubbub here