These days it’s fairly common to run a local environment for web development. Whether you’re running Apache, Mongrel, or the App Engine SDK, we’re all starting to see the benefits of having a production-like environment right there on your laptop so you can iteratively code and debug your app without deploying live, or even needing the Internet.

However, with the growing popularity of callbacks and webhooks, you can only really debug if your script is live and on the Internet. There are also other cases where you need to make what are normally private and/or local web servers public, such as various kinds of testing or quick public demos. Demos are a surprisingly common case, especially for multi-user systems (“Man, I wish I could have you join this chat room app I’m working on, but it’s only running on my laptop”).

The solution is obvious, right? SSH remote forwarding, or reverse tunneling. Use a magical set of options with SSH with a public server you have SSH access to, and set up a tunnel from that machine to your local machine. When people connect to a port on your public machine, it gets forwarded to a local port on your machine, looking as if that port was on a public IP.

The idea is great, but it’s a hassle to set up. You need to make sure sshd is set up properly in order to make a public tunnel on the remote machine, or you need to set up two tunnels, one from your machine to a private port on the remote machine, and then another on the remote machine from a public port to the private port (that forwards to your machine).

In short, it’s too much of a hassle to consider it a quick and easy option. Here is the quick and easy option:

$ localtunnel 8080

And you’re done! With localtunnel, it’s so simple to set this up, it’s almost fun to do. What’s more is that the publicly accessible URL has a nice hostname and uses port 80, no matter what port its on locally. And it tells you what this URL is when you start localtunnel:

$ localtunnel 8080
Port 8080 is now publicly accessible from http://8bv2.localtunnel.com

What’s going on behind the scenes is a web server component running on localtunnel.com. It serves two purposes: a virtual host reverse proxy to the port forward, and a tunnel register API (try going to http://open.localtunnel.com). This simple API allocates a port to tunnel on, and gives the localtunnel client command the information it needs to set up an SSH tunnel for you. The localtunnel command just wraps an SSH library and does this register call.

Of course, there’s also the authentication part. As a free, public service, we don’t want to just give everybody SSH access to this machine (as it may seem). The user localtunnel on that box is made just for this service. It has no shell. It only has a home directory with an authorized_keys file. We require you to upload a public key for authentication, and we also mark that key with options that say you can only do port forwarding. Although, it can’t be used for arbitrary port forwarding… because it’s only a private port on the remote side, it can only be used with the special reverse proxy.

So there it is. And the code is on GitHub. You might notice the server is in Python and the client in Ruby. Why? It just made sense. Python has Twisted, which I like for server stuff. And Ruby is great for command line scripts, and has a nice SSH library. In the end, it doesn’t matter what it’s written in. Ultimately it’s a Unix program.

Enjoy!

I’ve long argued that failure is the only way a person can learn. This has been a very reasonating bit of wisdom for me, despite the fact people have often argued against it with rather sound logic. “Yes, you can learn from failure, but you can also learn from success!” Only after reading Jason Fried’s blog post from nearly one year ago to the day titled, Learning from failure is overrated have I realized exactly what’s wrong. We’re talking about different kinds of failure.

Learning happens when you correct the mismatch of an expected outcome from the actual outcome. This is where the idea of learning from failure actually comes from. Failure represents a mistake in judgment, a disparity between expectation and reality. Therefore, if your expectation is validated by success and there is no disparity, then you didn’t actually learn anything — you already knew.

However, validation of an expected but unsure success is obviously learning because it corrects the expectation of doubt you had. Furthermore, analysis of an unexpected success can result in learning from correcting the assumptions that led to expecting failure. A successful outcome, but a failure to expect it.

This is where the confusion comes in. Unfortunately it’s an issue of semantics. The meaning of the word failure in the context of “learning from failure” is this failure to know the outcome. In this way, it’s true: you can only learn from failure. However, it is only this specific instance of expectation-failure this applies to so absolutely.

Separately although related (which I think adds to the confusion), despite outcome-failures not necessarily teaching you what will work, they tend to be the strongest lessons experienced. I mostly attribute this to the greater level of disparity and correction made to your mental model that comes from expecting a failing outcome to succeed.

In any case, you learn from experience. There is no way to acquire knowledge without experience. In fact, that’s an entirely different argument. You can only acquire knowledge through experience. Not reading, not getting advice. But I’ll save that one for later.

In October 2009 I started a project called Notify.io and a month later announced it. I talked about how it will bring notifications to the web. Now that it’s basically alpha complete, I’ll give you a quick walkthrough of what makes it so great.

Overview
At a really high level, you can think of Notify.io as a notification router. As a web service, it provides a singleton endpoint for any web-connected program, whether a web application, desktop application or user script, to send notifications to somebody. For users, you can control what notifications you get and how you get them. In this way, Notify.io is like a global, web-accessable version of the popular Growl application for OS X (which should honestly just ship with OS X). Only it’s even better.

Desktop Notifications
The original inspiration for Notify.io was to make Growl more useful by fixing its ability to receive notifications from the Internet. Out of the box, Growl is effectively only good for notifications from sources running on your machine. If you wanted to get notifications from a web app, you’d have to wait for them to release a desktop notifier, which hopefully would use Growl to actually display the notifications. So you end up with all these desktop notifiers running for some apps, and have no option of desktop notifications for others.

This is probably the killer feature of Notify.io: it lets you get desktop notifications from any web app that supports it, which is an order of magnatude easier for them to do than build their own desktop notifier.

Sources and Outlets
The language of Notify.io is based around Sources and Outlets. Sources are pretty straightforward. They’re a source of notifications. They could represent an application, script, company, person (or perhaps object?) that can send you notifications.

Outlets repesent the other major feature of Notify.io. They’re ways you can get a notification. The Desktop Notifier is your first and default outlet, but is just one of several options. Currently supported Outlets besides Desktop Notifier are Email, Jabber IM, and Webhooks. Outlets to look forward to are SMS, Twitter, IRC, and perhaps telephone.

The magic is in routing notifications from Sources to Outlets. Currently this is a simple mapping of Source to Outlet. For example, you can get notifications from Source A on your desktop, while notifications from Source B go to IM. This simplistic routing is just the beginning. We’ll talk about how we’ll do advanced routing when we get to the Roadmap.

The Nio Client
For developers, it’s worth mentioning that the pipe for our Desktop Notifier is really just a Comet HTTP stream. It can be consumed by pretty much anything. We were originally talking with Growl and authors of other desktop notifiers of direct integration. This is still a possibility, but just so we could move forward, we built our own client for OS X. Clients for other systems are available (but not yet “officially” supported) or are in progress, including Windows and Android.

Our OS X client is called Nio, short for Notify.io, so you can pronounce it N-I-O, but I tend to pronounce it “neo”. It’s basically just an application that sits in your menu bar listening to HTTP streams (yes, plural) for notifications and pipes them into Growl.

For ease of installing streams, we made it handle files of the extension ListenURL. Once Nio is installed, you can download a ListenURL file containing a URL and it gets installed by Nio. The URL we give you is basically a “capability URL” or secret URL. This means streams are not super secure, but this is by design. If you wanted, you could share your URL with somebody so you both get notifications sent to that Outlet. You can always delete the Outlet and make another to disable that URL.

The other cool thing about our client is that it has a shell script notification hook. This means you can have notifications trigger a shell script that’s passed the notification details. This is pretty powerful because it means you can do things like create your own local logging, hear your notifications with text-to-speech, or make certain notifications trigger a more obstrusive means of notifying you, such as Quicksilver’s Large Type feature. This kind of programmability is central to our approach to design, as you’ll see later on in the Roadmap.

Simple API and Approval Model
For proper adoption, we need web apps to integrate Notify.io, so we have a super simple API for Sources. It’s a simple REST API based on an endpoint constructed by the target of your notification. Like Gravatar, we use an MD5 hash of a user’s email address to identify targets. For example, to send a notification to test@example.com, you’d do an HTTP POST to this URL:

http://api.notify.io/v1/notify/55502f40dc8b7c769880b10874abc9d0

You’d pass a few parameters, with at least your API key (meaning you need an account) and the text you want to send, and optionally an icon URL, link URL, title text and whether the notification should be “sticky”. That’s it. The request should respond immediately so it may be quick enough to be done inline in your app, but we recommend it be done asynchronously.

Then what happens is the first notification you send actually triggers a notification to that user that you want to send them notifications. If they accept, future notifications will be sent and your previous notifications will show up in their history. This may change to replay previous notifications on approval, but the point here is the user has to approve notifications before they get them. In this way, it’s similar to Jabber’s approval model and helps avoid spammers.

Public Service Software
Notify.io and its clients are open source. The service is free. Or rather, it’s not-for-profit donationware. Notify.io is being run under a model I’m developing called POSS, the goal of which is to automate/abstract away the maintainence and funding of its operation. The end result should be: the service exists, it’s open source, and some in the developer community can deploy changes. But no single person is financially responsible for it, and it’s run on maintained cloud infrastructure. In this case it’s mostly App Engine.

This means that Notify.io is not a startup. It’s public infrastructure. Ideally, I’m not even in the loop. It should be a self-sustaining public service. This is not fully realized, but it will be as it starts to consume more resources. For more information, you can read more on POSS or join our discussion group.

For now, the important thing is that Notify.io is open source. This means anybody can contribute bug fixes, new outlets, new desktop clients, etc.

Roadmap
Okay, sure, Notify.io is pretty cool now. But here are some of the major things that will be coming soon. Hopefully with your help!

Advanced Routing and Filters
From the beginning, I wanted really powerful routing and filtering. My evangelism of webhooks has given me the obvious answer to this, but in a more integrated way. Basically, how do you allow any routing scheme imagineable by users? Let them write code. Originally it was going to be powered by Scriptlets, but since I split the eval engine out as DrEval, it will be based on that.

Basically, just a imagine a UI with a little textarea for writing JavaScript that can make web calls. Route notifications based on your IM status, your location, what music you’re listening to, arbitrary time schedules, or anything you can code.

More Outlets
Obviously, more Outlets are good. Obvious ones are IRC, SMS, and Twitter DM. With Twilio we can do voice call notifications. Integration with push clients like the iPhone’s Prowl app would be easy to do. Our outlet system is very simple, so you can look at the source of our existing ones, write an outlet and it’s likely we’ll deploy it.

OpenID Support
Right now, you authenticate with Google. I don’t believe in creating authentication systems, and Google was the quickest given the platform. It’s also pretty popular and ensures you have an email address we can use. However, there are plenty of people that don’t like the idea of using their Google Account, so at some point we’ll support OpenID login and then go from there.

Multiple Email Support
Ideally, a web app can use whatever email address you used to register with them to send you notifications. However, unless Gmail is your primary email you use for registration, they’ll still need to ask you for your email. It’s the Gravatar model. So like Gravatar, we’ll need to let you add multiple emails to your account, allowing web applications to be able to send notifications based on any of them.

Convenience Libraries
Our API is simple, but people are lazy. We’re currently working on convenience libraries for popular langauges that it make that much easier to integrate with Notify.io. If you use a neat language, you should make a libnio package for it!

Ad-hoc Sources
Sources require an account, which is a bit heavyweight. Sometimes you want to create your own distinct sources to share with others or use in your scripts to easily send yourself notifications. This is the idea of Ad-hoc Sources, inspired by David Reid and capability URLs. The idea is simple: create an ad-hoc source and you get a secret URL. This URL acts just like the notify API endpoint, only you don’t need an API key. You can use this in public scripts or give it to others to send you notifications, and if it’s ever abused or falls in the wrong hands, just delete it and make another.

More Supported Clients
A developer in Japan started a Windows client based on Nio that we’re planning to support as our primary Windows client. Another developer is working on an Android client. iPhone users have Prowl, so once there is a Prowl outlet, you can get them on your iPhone. But Prowl is not free, so perhaps it would be helpful if we had our own iPhone client. There are the beginnings of a Linux/libnotify client. These are all ways you can start contributing to Notify.io. ;)

That’s about it. You can probably see why I describe Notify.io as the open notification platform of the web. It’s simple, powerful, and open source. It’s come a long way in just 3 months thanks to the contributions of Abimanyu Raja, Amanda Wixted, Mike Lundy, David Reid, Christopher Lobay, Hunter Gillane, Nakamatsu Shinji, and everybody that’s given user feedback so far. I recently made a quick screencast for the homepage that I’ll end with so you can see it in action.

Webhooks. User-defined callbacks on the web. Yes. YES! Wait, what does that even mean? That stupid website doesn’t seem to explain them at all!

Let’s explore this through an analogy. This analogy uses stockbrokers! Yay! (?)

Consider, before the web, before email, how people would interact with their stockbroker: the telephone. Whenever you wanted to manage your investments and find out what’s really going on in the market, you would call your stockbroker. It’s similar to how you might use your browser to visit Twitter to update your status and find out what’s going on with your friends, right? This is the basis of the analogy. Bear with me.

Imagine in this story your phone is a browser, a phone call is a web request, and the broker is a web application, like Twitter.

It’s 1989 and you like to think of yourself as a big time stock trader with an impressive portfolio. Good for you. You got to this point by staying in close touch with your stockbroker. You’d call her (!) to find out what was happening in the market, discuss whether buying certain stock was a good idea, and maybe put in an order. You were able to get information out of the system and put orders into the system by picking up your phone and calling your broker.

The problem is that you can only react to events in the market as quickly as you get the information. This means you’d have to call your broker quite a bit to stay on top of a fast moving, highly volatile market. A programmer might try to automate this with a script. You take a similar approach: you hire an assistant to deal with your stockbroker. Perhaps this is unheard of in reality, but stay with me.

Now an assistant is not unlike a computer program in theory. You can give them instructions and based on certain conditions and input, they’ll do as you say. For example, maybe you instruct your assistant to buy a certain type of stock after it behaves a certain way, and if there’s not enough money to buy the amount that you told them, they can transfer the money from your savings account after a quick confirmation phone call with you. In today’s world, that sort of thing is not terribly out of the question to automate with a script using the web APIs and programmatic infrastructure available to us now.

Again, in order to react to events in the market in a timely manner, it requires lots of phone calls to your stockbroker. Sure, your assistant can handle it, but it’s a lot of work and wastes a lot of time. In fact, it wastes you money because you pay for your assistant by the hour! If you were a programmer, your script would have to constantly poll the broker API requiring a touchy cron setup or a long running process that gets more inefficient the closer to real-time you want it to be. Not to mention it’s just more work than you should have to deal with. If only there was some way for your assistant or script to be notified when things happen so they could simply react.

One day, your stockbroker says they’re providing a new service they describe as callbacks. You give them a phone number, and they’ll call it whenever something interesting happens regarding stocks relevant to you. Wow! Well, you could give them your number, but there are two problems. First, it’s 1989 and you don’t happen to have a cell phone. You can’t always be reached. Second, even if you could, you’d still want your assistant to handle the events because they have been instructed with how to deal with them and will do a lot of the legwork for you. Luckily, your assistant has a cell phone and is always available, so you give the broker their number.

Now your assistant is working very efficiently making you lots of money. Their job is so much easier because they don’t have to do anything until they get a call from either you or, more importantly, the broker. They can finally react to events as they happen, without a lot of nonsense trying to stay on top of things. What’s more is they will automatically take care of whatever situations you’ve told them.

What a cool feature, this callbacks thing. Thanks, broker!

If you haven’t guessed, the callback feature in this story is the exact mechanics of webhooks. Webhooks would allow you to tell the web apps that you use to “callback” scripts you have online. These scripts will deal with whatever events that web app produces. These callbacks use the same protocol you use to talk to them and that you both know how to use: web requests. Putting your script online at a URL is analogous to your assistant that has a cell phone: it’s always available for direct connection at a persistent “phone number,” unlike you, with dynamic IPs and NATs and turning off your computer. And just like an assistant, the script can do a lot of interesting things for you that you wouldn’t necessarily want to do, most of which is a sort glue work of making different systems work together given some logic. For example, transferring money from your savings account to the broker account when there isn’t enough.

A concrete example of a story made possible from webhooks that might be a useful scenario for many of you involves Twitter. Let’s say Twitter supported webhook callbacks for when somebody follows you. Right now you get an email, and from there you can decide what to do manually: follow them back, block them, or do nothing. I used to go out of my way to block users that I knew were spam bots, but now there’s so many it’s not worth the time. And of course I also generally follow back people that I actually know. If Twitter would simply call a script of mine whenever somebody followed me passing along the user ID, I could very easily run this logic in a PHP script or a simple App Engine app. Or perhaps I’d use Scriptlets (ahem, which was made exactly for these kinds of web scripts). It would work like this:

First, use the Twitter API to look up the user from the ID, and grab their name. Then use the Facebook API to check if that name shows up in my list of friends on Facebook. If so, use the Twitter API to follow them back. Otherwise, if they’re following over 1000 users and that number is more than twice the number that’s following them (which is roughly the heuristic I use manually), use the Twitter API to block them. All automatic.

If you could do this, not only would you have glued together Facebook and Twitter in an interesting and useful way, but you’ve sort of extended the net functionality Twitter provides you. You could imagine perhaps someday Twitter releasing features that would do exactly what you just did. But they won’t. In fact, they’ll never release features that are so specific to you (who says my spam algorithm is the algorithm everybody should use?). This is how webhooks make the web more about you. You can start extending web applications and gluing them together just the way you want.

This is a win-win for web application users and developers. Users get more functionality. Developers can implement less.

Coming back to the stockbroker analogy, there is a type of order called a limit order where you say to buy or sell when it’s at a certain price. With webhooks (or the broker “callback” service), this is merely a convenience because you could easily set that up outside of their system. Speaking of that example, when I mentioned your assistant transferring money between accounts, that reminds me, you know that feature banks have called overdraft protection? If banks had webhooks, all banks could have overdraft protection. Wow, right? That would have been nice for me when I had Washington Mutual, a bank known for its lack of overdraft protection.

What else could you do in a world with webhooks? Basically everything would have a common event-driven infrastructure, allowing you, with just a little bit of scripting glue, to accomplish so much more and make the systems you use better, and more personal.

In my post about stdicon.com, I hinted at something else I built in collaboration with a few people that started from a Twitter update. I’m just now writing about it, but I’ve actually built a lot of things I haven’t blogged about yet. Granted, they’re all an artifact of my strange world, but I figure if I need them more than once, there’s a decent chance somebody else on this planet will need them someday. Anyway…

I present to you: TimeAPI.org

It started from wanting Chronic as a Service—a natural language datetime parser written in Ruby. You know, “in five hours” or “noon next tuesday.” Sure, there are good JS libraries, but actually, it’s a backend thing for me (webhooks). Unfortunately, I don’t use or have Ruby available all the time, so as part of my effort to make lots of tiny useful web services, I decided to make it a web service. Or rather, I decided somebody should.

Half an hour after I tweeted it, somebody had a prototype deployed. Then I worked with a friend to functionally polish it up. I’ve been meaning to use it for a simple Tweet-later service, but my first use of it was in Remindify, which was fairly recent (and is something I still need to blog about). In the process, I fixed a few bugs regarding timezones and became thoroughly frustrated with datetime programming. Haven’t we all.

However, the last thing I added was support for format strings. Why? Well, it crossed my mind before, since it would help in environments that have a hard time parsing ISO 8601 formatted datetimes (GLARES FURIOUSLY AT PYTHON). What finally made me implement it was so that I could use TimeAPI with the date command as a sort of cheap ntpdate replacement. Yes, this works:

date -s "$(curl "http://www.timeapi.org/utc/now?\a \b \d \I:\M:\S \Z \Y")"

For convenience, you could use this shortened URL:

date -s "$(curl -L http://j.mp/now-utc)"

Anyway, that’s kind of neat. Obviously, it’s more useful for parsing natural language. It’s a bit dumb when it doesn’t understand your natural language queries (throws up 500), but if you know how to use it and constrain it properly in your app, you’ll be fine.

Somebody else suggested adding the reverse functionality, sort of a time_ago as a service. Not a priority for me, but if anybody wants to add it, this is a completely open source service and I do deploy patches.

Two weeks ago it was being taped for a story on Fox News. One month ago it received a congratulatory visit from the mayor of Mountain View. Two months ago it was featured on the front page of the Mercury News. And four months ago was when we signed the lease for it. Hacker Dojo is officially on its way to something big.

Hacker Dojo, aspiring to be a global hub of innovation, is based on a simple idea: to provide a community center for hackers, thinkers and technologists to meet, discuss, learn and create.

It’s a non-profit, volunteer-run, member-supported operation that simply provides a space for hackers to do their thing 24/7, whether it’s working on side projects, organizing a meetup, building the next startup, giving a class, working remotely, or just hanging out with diverse yet like-minded people. The thing they have in common is the hacker spirit, a force that drives ordinary but curious people to create what can end up being extraordinary, such as the personal computer, the first video game system, or the technology powering the Internet.

A dojo is considered a place to do and train one’s craft with others. It’s most commonly used in the context of Japanese martial arts or other physical training. But the word “dojo” is simply defined as “place of the way.” In the case of Hacker Dojo, that way is the way of the hacker.

The hacker is not what most people think. Although 99% of the 100+ members of Hacker Dojo are capable of doing the things most people think “hackers” do, I’m 100% positive none of us have the intention of doing those things maliciously if at all. In fact, security has almost nothing to do with the way of the hacker. We define a hacker something like this:

A hacker is expert in their field, whether hobby or professional, that pushes the envelope of what’s possible through hands-on exploration, driven by relentless curiosity and a desire to challenge the status quo.

Steven Levy, author of Hackers: Heroes of the Computer Revolution, describes computer hackers as people that “regard computing as the most important thing in the world.” It’s about passion. It’s about something most of us can’t even describe. It inexplicably compels us to explore technology. To build things. To learn things. To be like the heroes in our field and achieve the remarkable, which is something I think everybody can relate to.

In fact, there is no reason why anybody shouldn’t aspire to be like a hacker, an innovator. This is what Hacker Dojo is about. We want to foster hacker culture so it can grow, develop and spread. We want to make its values explicit, and be the “place of the way” of the hacker.

If you still don’t understand the hacker, or want to learn more, take a look at the Hacker ethic article on Wikipedia. There’s also an excellent, short documentary available to watch online that shows you the energy and excitement of true hackers called Hackers: Wizards of the Electronic Age.

If you want to get involved in Hacker Dojo, come stop by or visit the website! We hold many events at the Dojo, but it’s also generally open for anybody to come and hack. If you’re too far to come to us, you may find another hackerspace nearby. For example, in SF there is the excellent Noisebridge hackerspace.

When you’re this obsessed with technology, so much that most “normal” people have no idea what you’re talking about, it helps to have a place where people don’t think you’re crazy. People have moved to Mountain View to be closer to Hacker Dojo. If you already live nearby, you should definitely take advantage of this place of the way of the hacker.

I’ve been playing a lot with Comet lately. It started with Notify.io, in which I decided to prove that HTTP streaming was a simpler alternative to XMPP in getting messages to the desktop. That went quite well, but it was easy because it wasn’t all that different from a socket connection. Then I built a yet-to-be-announced site that uses real-time updates, and I was forced to deal with Comet to the browser. That’s a bit more complicated.

Actually, I’m arguably a veteran of Comet in the browser. I was doing it before it had a name, all the way back in 2005. A friend and I were using it (without knowing that it was terribly novel) to build a real-time strategy game in the browser called AjaxWar. I haven’t really done a lot with it since, but I was hoping after almost 5 years there would be all kinds of advances in libraries and tricks that would make it super easy.

That was not really the case.

There are things, but not easy things. The Bayeux protocol? I guess all that would be easy to do if there were a lightweight Javascript library for it. But there isn’t really. There’s a jQuery one, but it’s completely undocumented. Plus I was just sitting there thinking, do I need all this? Handshaking? Message envelopes?

I was also hoping for actual persistent connections (that’s what we did in AjaxWar), but it turns out the standard today is long-polling. This is a semi-persistent connection that drops after every message and then reconnects and waits for the next message. I also wanted JSONP and cross-domain support. So I ended up using the dynamic script tag technique:

<script type="text/javascript">
	$(document).ready(function(){ waitForMsg(); });

	function waitForMsg() {
		$('body').append('\<script type="text/javascript" src="http://mycometserver.com/channel?callback=gotMsg">\<\/script>');
	}

	function gotMsg(msg) {
		// Do something with it
		waitForMsg();
	}
</script>

It’s fairly elegant in its simplicity and cross-browser support. But it has some weird side-effects that I’m not sure if any of the fancier systems got around. For one, it keeps the browser loading. While it’s waiting for messages, the browser says that page is still loading. I also ran into some issues where if I included, say, a Google Calendar widget on the page, it might not decide to keep the connection open (or even start it). I ended up putting a delay on the first call to waitForMsg() until after the calendar widget was likely loaded.

So it’s a bit brittle. You don’t know if it stops working. Therefore you can’t do retries. And you never know if you happen to miss a message between connections (unlikely, but something to worry about). Plus I think if you hit Escape it also kills it.

But this worked well enough for my projects. I knew that if I found a better way, I’d switch over to it, but it was good enough.

…heh…

Then today I decided to solve the problem right once and for all with a project called CometCatchr.

CometCatchr a lightweight Flash component to be used by Javascript that gives you a persistent connection for Comet streams.

I know a very small number of people won’t agree with my approach using Flash, but I tend to be pragmatic. I trust Flash is generally available and the benefits completely outweigh everything else to me. It’s also not new. Even the Bayeux protocol includes Flash as supported connection type. Still, I couldn’t find any simple Flash component that gave me what I wanted.

CometCatchr gives me Javascript callbacks on messages, maintains a persistent connection across messages, retries on lost connections, works in all browsers, supports (participating) cross-domain message sources, and just freaking works.

It was a drop-in replacement to my previous technique that worked right out of the box:

<script type="text/javascript">
	function gotMsg(msg) {
		// Do something with it
	}
</script>
<embed type="application/x-shockwave-flash" width="0" height="0" src="CometCatchr.swf?url=http://mycometserver.com/channel&callback=gotMsg"></embed>

It simplified my code, not just client-side, but now I don’t have to support JSONP callbacks on the server. CometCatchr also parses the JSON messages before passing the callback, so that’s taken care of too. I realize it seems less than ideal to couple this with JSON payloads, but I didn’t say it wasn’t an opinionated component.

In fact, it’s very opinionated. It likes single-line JSON messages sent via HTTP using chunked transfer encoding. That’s because that’s how I do Comet streams. Actually, that’s how Twitter does them, too. I’m quite alright with such constraints, but if you want to make changes, it’s MIT licensed and super simple to hack on.

So for the time being, I’ve more or less solved simple Comet to the browser, particularly for me. It also might be worth knowing that another motivation for building this component is that I intend to solve Comet and real-time stuff in the browser entirely… but uhh, yeah. Stay tuned.