The pause, Ubuntu and news

I’ve pretty much switched to Ubuntu on the laptop, which has been taking up most of my time. After a bit of tinkering almost everything works as expected, other than the Option 3G/EDGE wireless card that I’ve managed to get the OS to detect and even dial out with, but it is not functioning yet. Honestly, it is a bit of a downer; one that won’t allow me to get rid of the Windows XP install that is on the other partition as I travel quite a bit and I need to be connected all the time. And other than the Bombay and Bangalore airports, wifi is not free anywhere, which means the data card is my only savior.

There are other bits of news too that have been consuming more of my time. I’ve been reconsidering a lot of things, including the direction in which I am headed career-wise, things should be much clearer by the end of the week and hopefully there should be news that I can break by then.

Coming back to Ubuntu and Linux what I can say is that while the experience has improved vastly from the last time I had dabbled with being on the OS, it is far from being user friendly or idiot-proof. For the business user it is still a major pain to set things up straight (you should see the way how the Open Office UI is rendered, which is reminiscent of something from the 1980s than the 2000s).

After installing the fonts and the drivers (the Nvidia binary driver won’t allow me the max resolution of 1200 x 800 on my GeForce Go 6600 card, I had to fall back to the NV driver to get there), I had to wonder if any business user or even the average Joe would be able to put up with it.

Thankfully, most of my data is handled by online applications, other than the exception of my Firefox extensions (thank god for FBE) and settings and after an initial bit of getting used to each other, Evolution and I have gotten along famously well and that should allow me to say bye to the good old Outlook days.

Honestly, for Linux to get mass adoption someone has to do an Apple with it, which is apparently what Dell is trying to do. But with Dell they are just getting the hardware right, someone has to repackage Linux in a much more polished manner, along with the hardware, with good support. See, Ubuntu has 90% of the things right in it, but the 10% it does not have going for it is the important bit for which I can see most people paying for. Would it not be possible to charge 10% of Vista’s licensing costs to provide support and packaging for Linux on the desktop and still have a decent business going that could benefit everyone — Linux, companies and the users — involved?

A search inside Google’s heart

The New York Times does one of those rare pieces on Google’s searching ranking development center – Building 43 — today and as expected a zillion links are pointing at that article right now. Following are a couple of things that stood out for me.

Editorialisation of search results: For all our fear of Google servers slowly morphing into self-sustaining and hyper intelligent systems, the system still requires a fair bit of human intervention. Jason Calacanis and Mahalo just got another slide to add to their Powerpoint pitch.

Any of Google’s 10,000 employees can use its “Buganizer” system to report a search problem, and about 100 times a day they do — listing Mr. Singhal as the person responsible to squash them.

Variance in expected results: Not all searches are made equal, so are the complaints they receive about it. Sometimes you need a larger sample to validate an anomaly, making it an enough of an expected exception (for the user, that is) than an unexpected exception (which would be an errant result) for them to make changes to the core. That’s when you get the Deja Vu moment, “hey, this is nice and relevant, but it never behaves this way normally.”

But Mr. Singhal often doesn’t rush to fix everything he hears about, because each change can affect the rankings of many sites. “You can’t just react on the first complaint,” he says. “You let things simmer.”

How fast should fast be: Not enough, apparently. Users hit up Google faster than they are able to populate the index with fresh content. Two seconds is a nightmare of a metric to live with. Having known news operations pretty well that break news to earn their living, I can say that putting up even a dummy link or a flash on television or the internet takes close to two minutes. The again, if you see a huge query volume coming for a relatively uncharacteristic subject in that short a time, it should be a good indicator that something major is happening out there.

As an example, he points out what happens when cities suffer power failures. “When there is a blackout in New York, the first articles appear in 15 minutes; we get queries in two seconds,” he says.

Classify, rank and reclassify: There is no monolithic process that runs the show for Google. The final bland page you get to see is an aggregation of different classifiers, that are probably ranked again, running independently of each other. Think of it as the difference between wine and champagne. You get to drink a super champagne on the result page.

Classifiers can tell, for example, whether someone is searching for a product to buy, or for information about a place, a company or a person. Google recently developed a new classifier to identify names of people who aren’t famous. Another identifies brand names.

Pretty interesting reading to end the Sunday night. I’d ideally have liked to have written more, but I have a morning flight to catch and the road to woo for most of next week. Adios.

Update: Greg elaborates on my first point in a detailed and much better manner.

Technorati tags: google, NLP, classifiers, indexing, search, building 43

How I got into technology

Ethan Kaplan is one non A list blogger who has over time become a favourite of mine. Other than him being 28 and getting emotional in planes, he writes with a degree of lucidity and comfort that’s all too rare find these days. Today, he has posted an entry where he’s asking us to look back at where we come from:

Lets reverse that and see what technology did to us. And how we as people are defined by the machines at our disposal.

So here goes:

How I got into technology was mostly driven by necessity. I was not one of those gifted geeks who could magically figure out the difference between a transistor and a capacitor, understand circuit diagrams and make sense of how things work out. I grew up in a country where it was a huge thing to have a television set (that too black and white!) at your home in the 80s. As a kid, the exposure to technology started with the transistor, initially the huge brown valve-based ones that was later replaced by the solid state ones. This does not mean that I had a proclivity for opening them up. I used to leave them alone because I could not make much sense of them and I was more interested in what came out of it: the yearning to listen to something more than the run-of-the-mill state-run crap. In fact, I could say my dalliance with technology has always been led by the quest to learn and know more, but that was not necessarily tied to knowing more about the technology itself.

There were short-lived flirtations later, with DIY kits, rigging up small (and illegal!) FM stations, trying to cook up fancy antennas to first tap into the first generation (which most still are, even to this day) cable networks that we were not allowed to subscribe to at home and pick up any kind of weak RF signals and bounces on the radio. My first run in with a video gaming platform was when somebody gifted me an funny version (or a copy?) of the early Nintendo Gameboy. It had the classic game where you, as an ambulance guy, had to catch people jumping off buildings on fire at either ends of the screen. After living through many three lives and considerable thumb play, it broke down somewhere along the way, but that was all I needed to get hooked.

Unlike most fellow geeks, I did not grow up with computers and I have never seen a PDP. In fact, I had minimal interaction with computers through school and a fair bit of high school. It was more the VCR and corresponding technologies that used to keep me busy and at that age and time, VCRs were used almost exclusively for one thing: to watch porn. My first home computer was a Pentium I with 16 MB of RAM and a 4 Gigabyte hard disk drive that had Windows 95 pre-installed on it (yes, I started THAT late with computers). I had no prior experience in fiddling around with one and after changing considerable file type associations and even deleting a couple of DLLs or two, I at least had an idea of what not to do with the silly white box than what I should do with it.

It was around this time that the word ‘Internet’ began to worm its way into my conscience. I was then dabbling a lot in 3D animation (on the ancient 3D Studio for DOS, since I could not draw anything in 2D) and VRML was more hyped than Kentucky Fried Chicken in the country. Since India had terrible restrictions on internet use at that time (dial up and only shell access), I had no clue about what it was, other than this idea that it was a vast amount of information that I could dip into. That, coupled with the 3D ambitions, led me to believe that one day we could virtually walk into any library anywhere on earth and read whatever I wanted to if the internet dream were to come true for me. Everything I wanted to figure out was a gateway for more knowledge. It was just a means to an end and not an end in itself.

Somewhere along the way I had finished graduating in Economics and being the clueless sort I had no ambition of my own. The 3D animator career did not go anywhere since the big boys were playing with SGI systems and Maya and I could not bring myself to wring out that kind of money out of the parents. After getting a couple of robots to kill each other in two minute movies, the fascination pretty much came to an end. Time now was spent playing RoadRash and other assorted games, the names of which I can’t remember much about and there was still no internet on the horizon.

It was around that time that a friend gave me a CD which had something weird called ‘Linux’ on it. Since it was something new that I had no clue about, I had to know about it. Being one of those who don’t ever RTFM I’d struggled through an entire night, with rawrite and lilo for company, trying to get Redhat 5 installed on the poor Pentium I. Finally, with some help, I did mange to get it running along with X windows, but it had trashed the Windows 95 install and as you could imagine, my dad, who used the system to get real work done, was not mighty impressed by that.

Since the graduation was of no great help in getting a career, I decided to become one of the hordes who got into the ‘software engineer’ line of work. In a matter of 16 months I was supposed to have become someone who knew Core Java (Corba, anyone?), Oracle (Form Designer, anyone?), UML and many other bits of technology. Lofty course targets apart, the truth was that I was struggling with basic pointer logic and arrays (screw mutli-dimensional, even simple ones ended up giving me the creeps) during the 30 days for which that I attended the course. What it do for me was to give me access on a true multi-user system (an AIX box since the facility was affiliated to IBM), where I found the magical pleasures of ‘wall’ ‘talk’ and shell scripts.

Eventually, with good help from two buddies, I managed to write a simple program in C (no frame buffer crap, Iz loved mes ASCII text ) that in theory was intended to help fishermen with their catch and scripts to track when root was logged in and the commands he/she was using. Somehow, eventually, we managed to stage a DoS attack on the poor box that brought it down eventually. But the real breakthrough was that the facility had an ISDN line that we could hook up to every now and then. That was the first time in my life I actually saw a web page from a remote server in a web browser. It was nothing like the walk through library I used to dream about, but I don’t think that was the intended purpose of www.ftv.fr  at any point in its existence in any case.

As fate would have it, I had, quite by accident and by unrelated events in my personal life, ended up in Delhi doing my Post Graduate Diploma in Journalism from here. Through 9 months of the course, I had my first run in with the Mac OS and free-flowing bandwidth via a VSAT hook up, most of which was spent in trying to circumvent various access policies that were set in place by the administration. You cannot keep a man away from his ‘education’ for too long, can you? The fun and games ended when the time came to find gainful employment following the course and thankfully (considering how messed up the media industry was in India at that time), I got offers from a major print publication and a web solutions company that was also running an online magazine. And you could guess which way the ‘thirst’ took me.

The job was back breaking, but it also allowed me to trawl the internet at will and I could not really complain about that. What I did complain about was the content management system that the company used to have. It was awful to use. You could only add articles, there was just no delete or edit function. To edit an article, you had to manually delete the record from the MSSQL Enterprise Manager and enter the modified copy all over again. About the time I’d left that job, I had helped the guys design an edit form and in the meantime I also figured out the basic SQL syntax, since it was terrible to have to call one of the tech lads to just delete an article again.

The second job was with these guys who were running an even more archaic system that was based on Foxpro, FTP and untar. Right after my joining them, things happened in such a way that we had to make radical changes to the operations and figured it was not possible because nobody had the source code for the Foxpro programs. Thus started the move to a new CMS based on JSP and Oracle (the boom years, remember?) that just refused to scale up beyond 10 concurrent sessions. To show that such heavy lifting (and expensive software, since we were using JRun) was not required, I started replicating the system using PHP 3 and Apache on a Windows 98 machine with 16 MB RAM (Oh noes, not againz!). It was the first bit of code I’d written after a very long time (since the C days) and strangely, what I learnt at that time slowly started falling into place.

The next two years I picked up a fair bit of technology (the first run in with Postgresql), operations and product development. Which was followed by a stint here raging against the system, launching blogs and RSS feeds in the process after which I switched to where I am these days. I could go into finer details of what exactly do I do here, but that would quite not be about what technology did to me, so I’ll spare all of you that torture.

As I’ve repeatedly said, it has always been the urge to know more that has driven me to technology. Knowing it helps you do things better, it allows you to function more in sync with the technology team. The strange thing is that in pursuing all of this I’d started out being a journalist, moved from that into technology later and eventually ended up at the operations and business end of the deal. A major part of what I am today, I owe it to technology, without which I’d not have been able to access and amass the knowledge I have today.

Feel free to spread the meme.

Pirates of the Caribbean 3: Will this ever end?

Pirates of the Caribbean: At World’s End is basically the team behind the franchise on dope or something like it. They could comfortably have taken off an hour and a half of the running time from it and still not missed much in terms of completion and narration. The first hour and half is so awfully disjointed, confused and desperately looking for direction that most of us were left wondering where exactly were we going with this? They could have done this much much better and from what I have heard about Spiderman 3, it looks like this is going to be a summer of discontent for the ‘part’ movies.

When they took Jack’s life in the previous installment, the story writers also took the character’s soul with it. Sparrow in this episode is hollow and in a manner of speaking very Jim Carrey-esque while doing time in Davy Jones’s locker. It almost felt like they wanted to make up for the time they could not put him on screen during the first hour by splattering the screen with multiple Jacks when they finally could get him on it. Come on, the character’s charm is his singularity and uniqueness. And by showing three different sides of him in the same frame, they all but destroy it. Depp, as usual, does his bit very well and the movie has its regulation number of classic Sparrow quotes and moments, but those alone can’t carry a 2.5 hour mess all through to the end.

There are good performances splattered all over the place, notably from Geoffrey Rush, as usual, but in the end there is not much else to remember a movie that starts with people being hanged by the half-dozen and ends with some pseudo head given by Will to Elizabeth. And oh, the Keith Richards cameo was funny too, but why on earth did Chow Yun-Fat agree do to this to himself? If you are giving this one a miss, I can tell you that you are not missing much.

Technorati tags: pirates of the caribbean, at world’s end, jack sparrow, movies

Making out with Google Gears

Couple of days ago, Google took the wraps off Google Gears – a framework that brings persistence of data and state into the browser.

Gears consists of three modules:

1. LocalServer: The LocalServer provides a platform to copy and execute the live functionality within the local system. In a manner of speaking, it mimics the interaction that happens on a live webserver when a browser interacts with it. It also caches the images, JavaScript and other objects it may encounter on a Gears-enabled website.

   This is not a revolutionary approach. Companies like Zimbra have been doing this for a while now, but the difference is that Zimbra embeds the entire application within the server, while in the case of Gears the LocalServer is only a framework that allows you to build applications on top of it.
    

2. Database: This a simple SQLite 3 RDBMS that will store anything you want it to store.It provides a layer to access almost all of the SQLite’s features. It also includes support for full text searching with the Fts2 extension. And in true blue web programming style, it also leaves the door open for SQL injection attacks.
3. WorkerPool: These are the poor Orcs of Gears. All functionality triggered by the UI is passed on to the WorkerPool to execute in leisure. According to the documentation, “Scripts executing in the WorkerPool will not trigger the browser’s “unresponsive script” dialog.” None of the workers share any execution state and can at best be thought of as dumb crunchers, they just keep crunching away at whatever is thrown to them. As a result, there is no hierarchy in the system, because every worker is born equal (wow, the Googlers are commies!) it is up to the developer to do his design in such a way that if need be, he should place a traffic cop worker whose lone job is to track inter-worker messages.

First Blood

Anyway, the geekery apart, these are some of my initial impressions of Gears:

The installation is simple. Go to the website and download the installer. The product is available for the Windows, Mac and Linux platforms.

On Windows, the installer downloads the required files and lodges itself as an extension in Firefox and as a Browser Help Object (gears.dll) in Internet Explorer.

Following a restart, this is what you get in the browser options:

Add-ons options in Internet Explorer

 

Add-ons/Extensions tab in Firefox

Websites that are Gears-enabled will check if you have the framework installed and ask you for your permission to use Gears and its features.

The only major Gears-enabled product that is available now is Google Reader and what it allows you now is to download and store data and objects (images, files etc) locally, till you can sync it with the server again.

 

Once I had selected the offline mode, it downloaded 2000 items (which is a soft limit than actually downloading 2000 items that you may not have to download in the first place).

It took six minutes to download the items, which was not bad considering that I am on a lousy GPRS/EDGE connection that is barely usable these days. Your mileage may vary depending on the feeds you have subscribed to, whether they are full text or excerpts or if they have too many images in them and so on. And the data download happens only till the progress meter hits 50%, after which the syncing starts and no more data is downloaded. As a result you see 0% to 50% progress taking almost 90% of the time, while the 50% to 100% progress happens in a flash.

As expected, in the offline mode, new items load considerably faster since they are coming from a local database. The subsequent syncs also proceed much faster than the first one, which is only natural most times as it would contain much lesser data compared to the initial full sync. Interestingly, following the installation, all the reader URLs now have the ‘offsync=true’ flag attached to them. But I have to say that the framework is still very clunky and buggy. I often ended up with 0 unread items in offline mode and 100+ unread items in online mode with the same item set, as Items marked as read in offline was turning up as unread in online. Buggy!

Conclusion

So, what do I think about it? It is a promising concept, but it also requires a considerable bit of re-engineering of applications that can often be built far too easily for the web. Designing syncing and conflict resolution logic is not everyone’s morning cuppa and uptake would at least initially be slow for this one. As a technology, it is a bold gambit by Google. It is aiming to set the standards and in getting the first blow in and it is also trying to get developer muscle behind it (once the tide turns towards in terms of usage, the community and other companies will do the hard work for Google. It won’t have to keep racking its brains to release products all by itself).

I am still not convinced that Javascript and the extension model is the way ahead for a secure offline platform for Mozilla. Even though every application is to have its own sandbox sort of scenario, with the database and LocalServer locked down to conform to the Same Origin Policy, the WorkerPool seems agonistic to the application context in which it is being called and the Same Origin Policy is the first and the last line of defense. I am not too sure about this, but it does require further investigation. The rest of the security is provided by the platform (Firefox or IE), which again brings in its own set of problems. And since Gears is now being talked about as the offline storage framework for Firefox 3, this has to be thought out very well.

From a user’s perspective, this is quite a nifty addition. It would be very familiar for people who already use syncing in their daily life using existing software like Mail For Exchange. But at this point in time, it is a developer-oriented release, is clunky and anything but seamless to use. Then there is the major problem of being able to maintain state between two different computers. The Google engineers suggest a workaround for this by asking developers to “store offline preferences locally,” which is not really a workaround anyway. So there is plenty of work to follow in this space too.

Technorati tags: google gears, google reader, google, firefox, sqlite