How do you recover data from a formatted hard disk? Enter Data Rescue from PROSoft Engineering. I used Data Rescue II (only to realise afterwards that a new version was available). It took more than 2 days to scan the hard disk (1 TB hard disk, sector by sector analysis) and afterwards it recreated the files it found.

Recreation of the files, was also a lengthy process, and took nearly 10 hrs. After it recovers the files it presents a list of the kinds of files you might be interested in restoring. I selected the files I was interested in, research papers and my iPhoto Collection.

Recovery of the selected files (around 55.6GB) took around 3 hrs.

All in all I'm really grateful for such fantastic software. For all the windows users out there.... My NTFS hard disk has failed as well (bad start to the year! ). Currently I'm using ParetoLogic's Data Recovery Pro. The data recovery process is currently on going (since 5 days).

The chapter explores the challenges Facebook faced in analysing the data it is collecting and how existing RDMS solutions (MySQL and Oracle) were not up to the task of collecting and enabling analysis of highly fluid data such as clickstreams from millions of users (Currently 2.5 Petabytes is stored and new data is collected at 14 TB/day). The author goes on to discuss the solution they developed internally at Facebook (based on Cloud technologies such as Hadoop and unstructured data).

Analysis of large scale data is becoming a common problem in a large number of domains. Web companies such as Facebook, Google are not the only ones in the World that analyse huge amounts of data. Several scientific experiments such as the CERN LHC produce gigantic amounts of data that needs to be analysed (The recent book Fourth Paradigm by Microsoft Research explores data intensive scientific initiatives).

So many new skills are required to manage this data: designing storage architectures, high speed retrieval architectures, authoring data analysis workflows and finally communicating the results of the analysis. All these tasks are multi-disciplinary. Some tasks are related to Computer Science (design of data storage and retrieval systems), some to Business Analysis (authoring data analysis), some tasks belong to statisticians (the actual algorithms performing the analysis) and some to engineers (the underlying infrastructure for storing and processing the data).

Can this multi-discplinary approach to data management be termed as "Data Science". This is a term which I believe is increasingly gaining traction.

I don't want to experience such loss again, so I'm moving towards Cloud based backup. I gain a few things, but loose some as well. The service I selected is Mozy. They provide unlimited storage at an economical rate. However their backup/restore tool for the Mac does not support Proxies (their windows one apparently does). Moreover, Mozy's tool also does not allow me to browse my backups in a fine-grained fashion as Time Machine does. In Time Machine I can restore individual folders and files and browse my backup history over weeks and months. The Mozy tool does not provide such fine-grained history browsing.

Finally, uploading is such a hassle! It took me more than a day to upload a limited subset of data from my laptop (~60GB). Downloading fortunately is faster.

What do I gain from a cloud based backup solution? Hopefully I will not loose my data again.

However because there are certain advantages to local backups as well, I plan to do daily Time machine backups, on a new 1TB HD and weekly cloud backups. As for my pictures, I have a MobileMe subscription and those albums I shared there with friends and family I still have them. So in future I plan to upload all my new pictures to MobileMe.

Apple also provides upgrade options for iDisk and MobileMe Mail, however their prices are significantly more expensive an extra 60GB from Apple will set you back for an additional £59 (~ $100), while Google for 80GB charges only $20!.

So you might think that Apple's iDisk is atleast accessible from everywhere, while Google Storage can't be accessed as a drive? Think again, this is how:

For Mac Users:

Install gDisk. gDisk provides an accessible application for mounting your gmail drive on your mac.

For Linux Users:

The most popular way for Linux users to mount Gmail storage locally is to use FUSE. If your on ubuntu, this is how to do it

sudo apt-get install gmailfs
cp /etc/gmailfs/gmailfs.conf ~/.gmailfs.conf
chmod 600 ~/.gmailfs.conf

Now specify your gmail account credentials in ~/.gmailfs.conf and to finally mount the drive, just type:

mkdir gmail
mount.gmailfs none gmail

For Windows Users:

Just like for the Mac there are applications available like gDrive Shell Extension

1. First install the WebDAV Linux File System (davfs2). If you are using Ubuntu (as many people seem to be doing nowadays), you can install davfs2 by typing.

~# sudo apt-get install davfs2

2. To mount a dav share then, simply create a folder for it

sudo mkdir /mnt/idisk

and mount your idisk

sudo mount -t davfs https://idisk.me.com/$MOBILEMEUSERNAME /mnt/idisk/

Replace $MOBILEMEUSERNAME with your Mobile Me username. Davfs2 will ask for your password, and then you can start using your idisk.

"“Working harder” is a poor strategy which your competitors can trivially replicate. Instead, spend some time measuring what tasks add value to your business and at what imputed wages. Outsource those tasks which are below your desired imputed wage, automate any task where appropriate, and simply don’t do things which don’t add value."

We often yearn for more productivity and the answer usually is "work harder". But working harder is not the answer! Working harder implies that you spend more time on doing what you already do on a frequent basis. We all work on several things at once, and not all activities add real value. For instance if I analyse my day, I do in a sense try to "work hard", that is work at least 8-9 hours but I don't get much done in that time frame because all the various activities I do blur my real targets.

Faced with this predicament its always a good idea to sit back and analyse your day to day activities. You need to judge which truly add value to pursuits you deem valuable rather than waste your time under the illusion of getting work done. These are two ideas that have truly made me more productive and one idea that can:

1. Cluster related tasks

I go to the Gym regularly, but often if I have some groceries to purchase I purchase them while going to or form the gym. Similarly, if I go to the University I try to accomplish most of the tasks which I cannot do at home. On the contrary, if I did not cluster tasks doing some task again in the future might cost me more time than doing it along with some related task.

This idea works in unison with "contextual todos". When you define a todo item, define the context in which it can be carried out. Some tasks may be carried out at home, some while at the computer etc..

2. Automate trivial tasks

I try to automate as much as possible. Checking mails repeatedly when new ones arrive costs a lot of time in terms of lost focus and diverted attention. I've written an AppleScript that notifies me if a new mail has come that specifically mentions me in the TO or CC fields. If I don't get this notification I just don't check my mail.

3. Outsource as much as possible

I'm not in the position of outsourcing some of my tasks, but this can potentially be very useful for someone. Often while doing some task, I say to myself someone could be doing this and I could be working on something a lot more valuable. Indeed the author of the cited article outsourced numerous tasks that did not require his involvement.

Instapaper is a great app. I bought the paid version a few months ago. Every time I find some interesting article or blog post I save it to "Read it later" in Safari so that I can read it when its convenient for me. There are primarily three things I particularly like about the app.

1. Instapaper does not simply bookmark a page for you and show it to you later. It rather saves the page and formats it in a manner that makes it very suitable for reading in an iPhone. This means that the page is convenient to read (As you can judge form the screenshots below) and you can read those articles/blog posts offline.

2. Another amazing feature of the app is "tilt-scrolling". Click on the tilt-scrolling button in a the toolbar and if you tilt the iphone to the back, the page automatically scrolls down, tilt it forwards the back scrolls back up. The more you tilt the iphone the faster the page scrolls.

3. I can save pages directly to my Instapaper account from everywhere. From Firefox, from the iPhone Safari and Mac Safari, all saved pages are accessible and can be organised from the Instapaper.com site.

This is an app I'm glad to have purchased! Its worth every penny. You can get it from here.

I'm installing and deploying an experimental Eucalyptus Cloud. Eucalyptus is an open source framework for creating private cloud systems. I have been involved in Grid computing research for a number years now. Unfortunately there isn't a single Grid middleware that can be easily deployed by common users. Grids of course compared to contemporary cloud systems are more advanced in that they allow the aggregation of multi-site resources, while Eucalyptus aggregates resources in a single site (a local cluster).

I'm mostly interested at the usability aspect, how easy it is to deploy a cloud. I believe the reason why Grid has suddenly become unpopular is because Grid Middleware are not easy to use and deploy.

I will be blogging about my experiences in a series of posts. I will also make the VMs I create publicly available so that you guys can download them and play with a Eucalyptus Cloud.