Facebook threat destroys giant profile database

Legal threats from Facebook suits have led to the destruction of a massive database created by a researcher, looking to innocently examine social connections between friends.

According to the New Scientist, legal pressure forced Pete Warden to destroy the records, even though identical information can be pulled from Google’s cache, with other commercial search engines doing exactly the same thing.

Using ‘legal’ crawler software, he harvested information from Facebook profile pages you could get without logging in. From that he could get user’s names, locations, friends and interests, but he was planning to remove names and make details anonymous.

It was the fact that Warden didn’t ask for permission that Facebook didn’t take kindly to, as others had compiled similar databases after notifying Facebook beforehand.

But Warden was still very unlucky to have to destroy the data, as Facebook carried a file called ‘robot.txt’ which doesn’t prohibit the use of crawler software to sift through the data.

Warden said on his blog: “As you can imagine I’m not very happy about this, especially since nobody ever alleged that my data gathering was outside the rules the web has operated by since crawlers existed.

“So why am I destroying the data? This area has never been litigated and I don’t have enough money to be a test case.”

Warden used to work at Apple, which meant that he understood Facebook’s stance as getting the lawyers in was a “tempting first option”. But he said that there was good news as he noticed that there was a real commitment on the behalf of researchers to safeguard the data.

On the side of Facebook, it does seem that recent bad press over privacy and the availability of profile data has made it more touchy than ever before.