Featured IT News

KEYNOTE VIDEO: Samsung launches new S23 range to take on Apple and Google's best, with 200 megapixel camera on S23 Ultra

02 February 2023 in Home Tech

Samsung has two major flagship launches each year - its Galaxy S series, and its Galaxy Z series of folding…

Fujifilm Australia extends focus to cutting edge AI

02 February 2023 in Data

Imaging products and services company, Fujifilm Australia, is doubling down on AI with its latest move into the healthcare industry.…

SAP announces new regional strategic services partner initiative

01 February 2023 in Strategy

SAP Asia Pacifiic has announced a Regional Strategic Services Partner (RSSP) initiative to “amplify the capabilities and reach of established…

CSIRO researchers use machine learning to advance Alzheimer's research

01 February 2023 in Health

Scientists from CSIRO, Australia’s national science agency, and Queensland University of Technology, have partnered to use artificial intelligence to develop…

Pandemic saw productivity levels at Australian and New Zealand-based financial organisations improve but 'gains set to erode': forecast

31 January 2023 in Market

Operational productivity at financial services institutions has steadily improved since the pandemic, highlighting the need for Australian and New Zealand-based…

Skills gap among ICT talent expected to ‘intensify’ over year ahead: NetApp

31 January 2023 in Enterprise Staff

While the Federal Government and the local tech sector have a shared commitment to achieve 1.2 million tech jobs in…

Aussie Broadband delivers 400Gbps first in Telstra Wholesale wavelength services

31 January 2023 in Telecoms & NBN

Telstra Wholesale and Aussie Broadband have successfully trialled the use of live 400GE client services on two diverse links between…

US thinking of further tightening export curbs on Huawei: report

31 January 2023 in Technology Regulation

The US is thinking of cutting off Chinese telecommunications equipment vendor Huawei Technologies from accessing any American components, including those…

IIT Madras firm develops Indian mobile operating system

30 January 2023 in Development

An Indian Institute of Technology Madras (IIT Madras)-incubated firm has developed a local mobile operating system which it claims will…

Frustration reigns as Aussies spend nearly 100 million hours on hold

30 January 2023 in Business Telecommunications

New research by ServiceNow reveals that Australian consumers spent a staggering 96.5 million hours trying to resolve customer service issues…

Dr Michele Allan AO appointed new chair of SmartSat CRC

30 January 2023 in People Moves

Australian space research centre SmartSat CRC has appointed current board member Dr Michele Allan AO as its new Chair, following…

Parent firm of Indian defence contractor hit by Windows Alphv ransomware

30 January 2023 in Security

The parent company of a private defence contractor in India has been compromised by the Windows Alphv ransomware (aka BlackCat),…

Accenture's Angela Coble recognised for pioneering efforts in gender diversity and technology leadership with Order of Australia medal

28 January 2023 in Enterprise Staff

Angela Coble, the Managing Director of Technology, Transformation & Executive Advisory for Accenture Australia and New Zealand, has been recognised…

Katonic.ai picks Nik B as ANZ head of sales

27 January 2023 in People Moves

Sydney-based Katonic.ai has appointed Nikhil "Nik B" Bhardwaj to the role of ANZ head of sales.

Media sweep targets influencers: ACCC

27 January 2023 in Technology Regulation

The Australian Competition and Consumer Commission (ACCC) has this week started a sweep to identify misleading testimonials and endorsements by…

Breaking IT News

Poly releases guide to hybrid work etiquette - the manners and style for the new age

02 February 2023 in CIO Trends

Audiovisual and hybrid work solutions provider, Poly, has partnered with etiquette authority Debrett's to create a modern guide for the…

Porter Novelli helps company crisis management go next-level with half day data breach simulation

02 February 2023 in Data

Social marketing and brand reputation management firm, Porter Novelli, has developed a half-day data breach simulation to aid businesses in…

VIDEO INTERVIEW: Cybermindz's Peter Coroneos and iRest.org's Stephanie Lopez talk burnout prevention and recovery in Gala Dinner Keynote Sneak Peek

01 February 2023 in Security

With the Cybermindz.org Fundraising Gala Dinner on at the end of February, with Guest of Honour Minister Victor Dominello to…

Vodafone Oman and Ericsson 5G SA test to improve mobile calls

01 February 2023 in Business Telecoms

Telecommunications operators Vodafone Oman and Ericsson have conducted a test call introducing 5G standalone voice and data services for Vodafone…

BT and Stratospheric Platforms expand mobile coverage in new trial

01 February 2023 in Business Telecoms

UK-based telecommunications operator BT and aviation and aerospace component manufacturing company Stratospheric Platforms are delivering a mobile coverage trial using…

BeyondTrust updates and bundles Privileged Remote Access and Password Safe

01 February 2023 in Security

New releases of identity and access security specialist BeyondTrust's Privileged Remote Access and Password Safe have expanded capabilities for developers,…

Coviu appoints Jenny Aiken as Director of Coviu Global in Australia

01 February 2023 in People Moves

Virtual care engagement platform Coviu has appointed Jenny Aiken as Director of Coviu Global in Australia.

Q-Ctrl extends Series B funding round

01 February 2023 in Deals

Quantum technology company Q-Ctrl's Series B-1 fundraising round is bringing in US$27.4 million.

Technics launches new super audio CD player

01 February 2023 in Home Tech

Japanese audio equipment brand Technics has unveiled the SL-G700M2, a successor to the SL-G700 network/super audio CD player.

4 Benefits of implementing an HCM solution into your business

01 February 2023 in Accounting Software

GUEST OPINION: As a business owner, it's essential to constantly look for ways to improve your operations and increase productivity.…

Zoho adds team pipelines and more to Bigin

01 February 2023 in Business Software

The new version of Zoho's Bigin CRM solution for small businesses features team pipelines, a mechanism that allows multiple customer-facing…

Orange Belgium, Telenet sign 15-year fibre agreement

01 February 2023 in Business Telecoms

Telecom provider Orange Belgium and rival Telenet have signed acommercial 15-year wholesale agreement, which will provide access to the other…

 

Reviews

 

Opinion

 

Telecommunications

 

Popular News

Subscribe to Newsletter

*  Enter the security code shown:

WEBINARS ONLINE & ON-DEMAND

GUEST ARTICLES

VENDOR NEWS

Guest Opinion

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News

Comments

×

Warning

JUser: :_load: Unable to load user with ID: 3658
Wednesday, 07 April 2010 02:26

Pete Warden vs. Facebook: a case of too much data access

By

Pete Warden had a really great idea: to map the friendship interactions of Facebook users to aid with geospatial analysis of user relationships.  Facebook's lawyers had a different view.

The entire saga is laid out as irregular snippets on Pete Warden's blog but to assist readers, we will extract the salient details, mostly in Pete's own words.  Our analysis is at the end of this report.

A couple of years ago, Pete Warden created a start-up company called Mailana (short for Mail Analysis).  "My goal has been to use the data sitting around in all our inboxes to help us in our day-to-day lives." 

Some time later, the focus changed.

"I'd already applied the same technology to Twitter to produce graphs showing who people talked to, and how their friends were clustered into groups. I set out to build that into a fully-fledged service, analyzing people's Twitter, Facebook and webmail communications to understand and help maintain their social networks.

"It offered features like identifying your inner circle so you could read a stream of just their updates, reminding you when you were falling out of touch with people you'd previously talked to a lot, and giving you information about people you'd just met."

On March 17th, Warden announced that due to the threat of legal action from Facebook, he had destroyed all accumulated Facebook-derived data linked to the analysis of social networks.


His summary of events continues: "As you can imagine I'm not very happy about this, especially since nobody ever alleged that my data gathering was outside the rules the web has operated by since crawlers existed. I followed their robots.txt directions, and was even helped by microformatting in the public profile pages. Literally hundreds of commercial search engines have followed the same path and have the same data. You can even pull identical information from Google's cache if you don't want to hit Facebook's servers. So why am I destroying the data? This area has never been litigated and I don't have enough money to be a test case."

Despite the ill-will over the entire incident, Warden does express some sympathy for Facebook noting that he was about to release data that they had no idea was available.

"I know from my time at Apple that reaching for the lawyers is a tempting first option when there's a nasty surprise like that. If I had to do it all over again, I'd try harder not to catch them off-guard."

Despite this setback, he observes that it isn't particularly difficult for anyone to obtain data from Google and he is already working on a similar analysis on publicly-available data linked to Google Buzz.

Getting back to the history of the saga: "I noticed Facebook were offering some other interesting information too, like which pages people were fans of and links to a few of their friends. I was curious what sort of patterns would emerge if I analyzed these relationships, so as a side project I set up fanpageanalytics.com to allow people to explore the data. I was getting more people asking about the data I was using, so before that went live I emailed Dave Morin at Facebook to give him a heads-up and check it was all kosher. We'd chatted a little previously, but I didn't get a reply, and he left the company a month later so my email probably got lost in the chaos."

Pete next describes the hell that broke loose upon him.


Once he had around six months' data, he started to do some simple analysis - the initial publication of this was met with crickets and tumbleweeds (as he described the amazing amount of interest shown!).

Next, in a fit of boredom he took to the data with some colouring tools to highlight some of the simple patterns he'd noticed.

Within 2 days, his blog post had received something like 200,000 hits via YCombinator and Reddit. 

Next his mobile phone rang with Facebook's attorney on the other end.

"He was with the head of their security team, who I knew slightly because I'd reported several security holes to Facebook over the years. The attorney said that they were just about to sue me into oblivion, but in light of my previous good relationship with their security team, they'd give me one chance to stop the process.

"They asked and received a verbal assurance from me that I wouldn't publish the data, and sent me on a letter to sign confirming that. Their contention was robots.txt had no legal force and they could sue anyone for accessing their site even if they scrupulously obeyed the instructions it contained. The only legal way to access any web site with a crawler was to obtain prior written permission
."


"Obviously this isn't the way the web has worked for the last 16 years since robots.txt was introduced, but my lawyer advised me that it had never been tested in court, and the legal costs alone of being a test case would bankrupt me. With that in mind, I spent the next few weeks negotiating a final agreement with their attorney.

"They were quite accommodating on the details, such as allowing my blog post to remain up, and initially I was hopeful that they were interested in a supervised release of the data set with privacy safeguards. Unfortunately it became clear towards the end that they wanted the whole set destroyed. That meant I had to persuade the other startups I'd shared samples with to remove their copies, but finally in mid-March I was able to sign the final agreement."

"I'm just glad that the whole process is over. I'm bummed that Facebook are taking a legal position that would cripple the web if it was adopted (how many people would Google need to hire to write letters to every single website they crawled?), and a bit frustrated that people don't understand that the data I was planning to release is already in the hands of lots of commercial marketing firms, but mostly I'm just looking forward to leaving the massive distraction of a legal threat behind and getting on with building my startup.

I really appreciate everyone's support, stay tuned for my next project!"

An analysis of these events is on the final page.


This has set a dangerous (albeit privately agreed) precedent; one which is diametrically opposed to the judgement handed down in Google vs. Field where the judgement specifically referenced the robots.txt file stating that the plaintiff (Blake Field) was fully aware of the proper use of the robots.txt file and yet chose not to make use of it to bar his copyrighted works from being crawled and cached by Google. 

Field's legal action against Google was regarded rather poorly by the judge: "Field decided to manufacture a claim for copyright infringement against Google in the hopes of making-money from Google's standard practice."  The judgement also noted that "author granted operator implied licence to display 'cached' links to web pages containing his copyrighted works."

Although appearing to support Facebook's claims of doing all they can to protect the privacy of their subscribers, the action against Warden seems aimed more at protecting a valuable resource and revenue-stream; exactly the same motivator the company has for constantly fiddling with the privacy dashboards in the hope users won't notice the changes that make more and more data available.

Facebook should also check with the courts to ensure they haven't misunderstood the current legal standing of the robots.txt file!

 

 

Read 5156 times

Please join our community here and become a VIP.

Subscribe to ITWIRE UPDATE Newsletter here
JOIN our iTWireTV our YouTube Community here
BACK TO LATEST NEWS here




ENABLE HYBRID CLOUD & REDUCE NETWORK LATENCY WHITEPAPER

Hybrid cloud promises to bring together the best of both worlds enabling businesses to combine the scalability and cost-effectiveness of the cloud with the performance and control that you can get from your on-premise infrastructure.

Reducing WAN latency is one of the biggest issues with hybrid cloud performance. Taking advantage of compression and data deduplication can reduce your network latency.

Research firm, Markets and Markets, predicted that the hybrid cloud market size is expected to grow from US$38.27 billion in 2017 to US$97.64 billion by 2023.

Colocation facilities provide many of the benefits of having your servers in the cloud while still maintaining physical control of your systems.

Cloud adjacency provided by colocation facilities can enable you to leverage their low latency high bandwidth connections to the cloud as well as providing a solid connection back to your on-premises corporate network.


Download this white paper to find out what you need to know about enabling the hybrid cloud in your organisation.

DOWNLOAD NOW!

PROMOTE YOUR WEBINAR ON ITWIRE

It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site itwire.com and prominent Newsletter promotion https://itwire.com/itwire-update.html and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV https://www.youtube.com/c/iTWireTV/videos which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinars and campaigns and assistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.

MORE INFO HERE!

BACK TO HOME PAGE
David Heath

David Heath has had a long and varied career in the IT industry having worked as a Pre-sales Network Engineer (remember Novell NetWare?), General Manager of IT&T for the TV Shopping Network, as a Technical manager in the Biometrics industry, and as a Technical Trainer and Instructional Designer in the industrial control sector. In all aspects, security has been a driving focus. Throughout his career, David has sought to inform and educate people and has done that through his writings and in more formal educational environments.

Share News tips for the iTWire Journalists? Your tip will be anonymous

Subscribe to Newsletter

*  Enter the security code shown:

WEBINARS ONLINE & ON-DEMAND

GUEST ARTICLES

VENDOR NEWS

Guest Opinion

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News

Comments