Featured IT News

Bingo Industries pleads guilty to alleged demolition waste cartel: ACCC

16 August 2022 in Technology Regulation

Waste company Bingo Industries has pleaded guilty to criminal cartel offences relating to price fixing for demolition waste services in…

NT Acacia health system goes live at Katherine Hospital

16 August 2022 in Enterprise Solutions

The Northern Territory Government has rolled out the first stage of its $259 million Acacia patient health record system at…

ACCC to examine competition and consumer ‘concerns’ with social media

16 August 2022 in Technology Regulation

The state of competition for social media services and consumer concerns with the services in Australia is to come under…

Veteran advertising exec Wayne Moulton joins Hoppr

16 August 2022 in People Moves

Australian advertising company Hoppr has appointed industry advertising veteran Wayne Moulton as vice president of business development for Australia and…

Checkmarx launches managed security service provider program

16 August 2022 in Security

Application security testing solutions vendor Checkmarx has launched its managed security service provider (MSSP) program.

‘Smart Bin’ tech among CSIRO’s latest recycling innovations

16 August 2022 in Enterprise Solutions

Australia’s national science agency, CSIRO, has unveiled its latest innovations in recycling including Smart Bin Technology to automatically sort plastic,…

Forescout appoints Nextgen sole Australia, NZ distributor

16 August 2022 in Market

Automated cybersecurity firm Forescout Technologies has appointed Nextgen as its sole distributor for the Australia-New Zealand as part of its…

Lawyers sue CIA, sec firm for alleged spying on Assange, visitors

16 August 2022 in Security

Lawyers have filed a suit against the CIA, its director at the time Mike Pompeo and security firm UC Global…

Cisco pledge to fix all bugs revealed by sec firm Rapid7

16 August 2022 in Security

Networking giant Cisco has said it has either fixed or will fix bugs revealed in its products by security firm…

Aspen Medical implements Oracle Cloud apps

15 August 2022 in Health

Australian healthcare solutions provider Aspen Medical has worked with Oracle to consolidate its business processes across HR, finance, and projects…

Myths spread about news media bargaining code before review lands

15 August 2022 in Open Sauce

Former Australian Competition and Consumer Commission chairman, Rod Sims is continuing to try and spin the myth that the News…

Federal Court fines Google $60m for collecting data on the sly

13 August 2022 in Technology Regulation

The Australian Federal Court has fined Google $60 million for misleading consumers about collection and use of personal data on…

Watch: Guardians of Samsung's Flagship Galaxy can Flip, Fold and be best Buds 4 ever, with the crease decreased!

12 August 2022 in Your Tech

Samsung's fourth-get foldables are here: the Galaxy Z Flip4 and Z Fold4 have arrived, in bolder, lighter, more powerful and…

NBN Co releases revised SAU variation discussion paper

12 August 2022 in Telecoms & NBN

National Broadband Network operator NBN Co has released a new Special Access Undertaking (SAU)  discussion paper to the Australian Competition…

Digital tool helping cancer survivors cope with fear of recurrence

12 August 2022 in Health

A team led by UNSW Sydney researchers is investigating ways through e-health to increase access to much-needed treatment for fear…

Breaking IT News

Azul appoints Vaughan as APAC VP

16 August 2022 in People Moves

Java specialist Azul has appointed Dean Vaughan to the role of Asia Pacific vice president.

ATech appoints chair, operations manager, community manager and senior developer

16 August 2022 in People Moves

Brisbane-based cloud hosting and web development company ATech has appointed Gary Linton to the newly created role of non-executive chair,…

Canva introduces new way to brainstorm with Canva Whiteboards

16 August 2022 in Business IT

Visual communications platform Canva launched Canva Whiteboards, a new way to organise ideas and bring them to life.

Oracle NetSuite introduces referral program

16 August 2022 in Business Software

Oracle NetSuite's SuiteReferral program rewards businesses that invite others to use the NetSuite business software.

HyperX signs DJ Zedd as global brand ambassador

16 August 2022 in Entertainment

The international producer Zedd will collaborate with HyperX, the gaming peripherals team at HP Inc., and brand leader in gaming…

On the shoulders of the Honeywell giant

16 August 2022 in Development

Necessity is the mother of invention, and a popular invention generally sees many claiming parenthood. Indeed, many companies claim that…

Network security sales estimated to exceed $210 billion in the next five years: Dell’Oro Group

16 August 2022 in Business IT

Demand for network security—email security, firewall, security service edge, secure web gateway, and web application firewall technologies—is expected to remain…

Optus launches 5G Standalone

16 August 2022 in 5G

Telecommunications company Optus has launched its 5G Standalone network after close to year of testing including customer trials, offering network…

Snapchat+ adds four new features for subscribers

16 August 2022 in Entertainment

The Snapchat+ subscription service now has over a million paying subscribers globally, and from today features another four new exclusive…

BT snags $54.5 million contract with Sellafield

16 August 2022 in Business Telecoms

British telecommunications company BT has secured a £32 million five-year ($54.5 million) network services contract with nuclear site operator Sellafield.

Total telecom and pay-TV services revenue booms to $30.7 billion by 2026: forecast

16 August 2022 in Business Telecoms

Total telecom and pay-TV services revenue in Australia is expected to grow at a CAGR of 1.6% from US$20.2 billion…

Life is better with a soundtrack and Bose says you can have portability and audio quality together

15 August 2022 in Computers & peripherals

The king of home audio shows it's the king of portable audio also, with these lightweight, weatherproof, smart, highly-connected offerings.

 

Reviews

 

Opinion

 

Telecommunications

 

Popular News

Subscribe to Newsletter

*  Enter the security code shown:

WEBINARS ONLINE & ON-DEMAND

GUEST ARTICLES

VENDOR NEWS

Guest Opinion

Guest Interviews

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News

Comments

Wednesday, 07 April 2010 02:26

Pete Warden vs. Facebook: a case of too much data access

By

Pete Warden had a really great idea: to map the friendship interactions of Facebook users to aid with geospatial analysis of user relationships.  Facebook's lawyers had a different view.

The entire saga is laid out as irregular snippets on Pete Warden's blog but to assist readers, we will extract the salient details, mostly in Pete's own words.  Our analysis is at the end of this report.

A couple of years ago, Pete Warden created a start-up company called Mailana (short for Mail Analysis).  "My goal has been to use the data sitting around in all our inboxes to help us in our day-to-day lives." 

Some time later, the focus changed.

"I'd already applied the same technology to Twitter to produce graphs showing who people talked to, and how their friends were clustered into groups. I set out to build that into a fully-fledged service, analyzing people's Twitter, Facebook and webmail communications to understand and help maintain their social networks.

"It offered features like identifying your inner circle so you could read a stream of just their updates, reminding you when you were falling out of touch with people you'd previously talked to a lot, and giving you information about people you'd just met."

On March 17th, Warden announced that due to the threat of legal action from Facebook, he had destroyed all accumulated Facebook-derived data linked to the analysis of social networks.


His summary of events continues: "As you can imagine I'm not very happy about this, especially since nobody ever alleged that my data gathering was outside the rules the web has operated by since crawlers existed. I followed their robots.txt directions, and was even helped by microformatting in the public profile pages. Literally hundreds of commercial search engines have followed the same path and have the same data. You can even pull identical information from Google's cache if you don't want to hit Facebook's servers. So why am I destroying the data? This area has never been litigated and I don't have enough money to be a test case."

Despite the ill-will over the entire incident, Warden does express some sympathy for Facebook noting that he was about to release data that they had no idea was available.

"I know from my time at Apple that reaching for the lawyers is a tempting first option when there's a nasty surprise like that. If I had to do it all over again, I'd try harder not to catch them off-guard."

Despite this setback, he observes that it isn't particularly difficult for anyone to obtain data from Google and he is already working on a similar analysis on publicly-available data linked to Google Buzz.

Getting back to the history of the saga: "I noticed Facebook were offering some other interesting information too, like which pages people were fans of and links to a few of their friends. I was curious what sort of patterns would emerge if I analyzed these relationships, so as a side project I set up fanpageanalytics.com to allow people to explore the data. I was getting more people asking about the data I was using, so before that went live I emailed Dave Morin at Facebook to give him a heads-up and check it was all kosher. We'd chatted a little previously, but I didn't get a reply, and he left the company a month later so my email probably got lost in the chaos."

Pete next describes the hell that broke loose upon him.


Once he had around six months' data, he started to do some simple analysis - the initial publication of this was met with crickets and tumbleweeds (as he described the amazing amount of interest shown!).

Next, in a fit of boredom he took to the data with some colouring tools to highlight some of the simple patterns he'd noticed.

Within 2 days, his blog post had received something like 200,000 hits via YCombinator and Reddit. 

Next his mobile phone rang with Facebook's attorney on the other end.

"He was with the head of their security team, who I knew slightly because I'd reported several security holes to Facebook over the years. The attorney said that they were just about to sue me into oblivion, but in light of my previous good relationship with their security team, they'd give me one chance to stop the process.

"They asked and received a verbal assurance from me that I wouldn't publish the data, and sent me on a letter to sign confirming that. Their contention was robots.txt had no legal force and they could sue anyone for accessing their site even if they scrupulously obeyed the instructions it contained. The only legal way to access any web site with a crawler was to obtain prior written permission
."


"Obviously this isn't the way the web has worked for the last 16 years since robots.txt was introduced, but my lawyer advised me that it had never been tested in court, and the legal costs alone of being a test case would bankrupt me. With that in mind, I spent the next few weeks negotiating a final agreement with their attorney.

"They were quite accommodating on the details, such as allowing my blog post to remain up, and initially I was hopeful that they were interested in a supervised release of the data set with privacy safeguards. Unfortunately it became clear towards the end that they wanted the whole set destroyed. That meant I had to persuade the other startups I'd shared samples with to remove their copies, but finally in mid-March I was able to sign the final agreement."

"I'm just glad that the whole process is over. I'm bummed that Facebook are taking a legal position that would cripple the web if it was adopted (how many people would Google need to hire to write letters to every single website they crawled?), and a bit frustrated that people don't understand that the data I was planning to release is already in the hands of lots of commercial marketing firms, but mostly I'm just looking forward to leaving the massive distraction of a legal threat behind and getting on with building my startup.

I really appreciate everyone's support, stay tuned for my next project!"

An analysis of these events is on the final page.


This has set a dangerous (albeit privately agreed) precedent; one which is diametrically opposed to the judgement handed down in Google vs. Field where the judgement specifically referenced the robots.txt file stating that the plaintiff (Blake Field) was fully aware of the proper use of the robots.txt file and yet chose not to make use of it to bar his copyrighted works from being crawled and cached by Google. 

Field's legal action against Google was regarded rather poorly by the judge: "Field decided to manufacture a claim for copyright infringement against Google in the hopes of making-money from Google's standard practice."  The judgement also noted that "author granted operator implied licence to display 'cached' links to web pages containing his copyrighted works."

Although appearing to support Facebook's claims of doing all they can to protect the privacy of their subscribers, the action against Warden seems aimed more at protecting a valuable resource and revenue-stream; exactly the same motivator the company has for constantly fiddling with the privacy dashboards in the hope users won't notice the changes that make more and more data available.

Facebook should also check with the courts to ensure they haven't misunderstood the current legal standing of the robots.txt file!

 

 

Read 5057 times

Please join our community here and become a VIP.

Subscribe to ITWIRE UPDATE Newsletter here
JOIN our iTWireTV our YouTube Community here
BACK TO LATEST NEWS here

GET READY FOR XCONF AUSTRALIA 2022

Thoughtworks presents XConf Australia, back in-person in three cities, bringing together people who care deeply about software and its impact on the world.

In its fifth year, XConf is our annual technology event created by technologists for technologists.

Participate in a robust agenda of talks as local thought leaders and Thoughtworks technologists share first-hand experiences and exchange new ways to empower teams, deliver quality software and drive innovation for responsible tech.

Explore how at Thoughtworks, we are making tech better, together.

Tickets are now available and all proceeds will be donated to Indigitek, a not-for-profit organisation that aims to create technology employment pathways for First Nations Peoples.


Click the button below to register and get your ticket for the Melbourne, Sydney or Brisbane event

GET YOUR TICKET!

PROMOTE YOUR WEBINAR ON ITWIRE

It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site itwire.com and prominent Newsletter promotion https://itwire.com/itwire-update.html and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV https://www.youtube.com/c/iTWireTV/videos which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinars and campaigns and assistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.

MORE INFO HERE!

BACK TO HOME PAGE
David Heath

David Heath has had a long and varied career in the IT industry having worked as a Pre-sales Network Engineer (remember Novell NetWare?), General Manager of IT&T for the TV Shopping Network, as a Technical manager in the Biometrics industry, and as a Technical Trainer and Instructional Designer in the industrial control sector. In all aspects, security has been a driving focus. Throughout his career, David has sought to inform and educate people and has done that through his writings and in more formal educational environments.

Share News tips for the iTWire Journalists? Your tip will be anonymous