Featured IT News

CGI to acquire tech consultancy Unico

07 December 2021 in Strategy

Global IT and business consulting services firms CGI has announced that its subsidiary, CGI Technology and Solutions Australia Pty Limited,…

The price of electricity around the world

07 December 2021 in Energy

A recent report maps the price of electricity in every country around the world.

Coles plans major hiring of digital professionals for ecommerce business

07 December 2021 in Recruitment

Coles plans to hire more than 70 digital professionals to join its eCommerce business, in a move the retail giant says…

bp, Schneider Electric ink deal on low carbon energy solutions

06 December 2021 in Energy

Integrated energy company bp and digital transformation of energy management and automation provider Schneider Electric have signed a Memorandum of…

New hydrogen clusters to support future clean hydrogen industry

05 December 2021 in Energy

Three new regional hydrogen technology clusters have been identified to further support the development of Australia’s future clean hydrogen industry.

RACV powers Victoria’s Airey’s Inlet community centre with solar

05 December 2021 in Energy

The Royal Automobile Club of Victoria (RACV) has equipped Aireys Inlet Community Centre in Victoria’s surf coast region with solar…

New legislation aimed to protect consumers and support competition

04 December 2021 in Technology Regulation

The Telstra Corporation and Other Legislation Amendment Bill 2021, has passed, guaranteeing that important consumer outcomes continue to be delivered…

Aussie Broadband proposes to acquire Over the Wire in a recommended transaction

04 December 2021 in Telecoms & NBN

Aussie Broadband Limited (ABB) announced that it has entered into a Scheme Implementation Deed with Over the Wire Limited (OTW)…

Ubiquiti ex-worker held over data theft, extortion bid: DoJ

02 December 2021 in Security

An ex-employee of the wireless data communication and wired products maker Ubiquiti has been arrested and charged with data theft…

Don't be Square, go round the Block

02 December 2021 in Listed Tech

Square, the company best known for its card transaction processing service and its associated POS software, is changing its name…

Government partners with industry to stop scam text messages

02 December 2021 in Technology Regulation

The Federal Government says it has taken strong action to shut down scam text messages, making an important regulatory amendment…

Qualcomm delivers 3rd-gen Snapdragon 8cx for mobile computers, gaming and 7c+ for 'entry level' laptops

02 December 2021 in Home Tech

The race to perfect the Arm platform to power ultra slim and light Windows 11 and Chromebook PCs is hotting…

EU tech firms file anti-trust complaint against Microsoft

02 December 2021 in Technology Regulation

Open-source file syncing and sharing software company Nextcloud, along with 30 other technology companies in the EU, has filed an…

Frollo, Beyond Bank team up on Open Banking solution

01 December 2021 in Strategy

Australian Open Banking provider Frollois partnering with customer-owned bank, Beyond Bank, using Frollo’s CDR gateway to improve multi-banking and lending…

Qualcomm reboots 8 series flagship Snapdragon with new name, smarter smarts to fight everyone

01 December 2021 in Home Tech

With Apple arguably the world's most advanced SoC maker for smartphones, tablets and computers, the ARM and x86 competition is…

Breaking IT News

Epson simplifies colour matching with SD-10 spectrophotometer

07 December 2021 in Print Solutions

Epson's SD-10 spectrophotometer is said to be a highly accurate, compact and affordable spectrophotometer that can be used to digitise…

Nokia to drive KDDI’s 5G transition with standalone core and monetisation solutions

07 December 2021 in 5G

Nokia announced that Japanese operator KDDI has selected Nokia’s 5G Core and Converged Charging software to support its transition to…

Bendigo and Adelaide Bank automates its operations with EncompaaS cloud platform

07 December 2021 in Cloud

Bendigo and Adelaide Bank has picked the EncompaaS cloud platform to automate its regulatory and business governance obligations for electronic…

Fibre outperforms other tech during lockdown in NZ

07 December 2021 in Telecoms & NBN

Auckland experienced no significant drop in broadband performance compared with the rest of New Zealand during lockdown, the latest Measuring…

Mobile network investment continues but focus is on 5G in major cities

07 December 2021 in 5G

Australia’s three mobile network operators, Telstra, Singtel Optus and TPG Telecom made significant investments in mobile infrastructure between January 2018…

Applications now open for Regional Connectivity Program funding

07 December 2021 in Telecoms & NBN

The Government has launched the second round of a program aimed at further boosting digital connectivity in regional and remote…

Speedcast unveils network-optimised remote video streaming solution

07 December 2021 in Business Telecommunications

Speedcast, a communications and IT services provider, announced the launch of Speedcast SmartView, a network-optimised remote video and audio communications…

Sydney and Newcastle use artificial intelligence and edge computing to improve road mobility

07 December 2021 in Government Tech

Sydney and Newcastle are deploying artificial intelligence and edge computing technology in a trial to improve customer journey, safety, and…

Marketplacer raises $38 million

07 December 2021 in Deals

Marketplace platform provider Marketplacer has raised $38 million from a range of investors.

Agile organisations are the most successful, says Zendesk, and here's how you can be too

06 December 2021 in CIO Trends

Research commissioned by Zendesk, the SaaS-based customer support and experience platform, shows agility is tied to the longevity of business…

Tranya continues its journey of affordable bluetooth earbuds with the Tranya T20

06 December 2021 in Computers & peripherals

Tranya isn't new to iTWire or readers; this audio brand has a mission of making quality, engineer-tuned and feature-rich audio…

VIDEO Interview: CommScope's Sanjiv Verma talks the latest evolved IoT and smart building solutions for hotels and hospitality

06 December 2021 in Business IT

GUEST INTERVIEW: RUCKUS Networks is well known for its highly rated enterprise-class wireless solutions, with the company purchased by CommScope…

 

Reviews

 

Opinion

 

Telecommunications

 

Popular News

Subscribe to Newsletter

*  Enter the security code shown:

WEBINARS ONLINE & ON-DEMAND

GUEST ARTICLES

VENDOR NEWS

Guest Opinion

Guest Interviews

Guest Reviews

Guest Research

Guest Research & Case Studies

Channel News

Comments

Wednesday, 07 April 2010 02:26

Pete Warden vs. Facebook: a case of too much data access

By

Pete Warden had a really great idea: to map the friendship interactions of Facebook users to aid with geospatial analysis of user relationships.  Facebook's lawyers had a different view.

The entire saga is laid out as irregular snippets on Pete Warden's blog but to assist readers, we will extract the salient details, mostly in Pete's own words.  Our analysis is at the end of this report.

A couple of years ago, Pete Warden created a start-up company called Mailana (short for Mail Analysis).  "My goal has been to use the data sitting around in all our inboxes to help us in our day-to-day lives." 

Some time later, the focus changed.

"I'd already applied the same technology to Twitter to produce graphs showing who people talked to, and how their friends were clustered into groups. I set out to build that into a fully-fledged service, analyzing people's Twitter, Facebook and webmail communications to understand and help maintain their social networks.

"It offered features like identifying your inner circle so you could read a stream of just their updates, reminding you when you were falling out of touch with people you'd previously talked to a lot, and giving you information about people you'd just met."

On March 17th, Warden announced that due to the threat of legal action from Facebook, he had destroyed all accumulated Facebook-derived data linked to the analysis of social networks.


His summary of events continues: "As you can imagine I'm not very happy about this, especially since nobody ever alleged that my data gathering was outside the rules the web has operated by since crawlers existed. I followed their robots.txt directions, and was even helped by microformatting in the public profile pages. Literally hundreds of commercial search engines have followed the same path and have the same data. You can even pull identical information from Google's cache if you don't want to hit Facebook's servers. So why am I destroying the data? This area has never been litigated and I don't have enough money to be a test case."

Despite the ill-will over the entire incident, Warden does express some sympathy for Facebook noting that he was about to release data that they had no idea was available.

"I know from my time at Apple that reaching for the lawyers is a tempting first option when there's a nasty surprise like that. If I had to do it all over again, I'd try harder not to catch them off-guard."

Despite this setback, he observes that it isn't particularly difficult for anyone to obtain data from Google and he is already working on a similar analysis on publicly-available data linked to Google Buzz.

Getting back to the history of the saga: "I noticed Facebook were offering some other interesting information too, like which pages people were fans of and links to a few of their friends. I was curious what sort of patterns would emerge if I analyzed these relationships, so as a side project I set up fanpageanalytics.com to allow people to explore the data. I was getting more people asking about the data I was using, so before that went live I emailed Dave Morin at Facebook to give him a heads-up and check it was all kosher. We'd chatted a little previously, but I didn't get a reply, and he left the company a month later so my email probably got lost in the chaos."

Pete next describes the hell that broke loose upon him.


Once he had around six months' data, he started to do some simple analysis - the initial publication of this was met with crickets and tumbleweeds (as he described the amazing amount of interest shown!).

Next, in a fit of boredom he took to the data with some colouring tools to highlight some of the simple patterns he'd noticed.

Within 2 days, his blog post had received something like 200,000 hits via YCombinator and Reddit. 

Next his mobile phone rang with Facebook's attorney on the other end.

"He was with the head of their security team, who I knew slightly because I'd reported several security holes to Facebook over the years. The attorney said that they were just about to sue me into oblivion, but in light of my previous good relationship with their security team, they'd give me one chance to stop the process.

"They asked and received a verbal assurance from me that I wouldn't publish the data, and sent me on a letter to sign confirming that. Their contention was robots.txt had no legal force and they could sue anyone for accessing their site even if they scrupulously obeyed the instructions it contained. The only legal way to access any web site with a crawler was to obtain prior written permission
."


"Obviously this isn't the way the web has worked for the last 16 years since robots.txt was introduced, but my lawyer advised me that it had never been tested in court, and the legal costs alone of being a test case would bankrupt me. With that in mind, I spent the next few weeks negotiating a final agreement with their attorney.

"They were quite accommodating on the details, such as allowing my blog post to remain up, and initially I was hopeful that they were interested in a supervised release of the data set with privacy safeguards. Unfortunately it became clear towards the end that they wanted the whole set destroyed. That meant I had to persuade the other startups I'd shared samples with to remove their copies, but finally in mid-March I was able to sign the final agreement."

"I'm just glad that the whole process is over. I'm bummed that Facebook are taking a legal position that would cripple the web if it was adopted (how many people would Google need to hire to write letters to every single website they crawled?), and a bit frustrated that people don't understand that the data I was planning to release is already in the hands of lots of commercial marketing firms, but mostly I'm just looking forward to leaving the massive distraction of a legal threat behind and getting on with building my startup.

I really appreciate everyone's support, stay tuned for my next project!"

An analysis of these events is on the final page.


This has set a dangerous (albeit privately agreed) precedent; one which is diametrically opposed to the judgement handed down in Google vs. Field where the judgement specifically referenced the robots.txt file stating that the plaintiff (Blake Field) was fully aware of the proper use of the robots.txt file and yet chose not to make use of it to bar his copyrighted works from being crawled and cached by Google. 

Field's legal action against Google was regarded rather poorly by the judge: "Field decided to manufacture a claim for copyright infringement against Google in the hopes of making-money from Google's standard practice."  The judgement also noted that "author granted operator implied licence to display 'cached' links to web pages containing his copyrighted works."

Although appearing to support Facebook's claims of doing all they can to protect the privacy of their subscribers, the action against Warden seems aimed more at protecting a valuable resource and revenue-stream; exactly the same motivator the company has for constantly fiddling with the privacy dashboards in the hope users won't notice the changes that make more and more data available.

Facebook should also check with the courts to ensure they haven't misunderstood the current legal standing of the robots.txt file!

 

 


Please join our community here and become a VIP.

Subscribe to ITWIRE UPDATE Newsletter here
JOIN our iTWireTV our YouTube Community here
BACK TO LATEST NEWS here

PROMOTE YOUR WEBINAR ON ITWIRE

It's all about Webinars.

Marketing budgets are now focused on Webinars combined with Lead Generation.

If you wish to promote a Webinar we recommend at least a 3 to 4 week campaign prior to your event.

The iTWire campaign will include extensive adverts on our News Site itwire.com and prominent Newsletter promotion https://itwire.com/itwire-update.html and Promotional News & Editorial. Plus a video interview of the key speaker on iTWire TV https://www.youtube.com/c/iTWireTV/videos which will be used in Promotional Posts on the iTWire Home Page.

Now we are coming out of Lockdown iTWire will be focussed to assisting with your webinatrs and campaigns and assassistance via part payments and extended terms, a Webinar Business Booster Pack and other supportive programs. We can also create your adverts and written content plus coordinate your video interview.

We look forward to discussing your campaign goals with you. Please click the button below.

MORE INFO HERE!

INTRODUCING ITWIRE TV

iTWire TV offers a unique value to the Tech Sector by providing a range of video interviews, news, views and reviews, and also provides the opportunity for vendors to promote your company and your marketing messages.

We work with you to develop the message and conduct the interview or product review in a safe and collaborative way. Unlike other Tech YouTube channels, we create a story around your message and post that on the homepage of ITWire, linking to your message.

In addition, your interview post message can be displayed in up to 7 different post displays on our the iTWire.com site to drive traffic and readers to your video content and downloads. This can be a significant Lead Generation opportunity for your business.

We also provide 3 videos in one recording/sitting if you require so that you have a series of videos to promote to your customers. Your sales team can add your emails to sales collateral and to the footer of their sales and marketing emails.

See the latest in Tech News, Views, Interviews, Reviews, Product Promos and Events. Plus funny videos from our readers and customers.

SEE WHAT'S ON ITWIRE TV NOW!

BACK TO HOME PAGE
David Heath

David Heath has had a long and varied career in the IT industry having worked as a Pre-sales Network Engineer (remember Novell NetWare?), General Manager of IT&T for the TV Shopping Network, as a Technical manager in the Biometrics industry, and as a Technical Trainer and Instructional Designer in the industrial control sector. In all aspects, security has been a driving focus. Throughout his career, David has sought to inform and educate people and has done that through his writings and in more formal educational environments.

Share News tips for the iTWire Journalists? Your tip will be anonymous