Hello!

You've landed in the archive of the Cambrian House community. We've kept some pages here for posterity but the community is no longer active. Now we market the technology that made our early crowdsourcing a success.

Can we help you get to Cambrian House the company? – Come on over.

Are you seeking crowdsourcing technology? – Check out Chaordix by Cambrian House.

Thanks for dropping by
The Cambrian House Crew

Close [x]
Cambrian House

I think web 2.0 and CH are the missing link for ordinary people to contribute meaningfully in the new ways of creating wealth in this so called new economy.
Cycko, Feb 2007

Cambrian House began as a crowdsourcing community using a wisdom of crowds based approach to discover new business and technology ideas. These pages are being kept online as a technology demo to showcase Chaordix™.

Looking to harness the power of your crowd? Find out about Chaordix™ - technology that enables enterprises to get the most out of crowdsourcing.

Distributed peer-to-peer backup service

whoyouknow
whoyouknow is offlineSend a Message to whoyouknowAdd whoyouknow as a FriendSend a Hat Tip to whoyouknow
  • Submitted by: whoyouknow
  • Created: May 28, 2007, 11:51 pm
  • Share on Facebook
  • Promote
 

Join Cambrian House

People

Ideas

Businesses

Connect with talented people. Collaborate on ideas. Realize your vision.
It's free! Like love in the sixties!

The Idea

There are many online backup solutions available but each requires the subscriber to trust an organization with holding your personal and private data in a secure manner.

This new service provides a reliable online backup without the need for large data centers, large amounts of data storage, or trusting a 3rd party. Instead, it uses the available hard drive space of the PCs of trusted friends and family members that are connected to the Internet.

Subscribers will invite their close friends, family members, or trusted colleagues to use the service and specify that their data is stored only on computers owned by people in their network. The system saves parts of the subscriber's files on each computer such that no single person, other than the owner, can re-create the original file.

Think desktop backup using BitTorrent like technology.

I thought of this idea when I was...

I have lost significant amount of work on my computers due to disk failures and sometimes human error. I also help out my neighbours and many have also lost data due to viruses, hardware problems, or accidental deletion.

A more complete PDF describing the problem and potential solution can be found at:

http://www.whoyoukno...Backup%20Concept.pdf


Comments Posted

Moogy
Moogy Posted: May 29, 2007, 7:13 am

In principal I like you idea..

The problem is disk space... or the classic crash an re-install.

The logistics would be to complicated

]V[oogy

JelmerBV
JelmerBV Posted: May 29, 2007, 9:12 am

Nice idea, but a problem is how to make the connection between the PC's secured enough.

TheGuru
TheGuru Posted: May 29, 2007, 10:06 am

I like this idea in theory. The main problem with P2P is that all your nodes won't be online all the time, and Murphy dictates that the file you really need at 2 am will be the one that uncle Fred has on his computer, but he's had a trojan virus and it wiped his hard drive last night, so you may never see your data!
Even if you scramble, encrypt, and redundantly spread out your data across several "trusted" friends and family, it will still not be as secure as it would be being backed up to a 500Gbyte hard drive ($150). Terabyte drives are now available at ~$300. . Also, there appears to be a large asymmetry in most broadband internet services (DSL or cable) - for example Bell, 4 Mbits downstream, 128 kbit (a measly ~16Kbyte/sec) upstream. So backing up even a modest 40 GByte hard drive will take about 32 days and inconvenience 40 of my best friends to the tune of a gigabyte each! Compare that to download rates of 4 megabits (~500K/sec or realistically 200-300K/sec) which yields 25-50 hours. Of course you can back up only "My Documents" etc, but you can do that easily and quickly to a CDR, or better yet to a "home-frame" terabit drive.

Brenden
Brenden Posted: May 29, 2007, 5:39 pm

I just worry about the disk space... I don't know about u but I keep filling my hard drive.

GreatRedSpirit
GreatRedSpirit Posted: May 29, 2007, 6:24 pm

TheGuru sums up my biggest concern with this method. To extend it a bit, this method of backup is completely inapplicable for anyone who is still on dial-up (You thought 32 days was bad?). For the time cost, I'd rather just spend ~$100 for a small external harddrive (my own backup is about 20GB), or just burn some DVDs; most computers have a burner nowadays and DVDs are nice and cheap. Or just CDs, which aren't as convienent but are even more accessable for people without DVD burners.

For implementation of the P2P backup, I'd suggest adding a Parchive style backup in case one of your friends has a problem and loses your data. Have each friend have a few PAR files, including what some other firends have so short of a massive widespead failure you can retrieve all of the PAR files for the backup, and in the case a friend loses their data, you can reconstruct it using the rest of the data and the PAR files.
This is assuming that the backup file would be stored in something like an archive broken into peices and then distributed (like WinRAR), and that Parchive can handle missing fragments of a gig or so, I've only used Parchive when ~50MB fragments were missing.

whoyouknow
whoyouknow Posted: May 29, 2007, 11:58 pm

The data is compressed and (optionally) encrypted using strong encryption (e.g. 448bit bloflish) before it even leaves your computer so the communication between your computer and another need not be secure. Only parts of the file are given to each node. In fact there's no need for encryption of the bits going to each node since each of the parts of a file on their own are useless to someone trying to use them.

This service is targetted at broadband users not dialup so that's one of the (key) pre-requisites I forgot to mention.

You only backup your data files, for Windows users that's typically My Documents. The initial backup will take a long time absolutely but you will not be backing up your entire hard disk, only data files. If you are a high end digital photographer or videoographer you'll likely look at alternate solutions.

Most people that talk about filling up disk drives are likely downloading movies or installing bloated software. Again, you would not backup full ripped movies to this service, those you would burn DVDs of etc.

If your a normal consumer you'll likely not have terabytes of actual data. I have 1,000 songs and that's under 4Gb plus my 5,000 digital photos require under 5Gb of disk space. Granted I have an old 4MP digital camera but double that and we're still talking 14 Gb of real data. If I just do my key documents, contacts, emails it's much < 1GB compressed.

I was using datadepositbox.ca for a while and the initial backup of 2Gb of data happened while I was sleeping. Once the initial backup was done, only incremental changes to files were distributed, not the entire file(s). This all happened in the background. As soon as I saved a file, it's changes were shipped off. Absolutely no human intervention. I slept very well at night.

The big issue with using a seperate hard drive for backups is that, although initially enticing due to low per GB HDD costs, if you have a fire, flood, etc. both your PC and the extra HDD are toast. All those personal digital photos taken over years are gone forever.

Backing up manually to CDs or DVDs requires human intervention which is a huge problem. Very few consumers burn their data to DVDs or CDs. I don't have numbers but based on my experience hardly anyone backs up except for technical folk.

The idea is that you're not inconveniencing your friends, this all happens in the background, and the consumption of bandwidth can be throttled similar to many P2P apps.

Hope that helps clarify and address some of the challenges.

Thanks for the great feedback, keep it coming!

fossiloflife
fossiloflife Posted: May 30, 2007, 12:44 am

ok one keeps cleanin his disk space!...how does one fight tht mentality.. and again security issues wil be a lot!

CraigH
CraigH Posted: June 4, 2007, 8:37 am

I think this is a great idea. There are lots of challenges to make it happen, but I believe it is achievable.

TheGuru
TheGuru Posted: June 11, 2007, 12:36 pm

A friend of mine uses Bell's backup service, and pays I think 5$ per month. Of course anyone with webspace or an ftp account, even GMAIL have have a substantial amount of free storage that could be accessed by an application running on the PC without any need for P2P at all. Yuo could even spread it over several gmail/hotmail accounts for redundancy.

whoyouknow
whoyouknow Posted: June 14, 2007, 3:51 pm

Yes, agree if you are a geek who knows how to do gmailfs or such esp. now that Yahoo is unlimited storage as well. I think Bell is re-selling a datadepositbox like service. Your comment supports the idea that people will pay, if in fact Bell is signing up subscribers for that service! Thanks for the info.

 

Post A Comment

Got something to say?
Log in to post a comment.