Digital Downloads are so 2010.

I used to do a lot of mobile dj'ing and at one point had a collection of 800 compilation CD's. When I gave the DJ lark up, I copied the whole lot on to a 500Gb USB portable drive and sold all the CD's on Ebay for a tidy sum. Somehow the thought of uploading the whole lot on to a cloud fills me with dread and ok, doesn't sound too bad at the same time - 'dread' because I will be entrusting the likes of Google with my entire compilation music collection and 'good idea' as I will be able to access my music anywhere I am via an available internet connection. The stumbling block for me here is the BT Internet allowance of 40Gb bandwidth a month will be exceeded in one fell swoop with all that uploading.

I come from the other end of the spectrum. the amount of MP3 files I have lost over the years I really cannot count!

To me, the cloud services on offer will be a good thing for those of us who want to store our music, video and photo files in a safe place without worrying if our hard drives will fail or what DVD did I back that collection of albums onto...

At 500GB that means you recorded at the full original quality of the CD's about 640MB per CD, these cloud services are not really ready to stream uncompressed music, it's really meant for fairly highly compressed version of your collection. I mean it "could" physically stream a 640 MB file off a server but you simply would not be doing this on a phone or other mobile device like a laptop as the data charges would be crippling, plus the chances would be that at full quality the stream would stutter regularly.

I hope you have backed up that drive really well Rob, the physical CD's had a much longer archival life than any hard drive as long as you took care handling them and I am sure you would have.

I know you are pretty good at thinking ahead on stuff like this and must say I was surprised to read you saying you had sold the masters?

My upload speed is about 800Kb to 1Mb and I have tried in the past to do very large uploads, like say 600 12MB photos or 7.2 GB roughly, takes a day or more to upload that much, so you would be looking at 70 days nearly to upload a 500GB collection, I also very much doubt that Google would even allow that much disk to be consumed on a cloud service like this, you pay serious money for this kind of level of cloud storage currently.

Some things I entrust happily to the web, but there is lots of stuff that I just could and would not.

Yep

Clouds are big big big arrays of hard disks in the sky and racks and racks of flying server computers, indeed the computers are often even more cloudy as there are usually many virtual computers running on each real physical one

In the cloud it is supposed to be so foggy that we can't actually see the real physical machines or even for that matter the boundaries between real or virtual machines, we are supposed to concentrate on the actual service delivered to us and the machine that does it is supposed to be pretty much invisible.

Services are things like storage (hard disks) processing (computing number crunching) content delivery (web serving) and so on.

The jargon these days talks about end-points, and end-point is the address of a service that does something for you and there are lots of types, you can have content stream end-points, a database endpoints, application endpoints, message queues all sorts of stuff.

Essentially this music in the cloud thing is just about downloading and playing at the same time (streaming), you still download the music data you just don't store a local copy on your device, the benefit is that for example I can get about 40 albums worth of compressed music on my phone because it only has 8GB of ram, with streaming my library is out there somewhere, anywhere really, I might play a song one minute that comes from a server in the USA and the next minute I play another track that is actually stored on some servers in France, it's supposed to make better use of resources and in some ways it's like we have moved back to a mainframe model, the one big giant computer in the sky, it's in the sky simply because it's everywhere

But just to confuse things you have private and public clouds, private clouds are where your company owns the hardware and buildings and infrastructure but provide a cloud like service based environment for all of your business units, this is really not unlike mainframe architectures of old.

I would never throw away my master copies Peter, I mostly even keep compact flash cards and SD cards as a use once device, i.e. they are my masters, my negatives as such, the archival properties of compact flash cards are way higher than photographic negatives or slide film or for that matter positive prints, so I prefer when I can afford it to buy new compact flash cards when the old one fills up. But I also have a couple of copies on disk and offsite backup as well.

I absolutely do not want to lose anything

I am not entirely ready to trust the cloud yet, last week one of our customers who was using a private cloud third party provider to run their entire business ERP process, i.e. to run their entire business suffered a disaster, or potential industrial espionage.

Someone took a JCB and rammed it into the datacentre right through the walls, once they were in they went round the racks and ripped out all the disks from the striped arrays, just chucked them on the floor and some went missing, some server racks were stolen, all sort of sh*t happened.

Then it turns out the datacentre company had not physically labelled the disks so they had no way to tell which array was which i.e. how to reassemble the arrays!

They ended up having to re-provision new servers and restore from tape but then they found out that the primary backup system was writing to one of these disk arrays as a staging area and the offsite backup stream was in essence happening a day later, so they ended up with one backup for the real time shop floor planning systems that was good for Friday and the main ERP system (our stuff) was only good up to Thursday night, nightmare!

The people who did this really vandalised this data centre and there were a lot more much larger companies affected by this, although this was one of our bigger clients.

So when you don't really know if your data is living in a shed in Mexico or a nuclear bunker in Sweden (like wiki-leaks) then I feel the cloud is a little bit dodgy to trust.

The very best systems have redundant power, comms and military grade physical protection like that Swedish centre (pretty sure it was Sweden but I could be wrong) the worst could well be worse than a shed in Mexico, the problem is that the cloud is so cloudy that you can't tell where your stuff really is, that's why I don't trust it.

That was very interesting indeed..................


I didn't understand a word..............

I see what you're saying oss.. I won't be using some cheap as chips cloud service.

However, we already use cloud services. We upload pictures to Picassa, Flickr, Facebook and others. We now upload videos to YouTube and the like for safe keeping (privacy setting can be added). I just see the new Apple icloud service and the ones from Google, Microsoft and others as being an extension of this. But I think it is more of a sense of where technology providers are taking us.

Everyone talks about the iPhone and the changes it is making to the communications industry, but other companies like Nikon and Canon are looking into instant uploads from professional level cameras to a cloud service (to be edited later), this is something suggested by the news agencies to allow Photographers to stay in the field knowing that somewhere else in the world their employers have someone editing their images ready to hit the news. This same technology would be a great for people taking pictures of their children and knowing it is already uploaded before they switch on their laptop. Not to mention the Internet ready TV systems out there which are promising to converge the Internet with our TV browsing.

All of this will one day link up. And it will be the cloud that joins it all together..

Trouble is you don't know for certain where Microsoft or Google are storing your data and you also don't know what the legal regime is in the physical countries that your data ends up in.

Regards Flickr I have 3 accounts, one for Ana, one for me and one is my jpeg backup store nothing public at all over 20,000 jpegs in that one now (still not my entire collection) but again my archives are backed up elsewhere, Flickr is just one branch of my backup strategy.

You make a good point about the camera companies, Canon and Nikon already have solutions to stream data off camera's but existing solutions usually rely on a computer and phone being nearby.

You can actually do exactly what you describe today with a Ey-Fi Pro X2 SD Card and the appropriate phone app, wifi link to the phone from the Ey-Fi to despool the images and a phone app to upload them to wherever you need asynchronously.

You are right it is all linking up and it is the future, but I still won't trust it with the master copies of anything I make, be it photos, software or documents.

That disaster story last week was not of a company that had skimped on hosting, they were using a high end hosting company and this highly respected hosting company boasted of their disaster recovery plans, they just hadn't thought of everything, might indeed be the reason someone attacked them, maybe to prove a point.

If you think about it, any website or email service is in effect a 'cloud' service to a degree.

But who do you trust? As an example -an email service that include the ability to store data as well as email, do I trust a small company like Tuffmail in Florida with backups in US only (lets say a datacenter in NY and a backup in Canada) with your email? Or a giant company like Yahoo who have many, many employees working round the clock maintaining servers all over the world, with all data shared between many geographic locations? Sure, Tuffmail in Florida will personally answer your support emails within an hour, and Yahoo will probably never do that. Oh, Yahoo is free, but Tuffmail charge $50 a year for a premium class service to include many services Yahoo will never include. Thats the trade-off.

As for data retention, I'd imagine this would be subject to T & C's of the service. I'm now using Google Applications to host emails, documents, photos, calendar, etc, and as far as I can see, data retention is permanent unless subject to a court order or you delete it (they of course will keep back ups for a time). If your using a paid service like I was doing, that data would surely expire at the end of the contract term, but I'd imagine they would keep data for many years after 'just in case'.

The security of data stored in the cloud, we'll never know, as the likes of Google are hardly going to reveal minute information about the security of their servers, but having used Google stuff for many years, I'll take my chance with them, they seem to know what they are doing, and its plain to see they take security quite seriously. I've migrated quite a few organisations over to Google Apps, and it was a painless, hassle free process, and still works as intended many years later, I've never had cause to contact Google.

I wouldn't say cloud storage is unsafe ; it offers many positives over storing data locally; example, your house is burgled or burned down; data gone for good. The key thing is user security (strong passwords, backup email address, SMS notification, etc).

Peter since the earliest days of internet web browsers when IT guys were creating a presentation to explain to the boss how things would hang together in their network for example, the internet was always a big picture of a cloud, like this.



Because this is how us IT folks thought about it for the last 20 years it just gradually became mainstream.

I remember putting this kind of diagram together for my clients when I was doing proposals 15 years back.

When you upload anything it ends up in a hard drive in a building somewhere, once upon a time the drive would have been inside a computer now it's in what is called a storage area network or SAN, when you upload something you are actually talking to a physical computer, that computer takes the bytes of the file you are sending and moves them to storage in the SAN, a hard disk ultimately, at the same time the application (computer code) that allowed this computer to save the file wrote some records to a database so that it could remember how to get your photo file back for you when you want to use it.

When I talk of a hard disk here we are actually taking huge arrays of hard drives row upon row of devices that each store multiple physical hard drives for example each of the rows in this image looks like it has 16 blades per slot, (this is a person sized cabinet by the way) each blade probably has upwards of 6 physical hard drives just like the ones in your home computer.



So this thing at a guess could store say 1000 Tera Bytes if each drive was of 1TB capacity.

A real data centre will have hundred of these cabinets, just think of the electricity required for all that, this is why the internet and all our connected systems are using upwards of 2% of global electric generation, it's also why they are starting to build these things up in the Arctic circle because it's cheaper to keep them cool up there

Basic cloud storage just moves a copy of the bytes in your files to hard drives on their storage companies computers, if you want your file back you get it back unchanged.

On the other hand something like the online Adobe Photoshop program will run as a program on a computer "somewhere on the web" and will be able to open your file from that hard drive at the storage company and will then allow you to issue instructions over the internet to make adjustments to your photo, this is a web based application. When you tell it to save changes it's just like doing the same thing on your own computer at home it has to write the changed data back to the hard drive at the storage company.

The cloud is all about doing things remotely instead of on your own hardware at home and when you are working with media you created like photos step one is to copy the media to somewhere in the web.

Yes this is the rub, all you have is service level agreements, you really don't know if Microsoft or Google or anyone else has made multiple redundant copies of your data that are fully protected and that you can retrieve easily in the event of a disaster. It is even possible these days that bits of your data could be stored on entirely different continents, a lot of Microsoft data centres are in places like Mexico some other ones in Italy and elsewhere, you just don't know the level of security of the buildings, you don't really know how good the offline and offsite backups are, just so many things you can't be 100% sure of.

Basically you don't know, ownership should still be yours but the data is not encrypted in any way and could be open to abuse by unscrupulous employee's of the cloud service operator.

Plus the location of your data may affect the legal jurisdiction that applies.

I will add another note, personally I use Flickr for offsite storage of my photos but Flickr is not an independent company and yahoo have proved to a potentially unreliable owner of Flickr, rumours have been around for years that they might shut down or sell off Flickr!

Now about 6 years ago when Flickr was a little startup company in Canada and got bought up by Yahoo there was a several day process where the data (the photos and Flickr database) was physically moved from Canada to the US to bring it to Yahoo's server and storage farms. Now, you must understand that Flickr is absolutely huge in terms of it's storage requirements these days the idea of moving it to a new location is mind boggling, so while I would be devastated if Flickr ever shut down, it's not my only backup and it is only capable of backing up jpegs and tifs anyway it can't store my RAW images, it would not be the end of the world but it would hurt after all the effort I put into Flickr over the years.

I also use Rackspace for online Microsoft Exchange and I will be moving this to Microsoft Office 365 next year to save some money, additionally I use an online specialist system for management of my software projects but again I have multiple local backups.

From a personal viewpoint, the cloud could give me access to computing power and application scalability that I could otherwise not afford or access. If I had a good idea for a commercial web application I could try it out fairly quickly on Microsoft's Azure platform, I would need their SQL database storage, Web hosting and Web Service platforms and some level of computation, Microsoft charges for this stuff in a very granular fashion, you pay for bytes of traffic, total compute time and bytes of storage. A successful web application could quickly start to cost a fair bit as more users means more consumption of all the above resources and that means a bigger bill from Microsoft but it would allow an application to scale (accept more and more users) much more easily than having to provide my own infrastructure.

So it has it's advantages from business viewpoint, but it is still dangerous territory for most businesses in my view, running mission critical apps this way is fraught with dangers.