HP gives up against Amazon

So the HP public cloud is no more. I suspect I might have been one of the larger users of it (for a few weeks back in 2012) so let me try to give a serious analysis of what this means. (HP announcement link)

Amazon AWS is currently supply-constrained. They could lower prices to gain more customers, but then they wouldn't be able to service those customers. This is an unusual position to be in, as almost all of us are in industries where the bottleneck to growth is in acquisition, not delivery. So they ease their prices down little-bit-by-little-bit as they resolve their supply constraints.

Eventually, AWS will start to be demand-constrained, and that's when all hell breaks loose, because then AWS can start doing some serious price cutting. I'd peg it for early 2017 at a guess, when suddenly the price cuts start accelerating until the economics for renting from AWS starts to look competitive with buying a server and putting it on a desk unsupported, un-networked and unpowered.

Google and Azure can survive Amazongeddon -- they have the money and it's a market that they definitely want to be in. Google App Engine is still a very cost-effective offering -- my total compute and storage budget leading up to launch day (and including it) for the Automated Estimator of Effort and Duration for Jira was $0.22 -- so much for big data analysis being expensive! At that level, price comparisons are utterly meaningless, so if that's profitable now (which is probably is), they can keep doing it.

HP have presumably decided that they don't have enough time to build out a solid customer base on the HP public cloud before Amazongeddon. The HP cloud team is betting that customers will want HP software to manage their clouds, and that an HP-backed public cloud is not worth doing. Operations Orchestration makes sense in a cloudy world, for example.

But there is a problem, because for all the talk of "hybrid public-private clouds", either private is cheaper/better/more secure or public is cheaper/better/more secure.

• If the answer is "private", then we will continue to have internal customer-owned datacentres, and HPE will continue to sell 3PARs, SureStores, Proliants and so on. 
• If the answer is "public", then after Amazongeddon, HP won't have a hardware business that anyone cares about.

Unfortunately, I believe the answer is "public", as do many, many other people. To say that "private" clouds are cheaper / better and more secure the majority of the time means that not only are there no economies of scale in a big data centre, that there are diseconomies of scale that are going to appear any moment now from out of nowhere.

This puts HP in the same position as Unisys was in the 1980s-1990s. Customers stopped buying Unisys mainframes, so Unisys had to turn into a services, software and support business. They had a bit of an edge in government and defence at the time, and they worked hard to keep it. I know plenty of people who have had good careers at Unisys, and presumably it's a nice place to work where there is innovation happening. But Unisys in 2015 is not the hallowed place that it was after the Burroughs / Sperry merger.

Without that core of hardware sales on which to stack software sales, Unisys struggled. So too will HP. (And so will Dell, unless Dell decides to take on Amazon... which they could and should.)

I feel sorry for Bill Hilf though, as he has had to lead teams through the collapse of high-end Itanium hardware and now through the failure of the only viable hardware future that HP had.

That said, I'm optimistic about HP Data Protector in particular. There will still be important data to backup and archive. Storing it efficiently for fast recovery will always matter. You can't discard a backup solution until the last of your 7-year-old backups have expired.

I'm hoping that HP will now do three things:

• Convert the HP cloud object storage device to something that works with S3. Since this feature will be irrelevant in January 2016 if they don't do this, it seems like a no-brainer in order to preserve the R&D investment done so far.
• Interface into lifecycle management of S3 -- if the "location" of a piece of media is "Glacier", then Data Protector should be able to initiate its re-activation as step 1 of a restore job. Again, this seems a no-brainer if you already are dealing with S3.
• I'd like to see the Virtual Storage Appliance delivered as an AMI (Amazon machine image). This isn't very difficult. Maybe there could be some fiddling around with licensing where the VSA reported its usage and customers paid by capacity per month, but even that's not really necessary.

If all this happens, then I suspect we'll continue to see HP selling Data Protector for another 30 years. If Data Protector is still useful for customers post-Amazongeddon as it is pre-Amazongeddon, then there would be no particular reason that Data Protector couldn't pass through this critical tipping point. In fact, since I doubt that BackupExec will handle the transition, Data Protector will probably pick up some market share.

Anyway, what are some immediate scenarios would this support?

• Customer A has a small Amazon presence and a large data centre with a StoreOnce system and some tape drives. They would like to deploy a VSA in the same region as their Amazon servers and replicate their data through low-bandwidth links back to their data centre. 
• Customer B has a somewhat larger Amazon presence. They have Data Protector in their office, and they want to backup their Amazon content to Glacier. 
• Customer C is closing down their data centre in house and moving their servers into the cloud. They want to take backups of their servers in their data centre and use StoreOnce replication to get them into their cloud where the data is rehydrated.

So if you are customer like A, B or C, feel free to contact to your account manager, suggest that you'd really like Data Protector to support you and see how you go. (Or get in touch with me and I'll collate some answers back to the product team.)

Greg Baker is an independent consultant who happens to do a lot of work on HP DataProtector. He is the author of the only published books on HP Data Protector (http://www.ifost.org.au/books/#dp). He works with HP and HP partner companies to solve the hardest big-data problems (especially around backup). See more at IFOST's DataProtector pages at http://www.ifost.org.au/dataprotector

@16z Outside of the USA, are we also seeing the same 16 themes in the way software eats the world? Here are my Oz impressions

I was perusing Marc Andresson's website. Marc is one of the better known venture capitalists and famously wrote that software is eating the world.

On his website there are 16 themes that he is seeing. Obviously, he invests big money in big startups, where I tend to be advising smaller and less splashy groups with little or no up-front cash. And north-western Sydney isn't Palo Alto. But there is quite a lot in common:

• Sensorification of the enterprise. Yes, the insurer-broking-customer system I've been developing with the best and brightest in the industry is doing that, and that's one of the big takeaways. We haven't even begun to tap what can be done with mobile sensors. This one is universal.
• Machine learning and big data. Obviously, I spend a lot of time helping customers with big data storage (as that's the other three-quarters of my work). Backup and recovery of big data is an issue that no-one knows how to do properly (myself included). But on the machine learning side, yes, I'm seeing this everywhere. There was the project where I was extracting keywords out from emails ( http://blog.ifost.org.au/2014/05/keywords-from-emails-on-google-app.html ) and displaying a colour summary of what the author of the email thinks about those topics. The data we got from the big data analysis I did of COI group's staff attitudinal surveys was extraordinary -- did you know that there are 5 distinct different ways to be a successful organisation, and another 14 to be unsuccessful? In another we found that a flower shop should stay open an hour later.
• Containerisation is not a trend I'm seeing yet in customers with existing infrastructure. What I am seeing is customers simply not having their own data centre presence -- not even maintaining a server rack or a local server. So the containerisation I'm seeing is more around self-contained applications managed by different companies.
• Digital health. From the dental camera system that I've been developing, to the start-up I was talking to last week, software is eating up the entire medical device industry. The medtech industry (which is big in Sydney -- bigger than most people realise) is turning into a developer of sophisticated software for small cheap sensors.
• I'm not seeing the efficient online marketplaces trend as clearly. Perhaps because I'm based in Australia with a smaller less efficient market (in general) but specialty and uniqueness is still common, and I don't see any Australian companies racing to the bottom on price and staying in business.
• Bitcoin and blockchain. The retailers I'm speaking to don't see bitcoin as a priority, but they don't really have a problem with it. So it will come, and the volatility will reduce. On the other hand, if I put insurance contract binding information into the blockchain, I'm pretty sure my insurance customers would freak out and run hiding. So this trend will come from the retail side.
• Funnily enough, whether to do cloud-client computing is a pressing question that I'm researching at the moment. That is, for two projects I'm doing at the moment I know I have a lot of in-mobile image and sound analysis computation to do. Bandwidth from the mobile is a problem, but so is CPU time on the mobile. Where to do it? It's not obvious which is better.
• I'm not doing any work for Controlability any more (that stopped when I went to google), but we were well ahead of the curve on that. What we were building in Darwin and Brisbane (hardly centres of technology) are what a whole bunch of "Internet of things" startups are trying to do now. It was secure, it was sensible, it was cost-effective and it made the residential development companies money. So I'm a bit biased on this: I think the rest of the world is playing catch-up.
• I can't say much about online video, but it seems to me that the money previously being spent on face-to-face training and e-learning is there for the taking as these move to video. I still don't have any video content to sell for anything I'm doing (I'm still writing books) but I can see the market is ready for it. Australia was well ahead of this curve, as the business of doing face-to-face training went undead several years ago; from what I've seen India and China are well behind on this curve.
• The changes sweeping the insurance industry: yes: real-time data extraction from company systems to give a day-by-day risk premium. We're working on it.
• DevOps: the week before last I was talking to a very famous software company about their SaaS offering. They have a long, long way to go to get from "sysadmins of a box that runs an application" to "site reliability engineering". Maybe, just maybe, some of the other Australian SaaS players are doing it better, but I doubt it. So few companies here have enough scale that they can do any statistically meaningful analysis of their outages or incidents. So while lots of people will talk about it, it's going to be a long time before DevOps is going to make a measurable difference here, sadly.
• Failure and the culture of fail fast. I see no evidence of this anywhere here. The vast majority of start-ups are self-funded, boot-strapping and grabbed an opportunity that arose. The QUT CAUSEE study showed no correlation between the success of a start-up and the number of failed startups the principals had been involved in. The closest QUT found was a kind of bonus for pivoting: if you changed direction as a result of customer feedback directing the company to a new product or solution in that industry then that was a very good predictor of success.
• I don't have enough experience to comment on full-stack startup, virtual reality and crowdfunding
• While I used to be able to say with confidence that I was a security guru, I don't think I can comment on Andresson's theme; I'm probably losing it.

What I found interesting is what didn't appear, but are really clear themes that I'm seeing:

• Image analysis is everywhere. Every company has dozens of problems which can be solved by moderately simple image analysis techniques. There simply aren't enough knowledgeable gurus out there to do the work.
• Interfacing with the low-tech. I think there's a Silicon Valley bubble which assumes that everyone wants to be on-line with a smart-phone all the time. But there are many workers and customers that simply can't or won't do this. In the last few weeks: a startup the severely disabled on-line with assistive technology; the retail conversations were around simplifying warehousing procedures so that the tech un-savvy can cope; the edutech project getting tech-averse sport coaches to be able to put wet weather into their school's twitter, facebook and skoolbag systems.

The sports wet weather line app

It's a grey Saturday morning, not quite wet enough to be sure that sport is cancelled, and not clear enough to be sure it's on.

You dial the school's wet weather line, and it's engaged because hundreds of other parents are doing the same thing. When you finally get through, you hear that sport is on, but it may get cancelled later. As you bundle the kids into the car you toss up getting a fine for using your mobile to call the wet weather line on the way, or getting there and discovering it was cancelled half an hour ago.

Sound familiar?

It was getting to a Trinity vs Cranbrook game at 7:15 in the morning (yawn!) that made me come up with a better solution.

I've set up some servers that dial the wet weather lines repeatedly and check for any change in message. If the sportsmaster has changed the voicemail message, my servers grab the new messages as an MP3 file.

Then I wrote an app (Android version here, iphone coming soon ) which connects to my servers and polls for any changes.

Just leave this app open on a Saturday morning, and you'll see when there's a new message. You can play it with the touch of a finger, or you can get the children to do it for you. They'll be using your phone in the car anyway, of course.

So far I'm only monitoring a handful of Sydney private schools: King's (TKS), Cranbrook, Trinity Prep, Trinity Grammar and Trinity Junior school. This website: http://wet-weather-line.appspot.com/ has the latest list.

If your school has a wet weather line that you dial (even if there are digits to send), my app can get updates from it. Let me know through this form if you want this done. I'm using Twilio for my phone services, so it can monitor a phone line in any country.

I've been asked to incorporate this into one school's information app (which was easily done). 

I also figured I might as well offer a voicemail service for schools and other sports organisations. If you are coaching a team and need a convenient way of getting wet weather information out, email me (gregb@ifost.org.au) or use this contact form.

My goal: to make sure no-one ever has to get out of bed early on a cancelled-sport Saturday morning again.

Keywords from emails on Google App Engine

Today I was working on a project which needed to have a summary of comments (especially emails) sent by the staff working on it.

The software runs on Google App Engine (because I wrote it), so it was surprisingly easy to add this courtesy of AlchemyAPI.

App Engine can receive emails, that's just a matter of adding to the app.yaml configuration.

inbound_services:

- mail

- url: /_ah/mail/.+

  script: emailmgmt.app

  login: admin

Skipping most of the actual program, it was essentially this.

class EmailHandler(InboundMailHandler):

  def receive(self,message):

   (content_type,body) = message.bodies().next()

   a = alchemyapi.AlchemyAPI()

   data = a.keywords('html',body.decode(),{'sentiment':1})

   for k in data['keywords']:

      # store keywords and sentiment analysis 

   

That extracts the body out of the email, calls out to the Alchemy API, and returns the keywords from the email including the sentiment of the words around it (whether the word is positive or negative).

A bit of JavaScript had it displaying the keywords in colour. I sent the following email:

From: gregb@ifost.org.auTo: fun@app.appenginemail.comSubject: Site report 

They had some lovely stone gargoyles and some horrible fountains.

And out came some coloured summary keywords: stone gargoyles, fountains.

Greg Baker (gregb@ifost.org.au) is an independent consultant. If you are an industry leader and you need someone to catch your vision and help you make it reality, Greg might well be the right person to call.