A blog about technology, running tech companies, data science, religion, translation technology, natural language processing, backups, p-adic linguistics, academic lecturing and many other topics.
Search This Blog
Wednesday, 2 December 2015
My part in the making of WiFi
Between 1994 and 1996 I was working at CSIRO Radiophysics (which turned into Telecommunications and Industrial Physics). Terry Percival was my boss' boss, and Diet Ostry and I shared an office. This story happened just a little bit before Terry, Diet and the two Johns had starting applying the radio signal unsmearing algorithms that CSIRO ended up with patents for which formed part of the WiFi standard.
One day Dr Percival set me (fresh-faced, obnoxious, know-it-all graduate) the challenge of solving the hardest problem in radio communications at the time: how can A and B communicate reliably, if A can't detect C's signal, and C can interfere with B?
My thoughts on the matter was that everyone was mis-stating the problem. It's only going to be a serious problem if you want to broadcast at 2.4 GhZ. If you drop the frequency of the signal down to something so low that even an iron ore mountain is transparent to it, it would be a very strange environment where A & C couldn't communicate.
So therefore, the real problem was that we were trying to do high speed networking. On the contrary, what we should be researching is extremely low-speed networking. How could we have useful and reliable communication at only a few bits per second?
Latency, jitter, high-speed CPUs to perform processing -- all these hard problems go away when you are only dealing in bits per second.
There were three other very good reasons why I thought low-speed networking was the right thing to look at, too: Linux, mining and submarines.
At the time, Linux was just making inroads into our thinking. The business world was dominated by IBM mainframes and (even in 1996) Windows 3.11 crashing was a daily experience for most people's workday.
The prevailing opinion that the team in the signal processing wing of CSIRO Radiophysics developed was that source-available (free-to-modify) software was unstoppable, and in a short time would conquer everything else, particularly Microsoft. After all, if the source was available, the program could never truly become unavailable or die, like proprietary software would. Software distribution bloat was about to go away, because we would all be getting our software in source form and compiling it. The days of elegant software that did exactly what it was supposed to without cruft were just around the corner because of the massive growth in volunteer developers who would tidy up anything and everything.
Which led me to the conclusion that we wouldn't really need high speed networks. The future was going to be everyone having these extremely reliable, high performance desktops (32-bit Linux never crashed; and the difference in this and also in performance was night and day compared to 16-bit Windows 3.11). All the software we would ever want would already be on our local harddisks -- all of it free -- and that there simply wouldn't be enough "stuff" to send over a network to even justify upgrading existing 9.6k modems. (I used to dial in on a 2.4k modem most of the time, myself).
I had been working on a related geophysics project as well. It was deployed on Linux (tying into the future-of-operating-systems theme) and deployed radio transmitters and receivers down boreholes in order to draw conclusions about the kinds of rocks in a region. It seemed like geophysical technologies were going to be a significant part of Australia's research future (at least I got something right!), and the need to deliver communications down into mines (where very low bandwidth would be inevitable) seemed like a worthwhile research direction.
The issues with the Collins class submarines at the time (including: how do we communicate with a submarine deep underwater?) made it seem to me like all the arrows were pointing at low-speed rather than high-speed communication.
I was so convinced that Terry and Diet (and John Deane, who was just down the corridor; and John O'Sullivan whom I think I interacted with a couple of times) were on the wrong track that I ended up quitting CSIRO and joining a private consultancy. This probably diverted me away from academia altogether which is where I otherwise would have gone. With the funding cuts that have hammered Australian research in the last few years, I'm kind of glad about this.
And it was fortunate for everyone else that I quit; I suspect I would have been a pain to work with if I'd stayed, and I'm sure I would have tried (probably unsuccessfully) to push the research in all the wrong directions. I suspect that I might have done such a bad job on the team that they might well have never made any progress to what we now call 802.11b Wi-Fi. On this basis, can I claim that I played a role in the creation of WiFi? By leaving and letting the team hire someone who actually had a clue what they were doing?
I'd like to say that I learn from my mistakes.
Before I got the private consultancy job, I applied for a quant-like role at County Natwest which in the end I turned down (again another lucky save given their history later). I was asked how I thought that County Natwest could make use of the Internet. My answer was that since no-one in their right mind would transfer money over the internet, that all it could be was an information portal.
A decade later (in 2007) I left Google because I was fairly convinced that it was going to fall apart in a few years as Wikipedia became ever more trustworthy that it would become everyone's first point of call for search. It's 2015 now as I search using Google over my home WiFi connection from a proprietary operating system: I have to admit that Bill Gates, Eric Schmidt and Terry Percival were right, and I was wrong.
Based on this, feel free to ignore anything in this blog that you disagree with, since it's almost definitely wrong. But I still think I'm right when I say that my my book of nerd-geek poetry has the best poems about nuclear physics you'll ever see. (And some fun stuff with robots, AI, first contact, and all sorts of other topics. There's even a vampire-at-the-blood-bank.) You really should go and buy it for yourself or your nearest and dearest nerd-geek friends. Here's the Amazon link: When Medusa went on Chatroulette.
Friday, 27 November 2015
Buying Data Protector licenses and support contracts online
Over the next few days you will see my Data Protector online store add some new products:
As it turns out, even though they are sold from store.data-protector.net, these Training Units can be used for just about anything (e.g. the new HP Records Manager (HP RM) / TRIM e-learning modules, face-to-face courses at HP, VILT classes.)
Greg Baker is an independent consultant who happens to do a lot of work on HP DataProtector. He is the author of the only published books on HP Data Protector (http://www.ifost.org.au/books/#dp). He works with HP and HP partner companies to solve the hardest big-data problems (especially around backup). See more at IFOST's DataProtector pages at http://www.ifost.org.au/dataprotector, or visit the online store for Data Protector products, licenses and renewals athttp://store.data-protector.net/
As it turns out, even though they are sold from store.data-protector.net, these Training Units can be used for just about anything (e.g. the new HP Records Manager (HP RM) / TRIM e-learning modules, face-to-face courses at HP, VILT classes.)
Greg Baker is an independent consultant who happens to do a lot of work on HP DataProtector. He is the author of the only published books on HP Data Protector (http://www.ifost.org.au/books/#dp). He works with HP and HP partner companies to solve the hardest big-data problems (especially around backup). See more at IFOST's DataProtector pages at http://www.ifost.org.au/dataprotector, or visit the online store for Data Protector products, licenses and renewals athttp://store.data-protector.net/
How to avoid getting lumped with the tickets that take forever to resolve -- Sydney Atlassian Devops Groups
For those that were (and those that weren't) at the Sydney Atlassian Devops Meeting last night, my presentation is here: http://prezi.com/ukofmue_rgf0/
Devops poetry
I gave a talk last night at the Sydney Atlassian Devops Meetup, and was inspired to write something for it. If you like this, you'll enjoy When Medusa Went on Chatroulette -- go and pre-order it today!
I've wrestled a Python or two;
I've built an app (along with a friend)
And run a VM with Haiku.
I've written Java and downloaded Go;
And Haskell, I love -- it's a joy.
Ansible, Salt and Chef, those I know:
I'll debug a Cloudfront deploy.
I've found a bug in an enterprise app,
By strace'ing through its coremem dump.
I've patched up live a signal break trap
To fix up a subroutine jump.
Storage and routers and syscalls are fine,
Also VOIP service call groups.
Mapping, reducing, data refined:
I set up the office Hadoop.
I am SRE -- I will fix anything:
Deploying it twice in an hour,
Split loads across sites based upon ping,
Google is awed at my powers!
So why am I stumped and lost more or less
With a bug that I cannot mend?
Please help if you can, because I confess
Yes, docker is crashing again.
Everyone I know in Devops
Perl is no worry, and I grok Ruby gems;I've wrestled a Python or two;
I've built an app (along with a friend)
And run a VM with Haiku.
I've written Java and downloaded Go;
And Haskell, I love -- it's a joy.
Ansible, Salt and Chef, those I know:
I'll debug a Cloudfront deploy.
I've found a bug in an enterprise app,
By strace'ing through its coremem dump.
I've patched up live a signal break trap
To fix up a subroutine jump.
Storage and routers and syscalls are fine,
Also VOIP service call groups.
Mapping, reducing, data refined:
I set up the office Hadoop.
I am SRE -- I will fix anything:
Deploying it twice in an hour,
Split loads across sites based upon ping,
Google is awed at my powers!
So why am I stumped and lost more or less
With a bug that I cannot mend?
Please help if you can, because I confess
Yes, docker is crashing again.
Wednesday, 25 November 2015
When Medusa went on Chatroulette
And now for something completely different: my poetry collection is now available for pre-order from Amazon.
They are all happy and cheery poems about technology, geekdom, nuclear physics, time travel, first contact and all the really important things in life. There are no soppy love poems, and no heart-felt tales of loss. These poems will make you smile and laugh and then ponder for a little while if you feel like it. At the very least, if you're a geek-nerd kind of person, they will lighten up your day.
It's a nice Christmas present for the geek in your family. Or a colleague. Or you can treat yourself.

Pre-order link: Amazon.
Warning: these poems are totally unsuited to liberal arts majors as they won't understand most of them. Recommended for science and sci-fi fans only.
They are all happy and cheery poems about technology, geekdom, nuclear physics, time travel, first contact and all the really important things in life. There are no soppy love poems, and no heart-felt tales of loss. These poems will make you smile and laugh and then ponder for a little while if you feel like it. At the very least, if you're a geek-nerd kind of person, they will lighten up your day.
It's a nice Christmas present for the geek in your family. Or a colleague. Or you can treat yourself.

Pre-order link: Amazon.
Warning: these poems are totally unsuited to liberal arts majors as they won't understand most of them. Recommended for science and sci-fi fans only.
Send me your HP Data Protector support contract renewal
I've made some arrangements with HPE about support contracts for Data Protector . In short, if you are about to renew your support contract -- or you think it is due soon -- send me an email (gregb@ifost.org.au) and I'll look at it.
In the past, I've often worked with customers to explain what it is that the support contract covers, identify whether there are any unnecessary line items (quite often there is) and suggest what to renew and what not to renew.
What's changed is that I can now negotiate slightly better prices on your behalf as well. So even if your contract is perfect, I should still be able to save you some money.
Greg Baker is an independent consultant who happens to do a lot of work on HP DataProtector. He is the author of the only published books on HP Data Protector (http://www.ifost.org.au/books/#dp). He works with HP and HP partner companies to solve the hardest big-data problems (especially around backup). See more at IFOST's DataProtector pages at http://www.ifost.org.au/dataprotector, or visit the online store for Data Protector products, licenses and renewals at http://store.data-protector.net/
In the past, I've often worked with customers to explain what it is that the support contract covers, identify whether there are any unnecessary line items (quite often there is) and suggest what to renew and what not to renew.
What's changed is that I can now negotiate slightly better prices on your behalf as well. So even if your contract is perfect, I should still be able to save you some money.
Greg Baker is an independent consultant who happens to do a lot of work on HP DataProtector. He is the author of the only published books on HP Data Protector (http://www.ifost.org.au/books/#dp). He works with HP and HP partner companies to solve the hardest big-data problems (especially around backup). See more at IFOST's DataProtector pages at http://www.ifost.org.au/dataprotector, or visit the online store for Data Protector products, licenses and renewals at http://store.data-protector.net/
VMware ESX 6.0 bug with CBT
VMware has announced another CBT problem. Just a reminder, this is not a problem that HPE can do anything about in Data Protector -- it's a problem with the APIs that VMware have supplied for HPE to use.
If you are doing VEAgent backups of your VMware environment (which is quite common) and you have any incrementals scheduled (also quite common), and you are running ESX 6 (which is lots of people) and you are using CBT (which you really, really would want to do normally).... then you might want to be aware that (yet again) VMware have announced that your backups could well be painfully broken.
Here's VMware's KB article:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2136854
There are several solutions:
Only do full backups. Hmm, that's a lot of data. Probably OK if you are going to a StoreOnce dedupe, but that's going to turn into a lot more tape.Turn off CBT. Ouch, that's going to hurt performance.Downgrade to ESX 5.5. I don't see anyone doing that.Using the DataProtector disk agent and automated disaster recovery module. This is actually cheaper (no extension licenses required!) and gets you both a file-level backup and an ability to restore a virtual machine from nothing. I recommend this as a better approach generally, but particularly now when we can't trust our VM-level backups.- Apply the patch that VMware has now released.
Less easy solutions, but things to think about:
- Migrate all your virtual machines to Amazon machine images. (Or Google, or Azure. Pity it can't be HP any more). It's inevitable -- eventually -- that the economies of scale of the large cloud providers will overtake your ability to run things in your own data centre. So why not start planning for it now?
- Use a different virtualisation solution. This is not the first time that VMware have announced "by the way, all backups are broken". I suspect it won't be the last time either. KVM is very mature now and it's also free. Xen is in good shape too. Virtualisation technology is no longer cutting edge -- it's commoditised now. So why not pay commodity prices?
Greg Baker is an independent consultant who happens to do a lot of work on HP DataProtector. He is the author of the only published books on HP Data Protector (http://www.ifost.org.au/books/#dp). He works with HP and HP partner companies to solve the hardest big-data problems (especially around backup). See more at IFOST's DataProtector pages at http://www.ifost.org.au/dataprotector, or visit the online store for Data Protector products, licenses and renewals athttp://store.data-protector.net/
Subscribe to:
Posts (Atom)


