We've written about BIG data before and while some reckon it's sexy, you better roll up your sleeves because you'll invariably need to do a lot of 'janitorial' (a.k.a. shit) work first!
Ron Sandland recently wrote about the new phenomenon of 'big data' - weighing up the benefits and concerns. Terry Speed reflected on the same issue in a talk earlier this year inGothenburg, Sweeden noting that this is nothing new to statisticians. So what's all the fuss about? Here's another take on the 'big data' bandwagon.
Professor Joe Gani was a distinguished mathematical statistician and was Chief of CSIRO's Division of Mathematics and Statistics between 1974 and 1981. In a 2008 interview he reflected upon his CSIRO years. What's extraordinary is how very little has changed with respect to CSIRO's penchant for change!
As we've long been advocating - it's time this endless, unproductive cycle of deconstruction and reconstruction ceased (or at least had a longer return period).
If you think all the current rhetoric about CSIRO needing to become more relevant, more commercially focused and 'customer' driven is new, reflect for a moment on Professor Gani's experience more than 40 years ago.
The following is an extract from a much longer interview posted on the Australian Academy of Science's web site here.
(We have emphasized pertinent comments with underlining and red)
Interviewer: How were things going at the CSIRO? Prof. Gani: I had seven years with CSIRO, between 1974 and 1981. The general consensus was that it went brilliantly...People in CSIRO still come up to me saying, 'Oh, that was the first time I met David Cox, when you brought him out,' or, 'It was the first time I heard John Nelder giving a talk,' 'It was the first time John Tukey came out and talked to us.' So the programs were very good. We did broaden our area.
Interviewer: Chris Heyde joined you early in the year? Prof. Gani: Yes. Chris had been a reader at the ANU but he joined me and became assistant chief. He was very, very good, and helped to develop all sorts of things. We had a splendid time.
But at the end of seven years, as usual in the CSIRO, there was a review. By that time, the head of the institute which contained our division - renamed as the Division of Mathematics and Statistics (DMS) - was John Philip, who was also the Chief of the Division of Environmental Mechanics. He was a difficult character, and although our relations were polite we didn't get on particularly well.
He put himself at the head of the committee of review. We thought we'd done brilliantly, and Paul Wild, who was then the head of CSIRO, agreed. But the review, when eventually it came out, was schizophrenic.
The first part said, 'This division has become internationally renowned. It has done extremely well, blah, blah, blah. It has done all that we had asked of it.' Part two said, 'Unfortunately, that's not what the government wants.What it wants is a more commercially oriented enterprise. It wants a return to the more consultative aspect of before.' We had become a very strong research as well as consultative unit. They wanted a kind of return. I have no idea to what extent the views of John Philip influenced that.
Anyhow, in essence I was told that I had to change policy. Well, I'm not a very easy person to twist, and my attitude was, 'For seven years I've gone before my troops and told them they must do research, they must bring out papers, as well as consulting. Now I'm supposed to tell them, "No, I'm sorry, we're going to go back." I'm not going to do it.'
