By Hermann Wimmer
In the midst of consistent upgrades to existing technology that is seen as many to make our lives easier we often get asked if all this automation will eventually impact our view and need of whom we today call “experts”? Will the future be an era of agile generalists who can play every role, supported by data-driven decisions?
Data journalist Nate Silver, who was part of a panel discussion at Teradata Universe this April, surprised his listeners with an opening statement that was quite humble for a data journalist. He said that data is not magic and there is a considerable risk of using them inappropriately because, as he put it, “we’re all dumber than we think we are”. Predictions often fail, simply because fitting a data set is not the same as predicting future developments
For example, in the case of chronic diseases people often turn to the web not because they’re actually ill but actually because of all the media reporting on the subject. We have also seen situations where business specialists might commission data scientists to do technically sophisticated analyses and then come back with hardly surprising results like “people are more likely to buy warm boots in winter”.
Maybe we should then differentiate between self-proclaimed experts who mainly use data to justify their existing opinions, and experts who truly have expertise and use data to form their opinion and improve their decisions. The question then arises if the problem lies in the human race? Should we just step aside and let the data take all decisions? The answer again is no.
The question is whether data scientists did their homework properly in the first place: providing excellent data quality and the right analytic and predictive models. Because the old term ‘garbage in-garbage out’ remains valid and big data increases its relevance by orders of magnitude.
A good data scientist is someone who masters the art of designing experiments accurately and sticks to measurement rather than assumptions. He sees the dotcoms at the frontline of this new experimentation era with a daring attitude of simply trying everything
In a nutshell: once you have the right question, finding the right answer is easy. Well, it of course still depends on the technology you use. A good data scientist and a good journalist have one thing in common: their profession is mainly about finding the right question. And never forget that throwing a bunch of numbers at a decision-maker might not be enough to convince him. You need to be able to bring the data to life.
Hermann Wimmer is the International President of Teradata Corporation
