THE DEMISE OF THE EXPERT?
Big Data & it’s Analytics is changing the world. The biggest impact certainly is that data driven decisions as it stands, can augment or overrule human judgement. Ian Ayers in his book Super Crunchers says that statistical analyses forces to reconsider human instincts. This becomes even more essential through big data. Undeniably, the subject area expert, the substantive specialist would lose some of their sheen to statistician & data analyst, quantum & pace unknown.
We are already seeing the waning influence in many areas. Websites like Huffington Post, Gawker and Forbes’s content is primarily decided by data, not just the judgment of human editors. What people want to read is more clearly revealed by the data, than the judgement & gut feeling of a seasoned journalist. Coursera, the market leader in online education through videos, uses data on replay of videos to determine what parts are unclear. This information is fed back to the teachers to improve the content. Jeff Bezos did away work in-house book reviewers at Amazon after realising that the algorithms recommendations drove more sales.
The skills needed to make a change are changing. Dr. McGregor, caring for premature babies in Ontario is not a medical physician at all, she is PhD in Computer Science. She has data more than a decade of patient-years, the computer crunches the data & she offers the recommendations for treatment. Mostly, pioneers of big data come from outside the domain. A British physicist developed an algorithm to insurance claims & identify defective cars.
It’s a churning. The exact contours of technology is yet to emerge. Subject matter expert would certainly not die out. But the iconic supremacy would get diluted. They will have to share the podium with big-data geeks. The princely causation of times immemorial is now forced to share the limelight with humble correlation.
A NEW RELATIONSHIP BETWEEN SUBJECT MATTER EXPERTS & BIG DATA GEEKS WOULD EVENTUALLY EMERGE.