It’s now almost the middle of 2016. The use of data analytics is now very widespread, and new areas for its application are continuously being thought up across a range of business functions. Apart from the increased use of advanced statistics to analyse past trends to predict or recommend future action, there is also an increased exploration of using a broader range of techniques in machine learning. As customers to various businesses in our daily lives we are now used to being bombarded with all sorts of messages that seem to have been produced by trying to analyse our own previous activity or those of a broader demographic.
Although it may seem like a lot has been done, there’s actually still a long way to go in terms of maturity of the whole ecosystem that uses large volumes and varieties of data to achieve various benefits. Just a few areas in which we can expect to see improved results are as follows.
Improvement in quality of results
By now, just about everyone who reads online content such as this post must have also done a bit of online shopping, watched a video or two, read a news clip, used email and maybe done an internet banking transaction as well. After doing all that, they may start to see ads appearing online the next time they use the internet. The subject matter of these ads may be related to the subject matter of their earlier online transacting. This is a result of analytics at work, constantly mining what people are doing, and trying to understand it so that suggestions can be made about what they might also like to consider next. While these suggestions are sometimes relevant and of interest (for example, while browsing through the news), I find that more often than not they are not. And this is because it would take much more than one variable to understand a person’s interests, or because the assumptions built into the predictive algorithms may hold good for one or two scenarios, but not all. In some situations, maybe the user needs to see something that’s not totally related. For example, if I’d just gone online and looked at shoes, I might have done any of three different things: browsed and exited without buying any, browsed and saved a pair in my shopping cart and then exited without buying, or maybe actually completed a purchase. In each case, the messages or recommendations made to me the next time I logged in could be different, but often I find that they are not, and that’s where the quality of those messages still need to be improved.
Maturity in evaluating input data quality
Big Data is often needed for better quality of analytics results. The more the data, the greater the confidence in results, provided of course that the data is of good quality. And as there is a greater variety of data available, the increased richness of data could lead to even better result quality. But as the variety, volume and velocity of input data keeps increasing (for example, when streaming from IoT devices), automation is needed to scrub the data and attempt to keep its quality within acceptable limits. This can be a challenge, and dealing with this is an area in which efforts are still being made to show improvements.
Maturity in IoT interoperability
I refer to IoT because that is expected to be one of the biggest contributors of new streams of data, enabling a whole new range of analytics possibilities. At this time, however, we’re still a bit behind the predicted curve of IoT adoption for a number of understandable reasons. Progress in terms of development of the technology and adapting it to new applications continues, and currently this area is in that state where there are a number of different technologies, but no clear leading or widely adopted standard yet for certain definitions. This is expected to mature further in the months and maybe next couple of years to come. When that happens, there’ll be more data and APIs available to more consuming applications, leading to a greater range and richness in analytics possibilities.
More intelligent predictions
I refer here to the use of intelligence in making short term predictions, whether it is in applications of the type used in driverless cars or in breaking away from predictions based purely on extrapolating from past patterns to actually considering the possibility that past patterns may signal a need for future differences. For example, if I’ve been watching a particular TV show for two days in a row maybe I’d like to watch something very different next time. Or that I might want to visit a different type of restaurant for a change, even if I said somewhere that I like a certain type of food. Or that after buying my shoes I might be persuaded to buy a matching belt, instead of more shoes. The human mind is able to work with a huge number of variables when making dynamic decisions, and technology often doesn’t seem to be there just yet. The driverless car is another example that demonstrates this. Technology in this area is quite good, but it’s not really there yet, is it? At present, its creators are working on dealing with ethical decision making in first world urban driving scenarios, among other issues, but imagine a driverless car negotiating bad weather, vague mechanical noises, and competitors on its tail during an F1 race, or the totally different conditions of a cross terrain rally, or a crowded lane in old Delhi. There’s certainly room for further maturity.
This blog is listed under Data & Information Management Community