Myth 1: Big Data is New
In 1997, there was a paper that discussed the difficulty of visualizing Big Data, and in 1999, a paper that discussed the problems of gaining insight from the numbers in Big Data. That indicates that two prominent issues today in Big Data, display and insight, had been around for awhile. “But now it’s reaching us in new ways,” because of the scale and prevalence of Big Data, Ms. Crawford said. That also means it is a widespread social phenomenon, like mobile phones were in the 1990s, that “generates a lot of comment, and then disappears into the background, as something that’s just part of life.” [Never mind 1997, Big Datasets have been datamined by Telcos - the first large social network owners - large retailers, airlines and bottom feedin credit card companies for several decades]
Myth 2: Big Data Is Objective
Over 20 million Twitter messages about Hurricane Sandy were posted last year. That may seem sufficient for a picture of whom the storm affected. However, the 16 percent of Americans on Twitter tend to be younger, more urban and more affluent than the norm. “Very few tweets came out of Breezy Point, or the Rockaways,” Ms. Crawford said. “These were very privileged urban stories.” And some people, privileged or otherwise, put information like their home addresses on Twitter in an effort to seek aid. That sensitive information is still out there, even though the threat is gone. That means that most data sets, particularly where people are concerned, need references to the context in which they were created. [And of course, Twitter is not the only source of large reams of data, the reason so many people focus on it is, like the drunk looking for his key under the street-light, it is far easier to look there]
Myth 3: Big Data Doesn’t Discriminate
“Big Data is neither color blind nor gender blind,” Ms. Crawford said. “We can see how it is used in marketing to segment people.” Facebook timelines, stripped of data like names, can still be used to determine a person’s ethnicity with 95 percent accuracy, she said. Information like sexual orientation among males is also relatively easy to identify. (Women are tougher to pinpoint.) That information can be used to determine what kind of advertisements, for example, that people receive.It’s important to remember that whenever people start creating data sets, these become fallible human tools. “Data is something we create, but it’s also something we imagine,” Ms. Crawford said. [And they are prey to all the same biasses that small dataset work is prone to - being big doesn't make it better - all data models are wrong, by definition]
Myth 4: Big Data Makes Cities Smart
“It’s only as good as the people using it,” Ms. Crawford said. Many of the sensors that track people as they manage their urban lives come from high-end smartphones, or cars with the latest GPS systems. “Devices are becoming the proxies for public needs,” she said, “but there won’t be a moment where everyone has access to the same technology.” In addition, moving cities toward digital initiatives like predictive policing, or creating systems where people are seen, whether they like it or not, can promote lots of tension between individuals and their governments. Sorry, IBM. Take that, Cisco. That goes for you, too, Microsoft, Ms. Crawford’s employer. All these big technology companies have Smart Cities initiatives. [Quite - as we have pointed out ad nauseam on this blog, freeing your data can be just another way of putting you in chains. But then, if you had nothing to hide, why wouldn't you want us to have your data, I hear you say.....]
Myth 5: Big Data Is Anonymous
A study published in Nature last March looked at 1.5 million phone records that had personally identifying information removed. It found that just four data points of when and where a call was made could identify 95 percent of individuals. “With just two, you can identify 50 percent of them,” Ms. Crawford said. “With a fingerprint, you need 12 data points to identify somebody.” Likewise, smart grids can spot when your friends come over. Search engine queries can yield health data that would be protected if it came up in a doctor’s office. [This is well known by anyone working with databases, but the Big Data fans - and open Data apostles - are strangely quiet about this issue]
Myth 6: You Can Opt Out
Last December, Instagram, the photo-sharing site, changed its terms of service to allow it to share customer’s photos more broadly, even use images in ads. What it didn’t have was a paid option, in which a person could, for a fee, not be part of that. Even if that option existed, Ms. Crawford said, this would imply a two-tier system — people who could afford to control their data and those who could not. [As we've argued before, the online world shows many worrying signs of devolving into a "feudal 2.0" system, with the digital serfs trading their data for access to crappy services relying on Ad funding, and you will only get truth by paying for it
1. Joining the social matrix
Social technologies are much more than a consumer phenomenon: they connect many organizations internally and increasingly reach outside their borders. The social matrix also extends beyond the cocreation of products and the organizational networks we examined in our 2010 article. Now it has become the environment in which more and more business is conducted. Many organizations rely on distributed problem solving, tapping the brain power of customers and experts from within and outside the company for breakthrough thinking.
[Its a transaction cost game - see our thoughts on this over here - and in very early days. It is also largely Just Another Channel, not a New Paradigm - that was so 2012]
2. Competing with ‘big data’ and advanced analytics
Three years ago, we described new opportunities to experiment with and segment consumer markets using big data. As with the social matrix, we now see data and analytics as part of a new foundation for competitiveness. Global data volumes—surging from social Web sites, sensors, smartphones, and more—are doubling faster than every two years. The power of analytics is rising while costs are falling. Data visualization, wireless communications, and cloud infrastructure are extending the power and reach of information.
[The problem with Big Data (or data analysis, as it used to be known in the Olde Days, c 2010) is that the low hanging fruit is soon plucked (if it hasn't been already - people used quaint old ideas like Statistics and Operations Research to estimate these larger datasets and optimise things in them thar Old Days). The other issue is Beautiful Mind Syndrome* where people see patterns in the data where none exist (or worse, you anchor on non patterns you want to see) and chase after will o' the wisps. Big Data is a lot like SEO - typically once you've done the fairly obvious things (that you may well have already done - see Statistics, use of, above) you can shave small incremental benefits off things by analysing them to the nines, but a crap business will still be a crap business and you're better off changing that than crunching yet more data]
3. Deploying the Internet of All Things
Tiny sensors and actuators, proliferating at astounding rates, are expected to explode in number over the next decade, potentially linking over 50 billion physical entities as costs plummet and networks become more pervasive. What we described as nascent three years ago is fast becoming ubiquitous, which gives managers unimagined possibilities to fine-tune processes and manage operations.
[We have been active in this area for 20 odd years, and the hype still outpaces the reality unfortunately. Unit costs per sensor and the cost of integration, lack of standards and reliability of these systems at any scale are still huge barriers to widespread deployment. But it is coming, one day...see our take on it over here]
4. Offering anything as a service
The buying and selling of services derived from physical products is a business-model shift that’s gaining steam. An attraction for buyers is the opportunity to replace big blocks of capital investment with more flexible and granular operating expenditures. A prominent example of this shift is the embrace of cloud-based IT services. Cosmetics maker Revlon, for example, now operates more than 500 of its IT applications in a private cloud managed by an external provider. It saved $70 million over two years, and when one data center in Venezuela was hit by a fire, the company was able to shift operations to New Jersey in two hours. Moves like this, which suggest that cloud-delivered IT can be reliable and resilient, create new possibilities for the provision of mission-critical IT through external assets and suppliers.
[The best X As A Service operators are making thin margins at best, and increasingly having to grow by acquisition (Salesforce.com), and most are still heavily subsidised by their parent companies or running on investors money. The hassle factor of running key services remotely, with the same level of reliability and flexibility as in house, is still not a slam dunk for anything but the very basic commodity services. Exits (and valuations) are a function of hype over reality still]
5. Automating knowledge work
Physical labor and transactional tasks have been widely automated over the last three decades. Now advances in data analytics, low-cost computer power, machine learning, and interfaces that “understand” humans are moving the automation frontier rapidly toward the world’s more than 200 million knowledge workers.
[This we will watch with interest, as those 200 million are the bulk of the educated elite in developed countries, they are connected and (still) wealthy, and are not going to take this development lying down. Business leaders were able to push Globalisation when it was boomtime and only low end blue collar jobs were being hit, lets see if it still works with high end white collar ones, in Great Recession hit democratic countries. Also, the automation of knowledge work has been a holy grail through at least the last 3 main tech cycles - remember Knowledge Engineering, Artificial intelligence etc - and so far has still not got very far in practice]
6. Engaging the next three billion digital citizens
As incomes rise in developing nations, their citizens are becoming wired, connected by mobile computing devices, particularly smartphones that will only increase in power and versatility. Although several emerging markets have experienced double-digit growth in Internet adoption, enormous growth potential remains: India’s digital penetration is only 10 percent and China’s is around 40 percent. Rising levels of connectivity will stimulate financial inclusion, local entrepreneurship, and enormous opportunities for business.
[The next 3 billion citizens are very uninteresting by Western advertising economics standards, but Western price Ads (or selling data to advertisers) is required to fund the Western Dotcom industry at its current cost levels. It will thus be interesting to see the business models used for the next 3 billion, because most of the current Ad based ones won't work with Western based infrastructures]
7. Charting experiences where digital meets physical
The borders of the digital and physical world have been blurring for many years as consumers learned to shop in virtual stores and to meet in virtual spaces. In those cases, the online world mirrors experiences of the physical world. Increasingly, we’re seeing an inversion as real-life activities, from shopping to factory work, become rich with digital information and as the mobile Internet and advances in natural user interfaces give the physical world digital characteristics.
[Remember 2nd Life? All this was promised, back then. It tanked, and nothing has changed that reduces the ikelihood of another failure just yet]
8. ‘Freeing’ your business model through Internet-inspired personalization and simplification
After nearly two decades of shopping, reading, watching, seeking information, and interacting on the Internet, customers expect services to be free, personalized, and easy to use without instructions. This ethos presents a challenge for business, since customers expect instant results, as well as superb and transparent customer service, for all interactions—from Web sites to brick-and-mortar stores. Fail to deliver, and competitors’ offerings are only an app download away.
[Today, most of your competitors are still no better than you, finding the right App is a nightmare, it's not a given it'll work on your device, and there is a lot of stuff we still won't buy online - see below. But this is probably one of the areas that is tightening up faster than some of the others]
9. Buying and selling as digital commerce leaps ahead
The rise of the mobile Internet and the evolution of core technologies that cut costs and vastly simplify the process of completing transactions online are reducing barriers to entry across a wide swath of economic activity. Amped-up technology platforms are enabling peer-to-peer commerce to replace activities traditionally carried out by companies and giving birth to new kinds of payment systems and monetization models.
[Except that there is a large swathe of things we don't want to buy without physically examining them first, and a larger swathe we won't buy if we don't trust the seller, service, payment process etc. We are happy in the main with buying low cost, low risk commodities online. For everything else, we still like face to face communication. Its changing, but slowly, and in fact we are starting to see online retailers look at bricks and mortar stores to increase their penetration. Very few payment systems are "new", most have been thought about for 20-odd years (just look at all the dotcom initiatives), typically the blockers have been technology costs, vested interests or lack of user takeup]
10. Transforming government, health care, and education
The private sector has a big stake in the successful transformation of government, health care, and education, which together account for a third of global GDP. They have lagged behind in productivity growth at least in part because they have been slow to adopt Web-based platforms, big-data analytics, and other IT innovations. Technology-enabled productivity growth could help reduce the cost burden while improving the quality of services and outcomes, as well as boosting long-term global-growth prospects.
[I've been doing Internet strategy for nigh on 20 years, this area is always just about to take off in a big way. However data privacy, the problem of getting the last 20% to use new services to realise the financial benefits and related worries about democratic representation, the huge discrepancy between who pays and who benefits from public money, and expensive past failures are all big barriers to major adoption. We see no compelling short term drivers for major changes. In the medium term the cost of providing for the elderly, and public services, will force changes.].