Some would say that these problems are not concerning everyday life, but as an employee of a data analytics company, I can assure you that these are problems which you will encounter multiple times a day during work - so for me it is definitely an everyday problem. Additionally I would like to highlight this topic because both innovations, Blockchain and data science, bear massive potential to transform the way how several industries will operate for the next decades and it really seems like as these technologies could benefit from each other.
1. What is the Pain Point?
While working in a function alongside consulting firms which integrate data analytics and AI solutions, nearly everyone would come across one of the following apparent problems in the Big Data & data analytics area:
- Data Silos - multiple teams either store the same data or they store complementary, but separate, data. Each situation causes several problems like data redundancy and non-accessibility of existing information in the right place, which clearly hinders cooperation within a firm.
- Dirty Data - In nearly every firms data base you will find duplicate or incorrect data. This can happen due to bad communication, recklessness and inattention at work, or on purpose (e.g. for whitewashing your departments numbers.)
- Privacy Issues - Data analytics tools, especially the ones that are incorporating machine learning functions, have the ability to gather, analyze, and combine vast quantities of data from different sources, thus increasing the information-gathering capabilities of actors that use this technology. So bringing this to a company level means that employers that analyze their business with one of these tool will be able to gather more and more information about the activities of their employees and thus making their activities transparent. So while implementing data analytics tools as a firm you will often face conflicts with powerful unions and legislation if you are not 100% able to ensure the anonymity of your employees data.
- Data Security - for generating the most value out of your data and ensuring smooth operations of your analytics tools it is essential that your data is stored in one place. On the other hand while doing so, you make your firm vulnerable to cyberattacks. Continuous reports about security breaches and data lacks in firms underline the importance of this topic.
- Implementation Effort of Tools - there are hundreds of tools out there for different analyses and AI applications, from various providers. As a firm, if you want to implement one of the tools and you do not have the right internal capabilities in place, you always have to hire external resources to help you throughout the implementation phase and may set up a dedicated infrastructure. This will cost time and a lot of money, e.g. consultants for change management, data scientists to find the right data within the firm to get the tool running (which especially time consuming in legacy systems) and hard & software costs. So before making your data to gold, heavy investments have to be undertaken.
2. How could these Pain Points be reduced or eliminated in a blockchain-world?
In my idealistic blockchain world I would tackle all these above mentioned problems with an open source blockchain network for company data. My vision is to create a place where companies can store all their data anonymous in one place, but with the help of blockchain and its decentralization eliminate the risk of cyberattacks and data security concerns. All other pain points can also be solved with the inherent characteristics of Blockchain: Dirty data can be reduced by the decentralized consensus algorithm and cryptography, through which Blockchain validates data, making it almost impossible to be manipulated due to the huge amount of computing power that will be required. Another important advantage for data analytics is that not only that you will have less dirty data, but more structured data, as all data is already stored in a structured way which will definitely speed up analytic efforts. Additionally, the network effect will also help smaller firms to run powerful analytics as they can make use of the combined computational power in the network. The data Silos will no longer hinder fruitful cooperation as storing all data on the blockchain network will allow all departments and teams to centrally access all information that is already stored in the firms database, which will reduce redundant data and work - while still having the possibility to grant certain access right to individuals. Security-wise, Blockchain with its special protocols allowing varying degrees of anonymity, confidentiality, and privacy which can enable the protection of personal data, may even give the employees power over the personal data attached to their internal activities that is stored in the firm´s database. This potentially will facilitate companies to ensure concerning parties that they can not track every employee individually - making it easier to deal with unions and legislation. Regarding the the enormous effort and abundance and tools, the specific characteristics of the blockchain network can reduce the costs of both parties: solution providers and buyers. Imagine a network with the all company information in one place with the ability to grant certain groups specific access right to the data. If you combine this with several APIs that allow solution providers to use the data already stored structured in the network and offer their solutions on demand to firms interested in various analyses of their data, this bear tremendous business potential. Firms do not have to care anymore about special infrastructure and data structure and just grant solution providers the information that they need for the analysis and therefore additionally eliminate the risk of data theft as third parties do not have to search for the right information through all data bases. Besides that, the need for implementation partners will become obsolete as everything can be managed between the company and the solution provider directly on the network - making the implementation much cheaper. Solution providers will benefit from the ability to provide more tailer-made analytics, can charge higher prices, as the overall investment will become much cheaper for firms and preclude the possibility of poor implementation of their software through consultancy, which make firms unable to make use of the full software potential (I think there are much more advantages, but these are the first ones that came to my mind - feel free to add ;) ). Finally, when taking a look into the future, as Blockchain also integrates well with other advanced technologies, like cloud solutions and the Internet of Things (IoT), the technology will also work perfectly within the demanding environment of Industry 4.0 and the ever-increasing connected corporations that produce unimaginable big amounts of data. Thus it will facilitate analytic performance in an increasing complex environment for the years ahead.
These are just some thoughts from a guy that deals with Blockchain since several weeks - so this is just a first thought. Feel free to give your feedback! I would love to see comments on further possibilities of such a network, or also critical voices that explain why some of my thoughts are not feasible with Blockchain technology.
Please also find some interesting articles related to the topic under the following links (which I used for inspiration):
1. Blockchain and Big Data: the match made in heavens
2. How Blockchain Will Disrupt Data Science: 5 Blockchain Use Cases in Big Data
3. Why Data Silos Are Bad For Business
4. AI and the Future of Privacy
5. How Blockchains Will Enable Privacy
Do you like the idea? #upvote
Thanks for reading!
Markus