These last couple of years have taught an important lesson to all Data & Analytics specialists: agility is the key. Being able to pivot between different design patterns and approaches is increasingly important to thrive through supply chain volatility, accelerated digitalization, and disruption of business operations.
To turn these challenges into opportunities, and stay ahead of competition, companies must revise antiquate models based on centralized, static data. The centrifugal shift towards distributed architectures and multi-cloud infrastructures, emerged a few years ago, has today found its cultural equivalent in new, decentralized approaches to Data & Analytics. At the same time, the possibility to analyze data in motion in a dynamic manner allows to integrate actionable insights into decision making and business operations.
Let’s take a look at some of the most interesting Data & Analytics trends that have emerged or consolidated recently, and how these can create value for organizations in the next future.
Small & Wide Data
We have come to realize that Big Data is not always the answer. Accumulating information can lead to data sourcing and quality issues, plus requiring the implementation of deep learning analytical techniques whose cost and complexity may outweigh the results. We have also seen how quickly data can become irrelevant – companies run the risk of hoarding stale, useless information that cannot provide significant value.
Small & Wide Data have emerged as innovative approaches to enable the generation of valuable insights via less voluminous, more varied data. The former approach eschews data-hungry models in favor of tailored analytical techniques relying on limited amounts of data. The latter leverages the integration of heterogeneous sources, both structured and unstructured, instead of a larger single one.
Small & Wide Data can enable the access to advanced analytics and AI for smaller players, which cannot rely on enough information for conventional Big Data techniques. But bigger companies can also benefit from these approaches. As Gartner suggests, 70% of organizations will shift their focus from big to Small and Wide data by 2025.
Data Mesh
The current shifts towards decentralization and microservices can be said to underpin the very notion of Data Mesh. First introduced by Zhamak Dehghani in her 2019 article “How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh”, it purports to overcome the limitations of gargantuan Data Lakes and their reliance on hyper-specialized teams and financially-untenable ETL pipelines.
By contrast, Data Mesh can be seen as an organizational and architectural model that allows for a distributed, domain-driven data ownership. This ubiquitous data platform empowers cross-functional teams to operate independently, while offering greater flexibility and interaction between distributed datasets.
It is worth noting that the distributed Data Mesh architecture stems from a paradigm shift, rather than a Copernican revolution. It does not ditch altogether data lake advantages and principles – centralization is in fact retained for governance and open standards – but evolves them to increase business agility and reduce time-to-market.
Continuous Intelligence
Continuous Intelligence leverages Event Stream Processing and Real-time Analytics to integrate actionable insights into decision-making and business processes. This design pattern turns analytics into a prescriptive practice: ingest large volumes of data in motion – from sources like IoT, eCommerce transactions, traffic, weather – and leverage them to augment or even automate human decisions.
CI enables companies to analyze data on the fly, identify trends and root causes, and make real-time decisions that allow strategic differentiation in competitive, saturated markets. It is a transformative model that provides a plethora of opportunities – from detecting fraud in finance and improving customer experience in retail to implementing predictive maintenance in manufacturing and more. CI can be also employed to connect different branches and departments in a company, to share & leverage data in real time, optimize decision-making and thus increase productivity.
Thanks to its partnership with Radicalbit, Bitrock can integrate its D&A consulting services with Helicon, a cutting-edge Continuous Intelligence platform. This code-free SaaS solution enhances the management of data streaming pipelines with Machine Learning, dramatically accelerating the development of real time advanced analytics (descriptive, diagnostic, predictive and prescriptive). The platform offers efficient features such as a stream processing pipelines visual editor, with debugging capabilities, data exploration, and real-time ML monitoring, enabling the adoption of the Continuous Intelligence paradigm.
Analytics at the Edge
Speaking of IoT, a recent D&A trend concerns the decentralization of the very location in which the collection and analysis of data takes place. Edge Analytics means distributing information, analytics and technology closer to – or possibly within – the physical assets, i.e. the edge. In other words, it entails the possibility of avoiding in part or altogether the transfer to data centers and cloud environments, increasing the flexibility of the whole data infrastructure.
It is a growing trend – Gartner foresees that, by 2023, more than 50% “primary responsibility of data and analytics leaders will comprise data created, managed and analyzed in edge environments”. The reasons are multiple: for instance, provisioning analytics to the edge can have a positive impact on the speed in which data is processed, with actionable insights being generated in real-time. Stability is another case in point: avoiding data transfer means less disruption from connectivity issues. Finally, we have to consider compliance – leaving data “where it is” reduces the headaches deriving from different national regulations and governance policies.
For these reasons, Analytics at the Edge can bring significant benefits to a wide array of applications. Automotive risk mitigation is, for instance, a business case in which analyzing data in real time is fundamental to avoid collisions or breakdowns. Healthcare, on the other hand, can simplify the management of personal, sensitive data if this is not moved to cloud services or data centers located under different jurisdictions.
Data Democracies
The notion of Data Democracy concerns the creation of an ethical and methodological framework that removes the technological barriers to informed data management. It revolves around the principle that people, regardless of their technical know-how, should be able to access and trust available information during their daily operations.
The democratization of data impacts any kind of business organization, and bears upon both personnel and technology. Lowering the barrier to data means first of all offering upskilling programs aimed at data literacy development, whatever the function or seniority within the company. It also means rethinking data silos in favor of more flexible and transparent architectural models, such as Data Meshes (see above). Finally, it entails implementing efficient analytics and Business Intelligence tools on a company-wide level. One example is Sense, by our partner Qlik, that enables advanced, ML-powered analytics while helping develop data literacy.
As a real cultural shift, a Data Democracy can offer significant benefits to a company’s internal operations. It empowers non-technical employees to make fast, informed decisions without the support of IT or data experts – think of how this can help Product, Marketing, Sales team generate more value and save resources. Moreover, developing a corporate data culture may have a positive impact on an organization’s relationship with its stakeholder and the public at large. Data ethics informs data governance policies that promote privacy, cybersecurity, and a righteous management of customer data.
These are only some of the opportunities offered by the latest trends in Data & Analytics. If you want to know how Bitrock can help your company evolve its data strategy, and stay ahead of competition with cutting-edge solutions, send us a message – we’ll be happy to book a call!
Author: Daniele Croci, Digital Marketing Specialist @ Bitrock
