With our new Big Data, Small Talks column, dedicated to some members of our Data Engineering team, we would like to reveal some behind-the-scenes insights into the complex world of data and put a face to the professionals who drive digital innovation for our customers.
In this series of interviews, we will get to know some of our Bitrockers better, revealing their passion, expertise and the methodological approach they take to tackle the most challenging technological projects.
Our first Ambassador is Nicola Manca, Junior Data Engineer. Let’s get to know him better.
How do you collaborate with other teams to ensure that data is used effectively and consistently?
Collaboration always starts with a clear definition of the data: what arrives, in what format, with what quality guarantees. We use shared documentation and establish alignment moments when necessary. The main objective is to prevent ambiguity and misunderstandings about the data from leading to a bottleneck and slowing down downstream developments.
Which emerging technology in the world of data do you think will have the greatest impact in the near future?
Data Mesh and Data Fabric are both steps towards an increasingly data-driven ecosystem, in which data becomes an accessible, reliable resource that is useful for business decisions.
Data Mesh is an organisational model that decentralises data ownership across various business domains, promoting collaboration and shared responsibility. This approach certainly has significant advantages in terms of data quality and mutual trust, so I believe it will have a major impact on data management. Data Fabric, on the other hand, is an architecture that unifies and connects data distributed across different systems without moving it, simplifying governance, security and access in complex contexts with large volumes of data.
These two approaches can also be combined, using Data Fabric as the technical infrastructure and Data Mesh as the governance model, allowing complex organisations in particular to benefit enormously from the advantages of both.
What is Bitrock’s Data Engineering team’s approach to integrating AI models into customer projects?
Every project and every client obviously has specific characteristics that must be considered when defining an operational strategy. However, throughout my experience, I have learned a series of best practices that allow me to approach the work in an organised and optimised manner.
We always start with a business need and assess whether it can be solved with AI. Then we assess data readiness, i.e. whether the data at our disposal is suitable for training a Machine Learning model. We perform an exploratory analysis of the data to understand what information it contains and how it can best be used.
Next, we need to build useful features for the problem at hand from the data and use them to train models, including using different ML architectures, to evaluate the preliminary results. If the results are promising, we then proceed to a more intensive exploratory phase, in which we seek the best possible approach for training. From here on, we begin to create the MLOps pipeline of training, testing, deployment and monitoring, which will then also be used in production.
What is the most valuable non-technical skill for a Data Engineer according to you?
I think it’s communication. It is often necessary to translate a technical concept into a message that is understandable and useful to those who do not have specific data skills, and this is not always easy. The impact of good communication of insights on the quality and effectiveness of the resulting business decisions should not be underestimated.
How is curiosity and experimentation encouraged within your team?
We organise internal discussion spaces on new ideas, tools, or experimental projects. When a team member would like to give a presentation on any topic that may be of interest to every one, we set aside a slot to get together, listen, and discuss. In addition, resources such as articles, seminars, and much more are often shared in the team channel to stay up to date on topics of interest.
We would like to thank Nicola once again for accompanying us on this first dive into the world of Data Engineering, which has shown us how data management is now at the heart of every innovation project, requiring not only technical expertise but also a deep capacity for collaboration and communication.
The passion and experience that our professionals bring to the table every day guarantee an approach that is not only technologically advanced but also aligned with the real business needs of our clients.
Do you have a complex digital transformation project or simply want to understand how Big Data can become your most accessible and strategic resource? Contact us.
