Following our first interviews, our column “Behind the scenes of coding” continues its journey to give voice to the driving forces behind innovation at Bitrock. While in the first article we explored the importance of simplicity and maintainability, today we delve into a realm where time is measured in milliseconds and data volumes challenge the limits of traditional architecture.
In this third instalment, we meet Carmelo Calabrò, Senior Software Engineer, to discuss predictive systems, extreme optimisation and the latest news from Kafka 4.
A story that highlights how Bitrock’s IT consulting is not just theory, but a constant search for technological limits to exceed our stakeholders’ expectations.
What is the biggest technical challenge you have faced and how did you overcome it?
The most complex technical challenge I have faced involves designing an architecture able to manage a high volume of data in near real-time and transform it into predictive information before the next batch arrives.
The context was the development of a proactive monitoring system whose goal was to generate malfunction alerts that could prevent customer reports. The most difficult constraint was time. The data arrived at a very high frequency and the entire cycle of cleaning, structuring and aggregation had to be completed in a matter of minutes. Overcoming this challenge required perfect synchronisation between every component of the system.
Which project have you been particularly proud of?
Definitely the development of a series of web applications for devices with minimal performance, but which had to guarantee a high level of performance. It was a meticulous job of extreme optimization: the application had to respond in a few milliseconds while maintaining maximum stability on limited hardware.
Although the technical success was rewarding, the real satisfaction came from the end user. Seeing people use those applications on a daily basis and noticing their satisfaction with the fluidity we had managed to achieve was the real reward. It is in those moments that you understand the real value of your work.
Which language has surprised you the most?
I have recently been working a lot with Kafka, delving into its scalability, which is fundamental for both performance and cost control. Kafka provides an excellent balance between consistency and order, but what surprised me the most is a new feature introduced with Kafka 4: Share Groups.
Traditionally, each partition is assigned to a single consumer to maintain order. With the new Share Groups, however, multiple consumers in the same group can simultaneously process different messages from the same partition. This changes everything: the number of partitions is no longer a constraint on scalability, which now depends only on the number of consumers. This is a revolution for cases where message order is not critical.
Conclusion
In this interview, we have seen how the technical challenge often turns into a mission: to make the invisible (data) a visible and useful tool for people. From real-time flow management to extreme scalability, the goal of our Bitrocker team remains the same: technical excellence at the service of business.
This concludes our “Behind the scenes of code” column, in which we have sought to offer an authentic glimpse into how we approach digital evolution as partners ready to break down complexity in order to build solid solutions.Are you ready to turn your technological challenges into concrete successes?Discover our integrated end-to-end and Agile methodology and all Bitrock services for the innovation of your business.