Bridging the Gap Between Machine Learning Development and Production

In the field of Artificial Intelligence, Machine Learning has emerged as a transformative force, empowering businesses to unlock the power of data and make informed decisions. However, bridging the gap between developing ML models and deploying them into production environments can pose a significant challenge. 

In this interview with Alessandro Conflitti (Head of Data Science at Radicalbit , a Bitrock sister company), we explore the world of MLOps, delving into its significance, challenges, and strategies for successful implementation. 

Let's dive right into the first question.

What is MLOps and why is it important?

MLOps is an acronym for «Machine Learning Operations». It describes a set of practices ranging from data ingestion, development of a machine learning model (ML model), its deployment into production and its continuous monitoring. 

In fact, developing a good machine learning model is just the first step in an AI solution. Imagine for instance that you have an extremely good model but you receive thousands of inference requests (input to be predicted by the model) per second: if your underlying infrastructure does not scale very well, your model is going to immediately break down, or anyway will be too slow for your needs.

Or, imagine that your model requires very powerful and expensive infrastructures, e.g. several top notch GPUs: without a careful analysis and a good strategy you might end up losing money on your model, because the infrastructural costs are higher than your return.

This is where MLOps comes into the equation, in that it integrates an ML model organically into a business environment.

Another example: very often raw data must be pre-processed before being sent into the model and likewise the output of an ML model must be post-processed before being used. In this case, you can put in place an MLOps data pipeline which takes care of all these transformations.

One last remark: a very hot topic today is model monitoring. ML models must be maintained at all times, since they degrade over time, e.g. because of drift (which roughly speaking happens when the training data are no longer representative of the data sent for inference). Having a good monitoring system which analyses data integrity (i.e. that data sent as input to the model is not corrupted) and model performance (i.e. that the model predictions are not degrading and still trustworthy) is therefore paramount.

What can be the different components of an MLOps solution?

An MLOps solution may include different components depending on the specific needs and requirements of the project. A common setup may include, in order:

  • Data engineering: As a first step, you have to collect, prepare and store data; this includes tasks such as data ingestion, data cleaning and a first exploratory data analysis.
  • Model development: This is where you build and train your Machine Learning model. It includes tasks such as feature engineering, encoding, choosing the right metrics, choosing the model’s architecture selection, training the model, and hyperparameter tuning.
  • Experiment tracking: This can be seen as a part of model development, but I like to highlight it separately because if you keep track of all experiments you can refer to them later, for instance if you need to tweak the model in the future, or if you need to build similar projects using the same or similar datasets. More specifically you keep track of how different models behave (e.g. Lasso vs Ridge regression, or XGBoost vs CatBoost), but also of hyperparameter configurations, model artefacts, and other results.
  • Model deployment: in this step you put your ML model into production, i.e. you make it available for users who can then send inputs to the model and get back predictions. What this looks like can vary widely, from something as simple as a Flask or FastAPI to much more complicated solutions.
  • Infrastructure management: with a deployed model you need to manage the associated infrastructure, taking care of scalability, both vertically and horizontally, i.e. being sure that the model can smoothly handle high–volume and high–velocity data. A popular solution is using Kubernetes, but by no means is it the only one.
  • Model monitoring: Once all previous steps are working fine you need to monitor that your ML model is performing as expected: this means on the one hand logging all errors, and on the other hand it also means tracking its performance and detecting drift.

What are some common challenges when implementing MLOps?

Because MLOps is a complex endeavour, it comes with many potential challenges, but here I would like to focus on aspects related to Data. After all that is one of the most important things; as Sherlock Holmes would say: «Data! data! data! (...) I can't make bricks without clay.»

For several reasons, it is not trivial to have a good enough dataset for developing, training and testing a ML model. For example, it might not be large enough, it might not have enough variety (e.g. think of a classification problem with very unbalanced, underrepresented classes), or it might not have enough quality (very dirty data, from different sources, with different data format and type, plenty of missing values or inconsistent values, e.g. {“gender”: “male”, “pregnant”: True}).

Another issue with Data is having the right to access it. For confidentiality or legal (e.g. GDPR) reasons, it might not be possible to move data out of a company server, or out of a specific country (e.g. financial information that cannot be exported) and this limits the kinds of technology or infrastructures that can be used, and deployment on cloud can be hindered (or outright forbidden). In other cases only a very small curated subset of data can be accessed by humans and all other data are machine–readable only.

What is a tool or technology that you consider to be very interesting for MLOps but might not be completely widespread yet?

This might be the Data Scientist in me talking, but I would say a Feature Store. You surely know about Feature Engineering, which is the process of extracting new features or information from raw data: for instance, having a date, e.g. May 5th, 1821, compute and add the corresponding week day, Saturday. This might be useful if you are trying to predict the electricity consumption of a factory, since often they are closed on Sundays and holidays. Therefore, when working on a Machine Learning model, one takes raw data and transforms it into curated data, with new information/features and organised in the right way. A feature store is a tool that allows you to save and store all these features.

In this way, when you want to develop new versions of your ML model, or a different ML model using the same data sources, or when different teams are working on different projects with the same data sources, you can ensure data consistency. 

Moreover, preprocessing of raw data is automated and reproducible: for example anyone working on the project can retrieve curated data (output of feature engineering) computed on a specific date (e.g. average of the last 30 days related to the date of computation) and be sure the result is consistent.

Before we wrap up, do you have any tips or tricks of the trade to share?

I would mention three things that I find helpful in my experience. 

My first suggestion is to use a standardised structure for all your Data Science projects. This makes collaboration easier when several people are working on the same project and also when new people are added to an existing project. It also helps with consistency, clarity and reproducibility. From this perspective I like using Cookiecutter Data Science.

Another suggestion is using MLflow (or a similar tool) for packaging your ML model. This makes your model readily available through APIs and easy to share. And finally I would recommend having a robust CI/CD (Continuous Integration and Continuous Delivery) in place. In this way, once you push your model artefacts to production the model is immediately live and available. And you can look at your model running smoothly and be happy about a job well done.


Main Author: Dr. Alessandro Conflitti, PhD in Mathematics at University of Rome Tor Vergata & Head of Data Science @ Radicalbit (a Bitrock sister company).

Interviewed by Luigi Cerrato, Senior Software Engineer @ Bitrock

Read More

On September 29, Fortitude Group held its annual convention, bringing together colleagues from different places for a day of fun and teamwork. The highlight of the event was the Fortigames, a two-hour challenge featuring soccer, volleyball, table tennis and even board games. To make the event even more exciting, a custom Fortitude Convention Companion App was developed, focused mostly on the games, but also convenient for helping attendees navigate the event schedule and locations.

The Fortitude Convention Companion App

The Fortitude Convention Companion App, developed as a Progressive Web App (PWA), was a marvel of teamwork and creativity. 

The Front-End team behind it (a special thanks goes to our colleagues: Stefano Bruno - Head of UX/UI, Mobile and Front-End Engineering; Gabriella Moro - Lead UX Designer and Daniel Zotti - Senior Front-End Developer) worked tirelessly to create a user-friendly and visually appealing interface. Despite the challenges and tight deadline, the team developed a successful app that made the Fortigames even more interactive.

The app was developed with a combination of three powerful technologies and development practices. 

Qwik, a modern and innovative framework, was chosen for its exceptional performance and developer-friendly experience. Its key advantages include:

  • Resumability: This Qwik's unique feature allows components to be resumed and rendered efficiently, leading to faster page load times and a smoother user experience.
  • Hydration: Qwik's hydration process is optimized for performance, ensuring that only the necessary components are hydrated and rendered, reducing the initial load time and improving overall performance.
  • Developer Experience: Qwik offers a delightful developer experience with its intuitive API, excellent tooling and strong community support.

Supabase, an open-source Firebase alternative, was selected for its complete backend solution, including a real-time database, authentication, and storage. Its key advantages include:

  • Real-time Database: Supabase's real-time database enables live data synchronization between the app and the backend, ensuring that users always have access to the latest data.
  • Authentication: Supabase offers built-in authentication features, including social login, email verification, and role-based access control, simplifying user management.
  • Storage: Supabase provides secure and scalable storage for images, videos, and other files, making it easy to store and manage application data.

Vercel, a high-performance cloud platform, was chosen for its seamless deployment and powerful features for scaling and optimizing applications. Its key advantages include:

  • Seamless Deployment: Vercel integrates seamlessly with GitHub and other Git providers, enabling easy deployment of applications directly from the code repository.
  • Global Edge Servers: Vercel's global edge servers ensure that applications are delivered with low latency and high performance to users around the world.
  • Scalability and Optimization: Vercel provides powerful features for scaling applications to handle increasing traffic and optimizing performance for various devices and network environments.

The app's development process also involved a combination of tools and techniques to ensure a streamlined and efficient workflow. Balsamiq was utilized for wireframing, providing a visual representation of the app's layout and user interface. Figma, the UI design tool, was employed to create detailed and interactive prototypes, allowing the team to refine the app's visual elements and user experience. Font Awesome, a comprehensive icon library, was incorporated to provide a consistent and visually appealing set of icons for the app's interface. To enhance development efficiency, an existing SCSS structure was adopted, leveraging its predefined styles and maintainability. Additionally, CSS modules were employed to encapsulate CSS rules within specific components, ensuring code isolation and preventing unintended style overrides. For version control and collaboration, GitHub was utilized to store and manage the app's open-source code, enabling the team to track changes, collaborate effectively, and maintain a consistent codebase.

Results

The result was a polished and user-friendly app that was essential to the success of the Fortigames. The Tigers (Yin) and the Dragons (Yang) teams battled fiercely on the soccer field, volleyball court, and table tennis table, with the Dragons winning in the end. The app played a vital role in bringing teams together and in making the competition even more fun.

The Fortigames app was a game-changer, showing the potential of technology to enhance teamwork, creativity, and fun. Thanks to the dedication and effort of the team behind it, the app provided a seamless platform for participants to connect, track results, and celebrate successes. On top of that, the app helped in making the Bitrock & Fortitude Convention 2023 a truly memorable experience.

Want to learn more about how the Fortigames app was built? Check out our colleague Daniel Zotti's complete article and the GitHub repo.


Main Author:  Daniel Zotti, Senior Frontend Developer @ Bitrock

Read More
Agile Software Development

How can it benefit your company?

In today's business landscape, change is a constant element. Companies that want to keep up with the times and stay competitive, have to adopt more agile approaches to software development. Agile methodologies are a set of principles and practices that enable quick adaptation to customer demands, market changes, and technological advancements. This adaptability allows businesses to seize new opportunities and effectively mitigate risks, ensuring personal growth and success.

What is the main purpose?

Agile methodology is a development approach based on principles such as flexibility, collaboration, and continuous value delivery.  The main objective of Agile methodology is to quickly and effectively respond to changes in project requirements and customer needs. It focuses on efficiently creating high-quality products with an emphasis on customer satisfaction.  It is an alternative to traditional Waterfall development models, which complete project phases sequentially and with little flexibility.
Agile is not defined by a series of specific development ceremonies or techniques. Instead, it's a group of methodologies that focus on tight feedback loops and continuous improvement.  When discussing Agile, it is essential to mention the Manifesto for Agile Software Development which emphasizes its key principles. The first one is to prioritize people, their individuality, and their interactions, rather than focusing solely on processes and tools.

Scrum: an example of Agile Methodology

One of the most well-known examples of Agile methodology is Scrum. Scrum is a framework that organizes work into defined time units called "sprints," usually taking 2-4 weeks. During a sprint, the team collaborates on a specific set of tasks known as the Sprint Backlog. In Scrum, there are three key roles: the Product Owner, the Development Team, and the Scrum Master. These roles work together to ensure the successful implementation of the Scrum framework and the delivery of high-quality products.

  • The Product Owner represents the needs of the customers and the business
  • The Development Team is responsible for doing the work
  • The Scrum Master supports the Scrum process by removing obstacles and promoting collaboration

What are the real benefits for customers?

Adopting an Agile methodology offers numerous advantages to customers seeking successful project completion and more effective satisfaction of their needs. Here are some of the key benefits that customers can expect from Agile:

Faster Delivery

Agile development enables faster delivery of products and services, promoting a quicker time-to-market and maximizing product lifecycle. Customers can see a positive impact on new features or improvements much earlier compared to traditional development models that may take months or years to deliver a complete product.

Active Involvement

Agile promotes active customer involvement in the development process. Customers are encouraged to participate in reviews, provide constant feedback, and engage in project planning sessions. This ongoing and direct involvement ensures that the developed product aligns with customer expectations, increasing overall satisfaction and minimizing the risk of creating an unused product.  By putting customers first and delivering exceptional experiences, companies can foster long-term loyalty and trust.

Flexibility and Adaptability

Requirements may change and customers may have new ideas during development. Agile allows quick adaptation to such changes without significant disruptions to the process.  Being able to anticipate and meet the changing needs of customers is a competitive advantage in today's ever-changing business landscape.

Continuous Improvement

Quality is at the heart of Agile, with a focus on constant testing and validation. Developers work closely with customers to ensure the product meets the required quality standards. Constant testing and validation drive improvement, enable experimentation with new ideas, and facilitate the implementation of changes throughout the software development cycle, ensuring a high-quality final product.

Risk Reduction

Agile development uses iterative, feedback-driven processes that help catch risks sooner and ensure that the final product meets customer requirements. This reduces the time and cost of fixing errors and results in fewer, cheaper mistakes.

Agile

Is Agile always the answer?

As observed up to this point, implementing agile methodology in an organization can bring numerous benefits, such as increased flexibility, improved communication, and faster delivery of products or services. However, there are also some situations in which adopting this methodology may not be the most appropriate choice and there are potential drawbacks and challenges to consider. Among others, we can mention:

Ineffective Planning and Execution

Ineffective planning and execution can lead to delays and inefficiencies.  Having a clear understanding of project goals, timelines, and milestones is essential for effective planning and execution, as it helps team members stay focused, prioritize tasks, and track progress towards the overall project success.

Unready Organizational Culture

When a team is resistant to change, it can have various negative impacts on the implementation of new processes, technologies, or strategies, which can ultimately lead to delays in project completion. Fostering collaboration, open communication, flexibility and providing team members with the freedom to explore new ideas are essential strategies for cultivating a culture of continuous improvement.

Lack of Stakeholder Involvement

When stakeholders are not fully engaged, conflicts and delays in project execution can happen. Educating stakeholders on the benefits of Agile and involving them in the early stages of the project can help address challenges related to their engagement.

Unclear Goals and Priorities 

Agile methodologies, while effective in enabling teams to adapt to changing circumstances and emerging requirements efficiently, can also expose them to risks. One of these risks is the potential for goals and priorities to become unclear, which can lead to confusion and miscommunication within the team, ultimately impeding progress and hindering the team's ability to deliver value effectively.

Insufficient Resources

When the team is faced with a lack of resources, they may become overloaded with work, which can negatively impact their efficiency. It’s essential to ensure that all the required resources, including team members, appropriate tools and a supportive organizational culture are provided in order to avoid burnout and enhance productivity .

Individuals and interactions over processes and tools

Transitioning to an Agile methodology involves a shift in how we approach projects and tasks.  We redefine our way of working and move away from traditional, sequential methods embracing a more iterative and collaborative approach that allows us to deliver value more quickly and effectively.  As we mentioned in the beginning, the first value in the Agile Manifesto  is “Individuals and interactions over processes and tools”.  In other words: people drive the development process and respond to business needs. By promoting a culture of collaboration, continuous improvement, and adaptability, we’re able to empower each individual to contribute with their unique talents and perspectives. By applying Agile principles and practices, companies can create an environment where innovation thrives, and customer needs are met.

 Working in an Agile team is not just a destination, but a mindset that keeps us responsive, adaptable, and focused on delivering value.


Main Author: Cristian Bertuccioli, Scrum Master Certified and Senior Software Engineer @ Bitrock

Read More
Solid.js

Reactivity has become increasingly popular in recent months, with many frameworks and libraries incorporating it into their core or being heavily influenced by it. Vue.js has reactivity at its core, while idiomatic Angular has adopted RxJS, and MobX has become popular among React developers as an alternative to the common Redux pattern. Reactivity has been one of the leading inspirations behind the original philosophy of React.

However, most libraries still use the VDOM or use some sort of batching process in the background to replicate responsive behavior, even when reactivity is first class.

Solid.js is a reactive library that prioritizes deep, granular reactivity and is designed to offer excellent performance and responsiveness without relying on the VDOM or batching processes. While Solid.js offers a developer experience similar to React, it requires a different approach to component reasoning. Let’s take a close look.

SolidJS in three words: reactive, versatile and powerful

Upon initial inspection, Solid.js code may appear to be similar to React code with a unique syntax. However, the API provided by Solid.js is more versatile and powerful, as it is not limited by the VDOM and instead relies on deep, granular reactivity. In Solid.js, components are primarily used to organize and group code, and there is no concept of rendering. We can explore a simple example:

Our React component is going to re-render endlessly. We render our interface, only to make it immediately stale by calling setValue. Every local state mutation in React triggers a re-rendering as components produce the nodes of VDOM. The process of re-rendering the interface is complex and resource-intensive, even with the use of the VDOM and React's internal optimizations. While React and Vue.js have implemented techniques to avoid unnecessary work, there are still many complex operations happening in the background.

Solid.js updates the value and that's it; once the component is mounted, there is no need to run it again. Unlike React, Solid.js will not call the component again. Solid.js doesn’t even care for createSignal to be in the same scope of the component:

In Solid.js, components are referred to as “vanishing” because they are only used to organize the interface into reusable blocks and do not serve any other purpose beyond mounting and dismounting. 

Solid.js provides more flexibility than React when it comes to managing component state. Unlike React, Solid.js does not require adherence to the “rules of hooks”, and instead allows developers to reason around scopes of modules and functions to determine which components access which states. This fine granularity means that only the element displaying the signal’s value needs to be updated; all the operations needed to maintain a VDOM are unnecessary.

Solid.js uses proxies to hide subscriptions within the function that displays the value. This allows elements consuming the signals to become the contexts that are actively called again. In contrast to React, Solid.js functions are similar to constructors that return a render method (like the JSX skeleton), while React functions are more like the render method itself.

Dealing with props

In Solid JS, getters are more than just a value, so to maintain reactivity, props need to be handled in a special way. Using the function deriveProps retains reactivity, while spreading the parameter object breaks it.  This process is more complex than using the spread and rest operators in React.

Note that we aren’t using the parenthesis to call for a getter in the case below:

We can also access the value directly.

 Although the process may seem familiar, the underlying mechanism is completely different. React re-renders the child components when the props change, which can cause a lot of work in the background to reconcile the new virtual DOM with the old. Vue.js  avoids this problem by doing simple comparisons of props, similar to wrapping a functional component inside React’s memo method. Solid.js propagates down the hierarchy of the signals, and only the elements that consume the signal are run again.

Side effects

Side effects are a common concept in functional programming that occur when a function relies on or modifies something outside its parameters. Examples of side effects include subscribing to events, calling APIs, and performing expensive computations that involve external state. In Solid.js, effects are similar to elements and subscribe to reactive values. The use of a getter simplifies the syntax compared to React.

In React, the useEffect hook is used to handle side effects. When using useEffect, a function that performs the work is passed as an argument, along with an optional array of dependencies that might change. React does a shallow comparison of the values in the array and runs the effect again if any of them change.

When using React, it can be frustrating to pass all values as props or states to avoid issues with the shallow comparison that React does. Passing an object is not a good solution because it may reference an anonymous object that is different at each render, causing the effect to run again. Solutions to this problem involve declaring multiple objects or being more literal, which adds complexity.

 In Solid.js, effects run on any signal mutation. The reference to the signal is also the dependence.

Just like React, the effect will be run again when the values change without declaring an array of dependencies or any comparison in the background. This saves time and work, while avoiding bugs related to dependencies. However, it is still possible to create an infinite loop by mutating the signal that the effect is subscribed to, so it should be avoided.

createEffect is to be thought of as the Solid.js equivalent of subscribing to observables in RxJS, in which we’re listening to all “consumed” observables - our signals - at the same time.React users may be familiar with how useEffect replaces componentDidMount, componentWillUnmount, and componentDidUpdate. Solid.js provides dedicated hooks for handling components: onMount and onCleanup. These hooks run whenever the component returns first or gets booted off the DOM, respectively. Their purpose in Solid.js is more explicit than using useEffect in React.

Handling slice of applications

In complex applications, using useState and useEffect hooks may not be enough. Passing down many variables between components, calling methods down deep, and keeping various elements in sync with each other can be challenging.  The shopping cart, language selector, user login, and themes are just a few examples of the many applications that require some sort of slice of state.

In React, there are various techniques available to handle complex applications. One approach is to use a context to allow descendant components to access a shared state. However, to avoid unnecessary re-renders, it is important to memoize and select the specific data needed. React provides native methods like useReducer and memoization techniques such as useMemo or wrapping components in  React.memo to optimize rendering and avoid unnecessary re-renders. 

Alternatively, many developers opt to define their Redux store and each of the slices. As Redux has evolved, modern Redux has become much easier to work with compared to its early days. Developers now have the option to use hooks and constructor functions, which handle concerns in a declarative manner. This eliminates the need to define constants, action creators, and other related elements in separate files for each slice.

Solid.js provides support for various state management libraries, and offers several  methods to implement different patterns. One useful method is the ability to wrap requests using resources.

Unlike React state hooks that hook into the virtual DOM, Solid.js signals are independent units that allow developers to write idiomatic JavaScript. This enables scoping signals in other modules and restricting access through methods, effectively turning signals into private singletons.

Hence, modules can act as slices of state, exporting public methods to interact with the data without the use of any external library. By declaring signals in a module scope, they can expose publicly available interfaces to shared state in all components. If signals were declared in components instead, they would be scoped to the function context, similar to the behavior of useState in React.

Furthermore, in Solid.js, API calls can be easily handled using the createResource method. This method allows developers to fetch data from an API and check the request status in a standardized manner. This function is similar to the createSignal method in Solid.js, which creates a signal that tracks a single value and can change over time, and the popular useQuery library for React.

While it may work to handle signals as different getters, at some point, it will be necessary to deal with complex, deep objects, mutating values at different levels, accessing granular slices, and in general operating over objects and arrays. The solid-js/store module provides a set of utilities for creating a store, which is a tree of signals to be accessed and mutated individually in a fully reactive manner. This is an alternative to stores in other libraries such as Redux or Pinia in Vue.js.

To set data in a Solid.js store, we can use the set method, which is similar to signals. The set method has two modes: we can pass an object that will be merged with the existing state, or pass a number of arguments that will explore our store down to the property or object that will be mutated.

For instance, let’s suppose that we have the store shown below:

We can set the user’s age to 35 by passing an object with the properties we want to update, along with a path that specifies where in the state tree to apply the update:

This will update the age property of the user object in the state tree. Furthermore, we can update the store object by passing an object that will be merged into the current one.

If we were to omit the user attribute as first parameter, we would replace the user object entirely:

Since the store is a tree of signals, which is itself a proxy, we can access the values directly using the dot syntax. Mutating a single value will cause the element to render again, just like subscribing to a signal value.

Store utilities

We have two useful methods to update our state. If we’re used to mutating a Redux store using the immer library, we can mutate the values in place using a similar syntax with the produce method:

The produce method returns a draft version of the original object, which is a new object, and any changes made to the draft object are tracked similarly to using immer. We can also pass a reconcile function call to setState. This is particularly useful when we want to match elements in the array based on a unique identifier, rather than simply overriding the entire array. For instance, we can update a specific object based on its id property by passing a reconcile function that matches the object with the same id:

This will update the object in the array with the same id, or add it to the end of the array if no matching object is found.

We can group multiple state updates together into a single transaction using the transaction utility method. This can be useful when we need to make multiple updates to the state atomically, such as when updating multiple properties of an object:

This will update the name and age properties of the user object in a single transaction, ensuring that any subscribers to the state will only receive a single notification of the change, rather than one for each update.

RxJS interoperability

We can easily work with both SolidJs and RxJS, another popular reactive library by using a couple of adapter functions. The reduce method we just talked about is shown as an example for subscriptions, similar to how services are handled in Angular.

From RxJS into Solid.js

We can turn any producer that exposes a subscribe method into a signal:

This directly handles subscription and cleaning up when the signal is dropped. We can define our signal by passing a function to track the value, and how to clean up. The set method emits the value to the contexts that are listening.

Turning our signals into observables

We can turn our signal into an Observable that exposes a subscription method, allowing it to act like a native RxJS observable.

Next, by utilizing the form method provided by  RxJS, we can transform our signal into a fully-fledged RxJS observable.

A Solid.js choice

Although it is relatively new, Solid.js has gained popularity among developers due to its unique features and exceptional performance. Compared to React, Solid.js provides useful tools out of the box and is as performant as frameworks like Svelte without the need for compilers. It is particularly suited for interfaces that require many updates to the DOM and is consistently fast even in complex applications handling real-time updates.

Solid.js offers a developer experience similar to React, but with cleaner methods and more choices.The library handles many different patterns and provides more transparency in code thanks to how scopes work natively in JavaScript. Unlike using hooks in React, there are no hidden behaviors when creating signals in Solid.js.

Using Solid.js with TypeScript solves many of the struggles developers face with complex applications made with React or Vue.js, reducing the time to market and time spent debugging issues with the VDOM. We would recommend it for any new project starting today.

Author: Federico Muzzo, Senior Front End Developer @ Bitrock

Read More

As a Software Architect, managing a software architecture that evolves to adapt to the business over the years can be a complex task. At the beginning of its life, the software architecture was scaling low and could be managed by one Back-End and one Front-End team as well as it consisted only of three layers: Single Page Application, Application Server and Database.

As the number of users and services increases, the need for a highly scalable architecture arises. At the same time, the old architecture evolves into Microservices which makes our application scalable in terms of development. In fact, each Microservices is assigned to different Back-End teams in charge of managing it with the advantage to choose and adopt autonomous releases and technologies. At this stage, the architecture is ready to add new functionalities while for the Front-End team it is difficult to manage it saving time and work. 

That’s when Microfrontends come into play. But what are exactly MFEs?

MFEs are technical and logical representation of a business subdomain in Domain-Driven Design (DDD) which allows for independent development and release of application parts. This approach avoids shared logic between subdomains and enables management by a single team.

Very important concept for applying this approach is that our architecture is domain-driven and not vice versa, so our MFEs will represent a precise subdomain. Each subdomain will be developed by a single team that will be free in choosing the right technology and methodology to implement and manage the functionality. To ensure good management of teams' work, it is important to design an architecture with very low coupling between Microfrontends (MFEs). This approach allows for independent development and release experiences.

But how can we apply the architecture and MFE? It is crucial to consider the specific context in which you are working. As always, the business domain will play a significant role in guiding the architectural decisions. The key problems that need to be addressed will be the following: Definition; Composition and Routing; Communication.

Definition

There are two techniques to define and to effectively implement the Microfrontends (MFE) architecture: Horizontal Splitting and Vertical Splitting. Let's explore each technique:

  • In Horizontal splitting, multiple MFEs are accommodated within a single view of the application. For example, a home page may consist of MFEs such as footer, header, article preview and article list. The main challenge with this approach is ensuring that teams can create a cohesive look and feel based on a common system design. However, there is a risk of creating strongly coupled components, which may deviate from the business-defined MFEs.
  • In Vertical Splitting, the application is divided into logical units based on Business subdomains. For example, there may be separate MFEs for the login page, home page, and catalog page, each designed as if it were a separate Single Page Application (SPA). This technique allows teams to have in-depth knowledge about a subdomain and facilitates the implementation of system design.

Composition and routing

Composition and routing are critical aspects of implementing the Microfrontends (MFE) architecture. There are three techniques that can be used to compose MFEs and handle routing:

  • Client-Side Composition: The app shell is responsible for merging MFEs and handling routing between them. The composition can be done at runtime or build time.
  • CDN-Side Composition: The app will be composed on the node closest to the user who requested it by using an edge-side markup language. This results in greater scalability and shorter wait time. Routing is handled at this level via urls to go from page to page.
  • Origin-Side: The app is composed using server-side rendering, with routing always handled server-side.

Communication

In the lifecycle of an MFE, there comes a point where communication is necessary. For example, an MFE may need to communicate that a user is logged in or have other MFEs perform actions. The most common communication techniques are: 

  • Data Sharing via Local Storage (such as a JWT token)
  • Parameter Passing via URL Query String
  • Parameter Passing via Events (subscribe, emit and replay mechanism)

It is important to choose the appropriate communication technique based on the specific needs and requirements of your application.

What is a Module Federation?

Module Federation is a powerful plugin introduced by Webpack in version 5.0, released in 2020. It enables the separate development of remote modules, allowing for the creation of a single application with ease. This plugin facilitates the implementation of an architecture based on the Microfrontends approach.

Let's imagine that our target business is an e-commerce site. After careful analysis, we identify three entities that can be designated as MFEs: the product list, the shopping cart, and potentially other components. The responsibility of implementing these MFEs is assigned to separate teams. Once the development of the MFEs is completed, the teams configure Webpack to make the remote modules available. 

productsTeam/webpack.config.js

cartTeam/webpack.config.js

Basically, for exported modules the Module Federation plugin exposes three properties:

  • Name which describes the name of the module.
  • Filename by default, it is named remoteEntry.js. This file acts as a manifest and provides all the necessary directions to import the remote module, including the name of the module, aliases, dependencies, and references to bundles. It is generated when the remote module is compiled, and those who use it can import it using its URL.
  • Exposes or a property that allows the paths to be made explicit via path to reach the functionality exposed by the module.

Let's now see how the team in charge of the app-shell will import the modules to merge them into one app. Firstly, we need to clarify what an app-shell is. Basically, it can be defined as a container of remote modules where part of the implementation of the MFE basics takes place

In client-side composition, the app-shell has the responsibility of not only composing the app but also providing the tools that allow the MFE to communicate and manage routing. With the proper implementation of the modules, but especially of the app-shell, we can have an architecture that is closest to the theoretical ideal of MFEs, which is low coupling, separation, and decentralization of modules managed by separate teams

app-shell/webpack.config.js

The configuration of the Module Federation plugin for our app-shell has some differences. Since the app-shell's function is to import, rather than export, we use the "remotes" property for this purpose. This property defines where we can find the remoteEntry.js file that remote modules expose. The structure is straightforward: we use the name of the remote module as the key (e.g., "products" or "cart") and then indicate the module name with the "@" symbol followed by the URL. This way, our app-shell bundle will contain all the necessary information to merge our modules and create our app.

Conclusions

In conclusion, the Module Federation plugin in Webpack 5 has revolutionized the software architecture by allowing dynamic code loading and sharing of dependencies between different applications. By leveraging this plugin and properly configuring the app-shell, we can achieve an architecture that closely aligns with the theoretical ideal of MFEs, with low coupling, separation, and decentralization of modules managed by separate teams.

​​It's important to note that this approach is relatively new. The community is actively involved and new frameworks are being created to simplify the implementation process. Other bundlers, such as Vite, are also providing plugins to developers with some differences compared to Webpack.

It is crucial to reiterate that this approach should only be taken if it aligns with the domain's requirements. If the potential problems outweigh the benefits, it may not be the right choice. Currently, big tech companies like Dazn, Facebook, and Zalando are utilizing this approach, keeping in mind the basic principles of MFEs and applying unique solutions to ensure optimal user experience and high scalability.

Author: Marco Bartiromo, Frontend Developer @Bitrock

Read More

The recent hype surrounding Apache Flink especially after the Kafka Summit 2023 in London sparked our curiosity and prompted us to better understand the reasons for such enthusiasm.  Specifically, we wanted to know how much Flink differs from Kafka Streams, the learning curve, and the use cases where these technologies can be applied. Both solutions offer powerful tools for processing data in real-time, but they have significant differences in terms of purpose and features.

Processing Data: Apache Flink vs Kafka Streams

Apache Flink is an open-source, unified stream and batch data processing framework. It  is a distributed computing system that can process large amounts of data in real-time with fault tolerance and scalability.

On the other hand, Kafka Streams is a specific library built into Apache Kafka that provides a framework for building different applications and microservices that process data in real-time.

Kafka Streams provides a programming model that allows developers to define transformation operations over data streams using DSL-like functional APIs. This model is based on two types of APIs: the DSL API and the Processor API.  The DSL API is built on top of the Processor API and is recommended especially for beginners. The Processor API is meant for advanced applications development and involves the employment of low-level Kafka capabilities. 

Kafka Streams was created to provide a native option for processing streaming data without the need for external frameworks or libraries. A Kafka Streams job is essentially a standalone application that can be orchestrated at the user’s discretion. 

Main Differences

As previously mentioned, Flink is a running engine on which processing jobs run, while Kafka Streams is a Java library that enables client applications to run streaming jobs without the need for extra distributed systems besides a running Kafka cluster. This implies that if users want to leverage Flink for stream processing, they will need to work with two systems.

In addition, both Apache Flink and Kafka Streams offer high-level APIs (Flink DataStream APIs, Kafka Streams DSL) as well as advanced APIs for more complex implementations, such as the Kafka Streams Processor APIs.

Now, let's take a closer look at the main differences between Apache Kafka and Flink.

  1. Integrations

How do these systems establish connections with the external world? Apache Flink offers native integration with a wide range of technologies, including Hadoop, RDBMS, Elasticsearch, Hive, and more. This integration is made possible through the utilization of the Flink Connectors suite, where these connectors function as sources within Flink pipelines.

Kafka Streams is tightly integrated with Kafka for processing streaming data. The Kafka ecosystem provides Kafka Connect, which allows for the integration of external data sources as events are journaled into topics. For example, using the Kafka Connect Debezium connector, users can stream  Change Data Capture stream events into a Kafka topic. A Kafka Stream topology can then consume this topic and apply processing logic to meet specific business requirements.

  1. Scalability

 Apache Flink is an engine designed to scale out across a cluster of machines, and its scalability is only bound by the cluster definition. On the other hand, while it is possible to scale Kafka Streams applications out horizontally, the potential scalability is limited to the maximum number of partitions owned by the source topics. 

  1. Fault tolerance and reliability

 Both Kafka Streams and  Apache Flink ensure high availability and fault tolerance, but they employ different approaches. Kafka Stream delegates to the capabilities of Kafka brokers. Apache Flink depends  on external systems for persistent state management by using tiered storage and it relies on systems like Zookeeper or Kubernetes for achieving high availability.

  1. Operation

 Kafka Stream as a library, requires users to write their applications and operate them as they would normally. For example, a Kubernetes deployment can be used for this purpose and by adding Horizontal Pod Autoscaling (HPA) , it can enable horizontal scale-out.

 Apache Flink is an engine that needs to be orchestrated in order to enable Flink workloads. Currently, Flink users can leverage a Kubernetes Flink operator developed by the community to integrate Flink executions natively over Kubernetes clusters.

  1. Windowing 

Both Kafka Stream and Flink support windowing (tumbling, sliding, session) with some differences:

  • Kafka Stream manages windowing based on event time and processing time.
  • Apache Flink manages flexible windowing based on event time, processing time, and ingestion time.

Use Cases

While both frameworks offer unique features and benefits, they have different strengths when it comes to specific use cases. 

Apache Flink is the go-to choice for:

  • Real-Time Data Processing: real-time event analysis, performance monitoring, anomaly detection, and IoT sensor data processing.
  • Complex Event Processing: pattern recognition, aggregation, and related event processing, such as detecting sequences of events or managing time windows.
  • Batch Data Processing: report generation and archives data processing.
  • Machine Learning on Streaming Data: train and apply machine learning models on streaming data, enabling real-time processing of machine learning outcomes and predictions.

Kafka Stream is the go-to choice for:

  • Microservices Architectures: particularly leveraged for the implementations of event-driven patterns like event sourcing or CQRS.
  • Kafka Input and Output Data Processing: transform, filter, aggregate or enrich input data and produce output data in real-time.
  • Log Data Processing: analyze website access logs, monitor service performance, or detect significant events from system logs.
  • Real-time Analytics: data aggregation, real-time reporting, and triggering event-based actions
  • Machine Learning:  train and apply machine learning models on streaming data for real-time scoring.

Learning curve and resources

The learning curve of a technology is not always an objective fact and can vary depending on various factors. However, we have attempted to provide a general overview based on factors like resources availability and examples.

The basic concepts of Kafka Streams, such as KStreams (data streams) and KTables (data tables), can be easily grasped. While the mastery of advanced functions such as the aggregation of time windows or the processing of correlated events, may require further exploration. The official Kafka Streams documentation, available on Kafka’s website at https://kafka.apache.org/documentation/streams/ serves as a valuable reference for learning, exploring and leveraging all its capabilities.

To get started with Apache Flink, it is recommended to learn the basic programming model, including working with data streams and data sets. Once again, mastering advanced concepts such as state management, time windows, or grouping, may require additional study and practice time. The official Flink documentation available on Confluent's website at https://nightlies.apache.org/flink/flink-docs-stable/ serves as a comprehensive resource for learning and exploring as well.

Conclusions

To cut a long story short, Apache Flink and Kafka Streams are two open-source frameworks with their strengths and weaknesses for stream processing that can process large amounts of data in real-time. 

Apache Flink is a fully-stateful framework that can store the state of the data during processing, making it ideal for applications that require complex calculations or data consistency. Kafka Streams is a partially-stateful framework and it is ideal for applications that require low latency or to process large amounts of data. Apache Flink is a more generalized framework that can be used for various applications, including log processing, real-time data processing, and data analytics. Kafka Streams is more specific to stream processing.

We can conclude affirming that the best framework for a specific application will depend on the specific needs of the application.

Author: Luigi Cerrato, Software Engineer @ Bitrock

Thanks to the technical team of our sister company Radicalbit for their valuable contributions to this article.

Read More
Apache Flink

Nowadays organizations are facing the challenge to process massive amounts of data. Traditional batch processing systems don't meet the modern data analytics requirements anymore.  And that’s where Apache Flink comes into play.

Apache Flink is an open-source stream processing framework that provides powerful capabilities for processing and analyzing data streams in real-time. Among its key strengths, we can mention:

  • Elastic scalability to handle large-scale workloads 
  • Language flexibility  to provide API for Java, Python and SQL
  • Unified processing to perform streaming, batch and analytics computations 

Apache Flink can rely on a supportive and active community as well as offers seamless integration with Kafka, making it a versatile solution for various use cases. It provides comprehensive support for a wide range of scenarios, including streaming, batching, data normalization, pattern recognition, payment verification, clickstream analysis, log aggregation, and frequency analysis. Additionally, Flink is highly scalable and can efficiently handle workloads of any size.

During the Kafka Summit 2023, Apache Flink received significant attention, highlighting its increasing popularity and relevance. To further demonstrate the growing interest in this technology, Confluent, a leading company in the Kafka ecosystem, presented a roadmap outlining upcoming Flink-powered features for Confluent Cloud:

  • SQL Service (public preview fall 2023)
  • SQL (general availability winter 2023)
  • Java and Python API (2024)

Unleashing the power of Apache Flink: the perfect partner for Kafka

Stream processing is a data processing paradigm that involves continuously analyzing events from one or multiple data sources. It focuses on processing data in motion, as opposed to batch processing. 

Stream processing can be categorized into two types: stateless and stateful. Stateless processing involves filtering or transforming individual messages, while stateful processing involves operations like aggregation or sliding windows.

Managing state in distributed streaming computations is a complex task. However, Apache Flink aims to simplify this challenge by offering stateful processing capabilities for building streaming applications. Apache Flink provides APIs, advanced operators, and low-level control for distributed states. It is designed to be scalable, even for complex streaming JOIN queries.

The scalability and flexibility of Flink's engine are crucial in providing a powerful stream processing framework for handling big data workloads.

Furthermore, Apache Flink offers additional features and capabilities:

  • Unified Streaming and Batch APIs
  • Transactions Across Kafka and Flink
  • Machine Learning with Kafka, Flink, and Python
  • Standard SQL support

Unified Streaming and Batch APIs

Apache Flink's DataStream API combines both batch and streaming capabilities by offering support for various runtime execution modes and by doing so providing a unified programming model. When using the SQL/Table API, the execution mode is automatically determined based on the characteristics of the data sources. If all events are bound, the batch execution mode is utilized. On the other hand, if at least one event is unbounded, the streaming execution mode is employed. This flexibility allows Apache Flink to seamlessly adapt to different data processing scenarios.

Transactions Across Kafka and Flink

Apache Kafka and Apache Flink are widely deployed in robust and essential architectures. The concept of exactly-once semantics (EOS) ensure that stream processing applications  can process data through Kafka without loss or duplication. Many companies have already  adopted EOS in production  using Kafka Streams.  The advantage is that EOS can also be leveraged when combining Kafka and Flink, thanks to Flink's Kafka connector API.   This capability is particularly valuable when using Flink for transactional workloads. This feature is mature and battle-tested in production, however operating separate clusters for transactional workloads can still be challenging and sometimes cloud services with similar aimings can take over this burden in favor of simplicity.

Machine Learning with Kafka, Flink, and Python

The combination of data streaming and machine learning offers a powerful solution to efficiently deploy analytical models for real-time scoring, regardless of the scale of the operation. PyFlink, a Python API for Apache Flink, allows you to build scalable batch and streaming workloads, such as real-time data processing pipelines, large-scale exploratory data analysis, Machine Learning (ML) pipelines, and ETL processes.

Standard SQL Support

Structured Query Language (SQL) is a domain-specific language used for managing data in a relational database management system (RDBMS). However, SQL is not limited to RDBMS and is also adopted by various streaming platforms and technologies. Apache Flink  offers comprehensive support for ANSI SQL, encompassing the Data Definition Language (DDL), Data Manipulation Language (DML), and Query Language.
This is advantageous because SQL is already widely used by different professionals including developers, architects, and business analysts, in their daily work.
The SQL integration  is facilitated through the Flink SQL Gateway, which is a component of the Flink framework  and allows other applications to interact with a Flink cluster through a REST API.   This opens up the possibility of integrating Flink SQL with traditional business intelligence tools.

Additionally, Flink provides a Table API that complements the SQL capabilities by offering declarative features to imperative-like jobs. This means that users can seamlessly combine the DataStream APIs and Table APIs as shown in the following example:

 DataStream APIs and Table APIs

Flink Runtime

As mentioned, Flink supports streaming and batch processing. It’s now time to delve into the analysis of the main difference:

Streaming works with bounded or unbounded streams. In other words:

  • Entire pipeline must always be running  
  • Input must be processed as it arrives   
  • Results are reported as they become ready  
  • Failure recovery resumes from a recent snapshot    
  • Flink can guarantee effectively exactly-once results when properly set up with Kafka

Batch works only with a bounded stream. In other words:

  • Execution proceeds in stages and running as needed
  • Input may be pre-sorted by time and key
  • Results are reported at the end of the job
  • Failure recovery does a reset and full restart 
  • Effectively exactly-once guarantees are more straightforward

Flink Runtime

Conclusions

In a nutshell, Apache Flink offers numerous benefits for stream processing and real-time data analytics. Its seamless integration with Kafka allows for efficient data ingestion and processing. Flink's stateful processing capabilities simplify the management of distributed computations, making it easier to build complex streaming applications. The scalability and flexibility of Flink's engine enable it to handle large-scale workloads with ease. 

Additionally, Flink provides unified streaming and batch APIs, enabling developers to work with both types of data processing paradigms seamlessly. With the support of machine learning and standard SQL, Flink empowers users to perform advanced analytics and leverage familiar query languages. Overall, Apache Flink is a powerful and versatile platform that enables organizations to unlock the full potential of their real-time data.

Author: Luigi Cerrato, Software Engineer @ Bitrock

Thanks to the technical team of our sister's company Radicalbit for their valuable contributions to this article.

Read More
AI

Artificial Intelligence (AI) is not just a buzzword; it's a transformative technology that is reshaping our reality. From work to social interactions, AI is making its mark.

In this blog post, we focus on the impact that AI is having, and will continue to have on the software development industry. We will try to answer to the crucial question: will AI replace the role of programmers entirely?

The State of the Art

To date, AIs already perform a wide range of tasks: from writing code, to conducting research in technical documentation, from revealing vulnerabilities to automating tasks from a simple prompt.

We will now mention a few of them, so that you better understand the scope of these tools. 

With GitHub Copilot it is possible to speed up the simplest and most repetitive tasks or the research phase in technical documentation with results that speak for themselves: code writing time halved, ability to perform 35% of tasks autonomously and an overall satisfaction rating of about 70%.

It turns out to be a tool - for now - that is integrated with human activity and does not aim at replacement, but upcoming evolutions could go in other directions.

Zapier, a task manager that allows to automate more or less complex tasks starting from a simple prompt, introduces a fundamental topic: using AI to automate web services that already exist through APIs, could allow even code novices to streamline, modify and customize existing Applications.
Finally, we would like to mention one last tool that is experiencing widespread adoption today: Snyk from Deepcode, an AI that can analyze an existing code and detect the presence of any vulnerabilities, thus preventing cyber attacks or malfunctions.

Risks and Opportunities

However, the hype around AI tools and OpenAI in particular seems to be overblown.

They are certainly useful tools in the design phase both in terms of organizing a project and structuring the code, but the implementation of the artifacts is something that needs to be perfected and needs even meticulous and constant review.

The greatest difficulties are encountered in giving the right input to tools: a requirement that is too narrow or too broad, without contextualizing the problem, leads to the generation of code that is difficult to reuse. 

Thinking about the AI applications in the enterprise, some of the main advantages and risks associated with the massive use of them in the programming sector become evident, as they  emerge in any other area.

A junior figure - to be considered as one with limited work experience in time -  if not properly directed by a colleague with greater seniority, can incur different risks using AI for code generation.

Settling for the first solution processed by the tool can lead to poor results or broken systems: this causes loss of confidence in the tool itself, leading to a vicious cycle of copy/paste to other sites such as Stackoverflow.

In case the developer does not have enough skills to understand the quality of the solution offered, correct or not, it prevents them from learning and growing professionally.

These and other kinds of problems are avoidable with the support of a work team that, by necessity, cannot be composed of juniors alone, and with the development of soft skills such as the ability to work in a team and continuous training.

A developer with medium/high seniority, on the other hand, can take advantage of the tools in different contexts and for different tasks. For example, he or she can benefit from AI tools to request documentation, solve common problems across projects (e.g., ORM, date management), perform code reviews with automated code production, create a test scenario, and verify that a block of code meets all the requirements of a given story.

All of this, of course, is possible by starting with a proper prompt and providing for review and a series of incremental improvements.
Indeed, it is essential to always check the quality of the solution produced in terms of conventions, nonobsolescence, and versatility of the code.

Conclusions

The importance of a well-built team, solid core competencies, an agile working methodology based on transparency and cooperation, and constant and open code reviews are still necessary for the success of any project.

So while the use of Artificial Intelligence tools can bring significant benefits, it is important to consider them as supporting tools and not as substitutes for human expertise and technical knowledge

It is essential to critically evaluate the results obtained, apply good development practices, and subject the generated code to human review to ensure correctness and quality.

Although some fear that in the coming years AI could completely replace developers, we are instead certain that the human figure will remain necessary. The new challenges that will emerge, especially in ethical and regulatory terms, will bring about the need for new task areas of competence for human beings.

In conclusion, AI is a powerful tool and the responsibility lies with us to ensure that we use it ethically and responsibly.

Author: Luigi Cerrato, Software Engineer @ Bitrock

Read More
Platform Engineering

Platform Engineering is one of the biggest trends that is undoubtedly here to stay, with a community and tool environment that are both expanding quickly. But there are still many unsolved questions, as with any (relatively) new trend. 

In this Blog post, we’ll try to understand the concept from scratch and answer some basic questions that both people within and outside the community are curious about.

Without further ado, here’s the essential information you should be aware of regarding Platform Engineering. Agreeing on a common definition is rather difficult; however, we can define Platform Engineering as the process of creating, developing, and maintaining the supporting infrastructure and apparatus necessary to the efficient operation of software applications.

In order to facilitate the development, deployment, and management of apps across various environments, Platform Engineering focuses on building scalable, reliable, and efficient platforms. 

Software Engineering, Infrastructure Engineering, and DevOps are all combined in Platform Engineering. Its main objective, indeed, is to provide faster software application and service delivery while also enhancing agility and flexibility in response to changing corporate needs. 

Or, in other words, to standardize processes in order to enhance the developer experience, quicken innovation cycles, and reduce the engineering organization's time to market. 

Platform Engineering allows development teams to build upon a strong foundation, which enables businesses to innovate more quickly and stay one step ahead of the competition.

The Platform team can gain a holistic understanding of developers' pain points and common difficulties inside the company by conducting user research, requesting user feedback, and getting internal buy-in from stakeholders. It can identify the features programmers require and provide a golden path that includes those answers. 

However, the Platform journey continues on. In order to ensure that developers are using the Platform and that it is actually improving developers' life, successful Platform teams maintain open lines of contact with developers and track technical KPIs.

DevOps vs. Platform Engineering

DevOps and Platform Engineering are closely related topics.

DevOps brings development and operations teams closer together, and focuses on using tools, systems and iterative processes to shorten feedback cycles. It represents a software lifecycle management’s philosophical and methodological approach that contributes to the creation of the Platform and is, in turn, one of the services delivered by it. 

Platform Engineering, on the other hand, can basically be considered as the discipline focused on the technology infrastructures and platforms’ design, development and management, on which digital services and software applications are delivered. Through the Platform, IT teams can access self-service mode for the digital assets they need, and digital business services are delivered.

Platform Engineering is also a development of DevOps, so to speak. DevOps outlines guidelines for simplifying development through automation, autonomy, and collaboration. These qualities are also crucial to Platform Engineering so the technique helps you achieve good DevOps performance.

DevOps is used as a methodology to compose Platform Engineering; one of the methodologies that, those who use Platform, find out-of-the-box. 

An Internal Corporate Service

Platform Engineering is definitely an internal corporate service that is addressed to all IT figures involved in creating digital initiatives within an organization.

Platform Engineering, in fact, can be seen as a technological-organizational moment that gives development teams access to services like monitoring and security in a "self-service" mode, which makes use of automations and the above-mentioned DevOps. 

It is a kind of large distributor of digital services within the organization, essential to any new initiative that needs access to digital assets.

Externally, however, Platform Engineering is realized in the tangible result of the application of the service built through the Platform itself. 

Impacts on IT Governance

The centralization of digital services within the Platform has significant impacts. 

First, the Platform enables considerable cost management and control, especially when it integrates elements usually external to typical corporate governance, as in the case of data centers and cloud services.

Another relevant effect resulting from Platform Engineering is the harmonization within a company's organizational set-up of all the offerings that arrive from suppliers in terms of products and services. These have to adapt to organizational security standards and methodologies for updating and maintaining infrastructure and applications. 

Therefore, IT Governance has to integrate into the Platform’s context, also embedding itself in a self-service delivered system that represents the enterprise technology asset’s backbone.

We can affirm that Platform Engineering can be seen as the answer to every CTO's dreams!

Conclusions

The Platform Engineering community started in 2021 with a handful of meetup groups in the USA (Austin) and Europe (Berlin). Over 10,000 Platform developers are now engaged, spread over 19 meetup groups worldwide. In light of this community movement, Platform Engineering should be taken more seriously by organizations.

And you? What are you waiting for? Discover more about Platform Engineering by listening to the latest episode of our Bitrock Tech Radio podcast, or get in contact with one of our experienced engineers and consultants!

Author: Franco Geraci, Head Of Engineering @ Bitrock

Read More

Hello, coding enthusiasts!

We will journey through the landscapes of two of the most famous programming languages in the software development world: Java and Scala. These languages are like two sides of the same coin in the JVM (Java Virtual Machine) world.

Let's dive in and explore these two captivating languages together!

Java: The Time-Tested Titan

Java, the brainchild of Sun Microsystems (now owned by Oracle), has been a pillar in the programming world for over two decades. Known for its "write once, run anywhere" philosophy , Java is everywhere – from enterprise servers to Android apps. Java's robustness, simplicity, and cross-platform capabilities have cemented its place as a reliable choice for developers.

Java is an object-oriented programming language that leans heavily on explicitness and clarity. It's like your reliable old friend – always there when you need it, and you know exactly what to expect. Developers who like control and predictability might find Java a soothing balm for their programming souls.

The extensive libraries and frameworks available for Java are one of its biggest strengths. Spring, Hibernate, Maven, and many more make Java development a breeze. Moreover, its fantastic community support and extensive documentation make troubleshooting Java relatively straightforward – there's always a fellow developer ready to lend a hand!

Scala: The Functional-Object Fusion

Scala, which stands for "Scalable Language" was introduced by Martin Odersky in 2004. It blends the best of both worlds: object-oriented and functional programming. Scala, the modern artist of the programming world, is designed to express common programming patterns in a concise, elegant, and type-safe way.

With Scala, you can write less code and accomplish more, thanks to its high-level abstractions. It's like a sleek, modern sports car – compact, efficient, and oh-so-stylish.

The Comparison: Why it's a Win-Win

Now, let's pit these two languages against each other. But remember, it's not about declaring a winner but understanding how each language can be the right tool for the right job.

Performance

Java and Scala run on the JVM, so their performance is quite similar. However, Scala's advanced features, like immutability and concurrency, might offer an edge in developing high-performance systems. But remember, performance depends more on the coder's skill than the language itself!

Syntax

Java's syntax is explicit and lengthy, and it values clarity over brevity, which makes Java code easy to read and understand. On the other hand, Scala's syntax is concise and expressive. It may take a little time to get used to, but once you do, you might fall in love with its elegance!

Paradigm

Java is a stalwart of object-oriented programming, but with the introduction of lambdas and streams in Java 8, it has embraced functional programming concepts. Scala, conversely, beautifully merges object-oriented and functional programming paradigms. So if you're looking to explore available programming without letting go of the familiar object-oriented structure, Scala could be your ideal companion.

Too long; didn't read

In a nutshell, let's break down the comparison of Java and Scala into 20 bullet points highlighting their pros and cons:

Java

Pros

  • Proven Stability: Java has been around for over two decades, and its stability is time-tested.
  • Wide Adoption: Used by millions of developers worldwide, resulting in a vast community and extensive resources.
  • Clear and Explicit Syntax: The protracted nature of Java makes it easy to understand, reducing ambiguity.
  • Robust Libraries and Frameworks: Spring, Hibernate, and Maven make development easier.
  • Platform Independence: The "write once, run anywhere" philosophy ensures cross-platform compatibility.
  • Excellent Documentation: Comprehensive and extensive documentation is available.
  • Job Market: Due to its wide adoption, there are many job opportunities for Java developers.
  • Gradual Introduction to Functional Programming: With Java 8 and beyond, developers can explore functional programming concepts.

Cons

  • Verbose Syntax: Java requires more lines of code to accomplish tasks, which can lead to boilerplate code.
  • Slower to Adapt Modern Features: Java tends to be slower in adopting new programming concepts.

Scala

Pros

  • Functional and Object-Oriented: Scala beautifully merges object-oriented and functional programming paradigms.
  • Concise Syntax: Scala code is typically shorter than equivalent Java code, leading to increased productivity.
  • Advanced Features: Scala has modern features like traits, case classes, and implicit.
  • Great for Concurrency: Immutable data and first-class functions make Scala an excellent choice for concurrent and reactive applications.
  • Java Interoperability: Scala can seamlessly interoperate with Java code and libraries.
  • Powerful Tools: Scala has powerful tools like SBT and frameworks like Akka.

Cons

  • Steep Learning Curve: Scala's advanced features and concise syntax might be hard to grasp for beginners.
  • Smaller Community: Compared to Java, Scala's community is smaller, meaning less support and resources.
  • Less Job Market: There are fewer job opportunities for Scala developers compared to Java.
  • Complexity: The flexibility and power of Scala can lead to overly complex code if not used judiciously.

Conclusions

The choice between Java and Scala should be based on your project requirements, your team's skill set, and your personal preference. They are both powerful tools in their own right and understanding their strengths and weaknesses will help you make an informed decision.

Happy coding!

Author: Franco Geraci, Head Of Engineering at Bitrock

Read More