How a Modern Integration MESH Changes our Roles and Responsibilities

There was a time not too long ago, before the cloud was a part of every enterprise technology conversation, when integration work was considered the purview of a specific architecture and engineering group.

If messages failed to send, or services failed to respond, application stakeholders would create a trouble ticket for the integration team to address.

In some ways, this separation of labor was effective enough at the time. There were a finite number of point-to-point integrations between systems, and most request/response traffic would flow through an integration backbone, such as an ESB or MQ server.

Even if a developer did some of the adaptation work in their own code, when their applications would be promoted, it would be standard practice to hand over the specifications and allow integration teams to manage how the transactions would play out, with jobs and traffic scheduling to hopefully avoid bottlenecks.

Whenever a blockage did occur, integration engineers would be first on the scene, because they were the most familiar with how their team set up each adapter and data transformation step branching off of that message bus.

In the modern application world, integration has become way too complex to be the purview of one group, or even one huge department, as our applications now surf on an ever-changing sea of services and hybrid IT architecture that is connected to partners, and the world at large.

Covering complexity with MESH

The software development community at large constantly invents new ways to bridge the divide between message queues and point-to-point integrations in order to talk to disparate systems. There were web services and SOA, then various shades of integration fabrics, and then containers and service meshes as today’s cloud native architectures evolved.

One company attempting to tackle today’s integration challenges, meshIQ, has coined a new acronym to break down the integration pattern. In this case, it’s not another service mesh—MESH stands for “Messaging, Event processing, and Streaming, across Hybrid cloud.”

Separately, each letter in the MESH represents a broad technology category. The plan is to bring together a flexible solution to integrate distributed and decentralized technologies. MESH in this sense is just as much a thing you do, as a particular solution you would buy, because it changes the relationships of DevOps, architecture, and operations teams to the integration process.

New stakeholders for integration work with MESH

Gone are the days of throwing adaptation requirements and service tickets ‘over the wall’ to someone else.

Today, the old organizational silo of an integration team is giving way to a platform engineering function. Platform engineering is done by a federated team, perhaps drawn from a council of other stakeholder departments, combining solutions from selected vendors to provide developers, architects and other roles with the self-service tools they need to do development, deployment, and integration themselves, with minimal support or intrusion.

The advantage of providing a MESH for platform engineering is immediately obvious to any teams that previously had to wait for provisioning budget reviews and change control boards. We want to distribute integration work, and observability into the state of integration, down to new stakeholders in order to deliver new functionality faster, and respond to problems with faster remediation.

In this story, we’ll discuss five unique new stakeholders for MESH that benefit from this sea-change in integration roles and responsibilities.

Shared Services / Platform Teams

Shared Services groups are the most likely owners of the platform engineering initiative. Often, this group inherits some of the workload left behind by the departure of the old centralized middleware integration team, with a notable exception. Rather than acting as a ‘service bureau’ for others, they lay the self-service infrastructure foundations for application delivery and operations tooling to serve other employee groups.

Primary considerations: Control and Governance

Since shared services is still responsible for the ongoing support of both legacy middleware integration layers and newer API and event-driven hybrid cloud architectures, they are looking to MESH to gain a single point of control over all middleware, investigate problems, and make changes to objects such as repairing a message queue or event bus when needed.

Shared Services manages the inventory of the integration architecture used for platform engineering activities, ensuring that all of the assets in play and connections are holding up under the workloads of other constituent teams, and updating or retiring assets when needed.

Enterprise Architects

Enterprise Architects (or EAs) view integration from a high level, but they also need to dig into the details. For any new project, they will establish a starting data schema or communication format, then visit the MESH to decide on group permissions, and set the standards for ongoing integration projects across the enterprise. Importantly, the perspectives of EAs should prevent technology sprawl and eliminate redundant or dead-end components as underlying cloud technologies keep changing.

Key considerations: Standardization and Consistency

EAs are always seeking higher levels of automation and reproducibility for Infrastructure-as-Code (IaC) recipes and scripts such as Ansible playbooks, which define the desired to-be state of each next release in relation to the integration standards they specify. In these situations, the MESH normalizes API interfaces across multiple clouds, middleware stacks and data services, so underlying application development teams can deliver their own part with consistency, without excessive redefinition or customization.

Application Owners

Application Owners may come from either development, or the sales or marketing side of the business. These individuals often own the P&L (profit & loss) responsibility for an application, gathering market demand signals and customer requirements while advancing the feature roadmap and ensuring that the application meets customer and regulatory demands. The scope, revenue targets and budget of the product influences integration choices.

Key Considerations: Auditability and Change Awareness

As software becomes more distributed and granularized into microservices, the number of service connections and API calls that comprise an application increases exponentially. The Application Owner would use MESH observability to keep track of all configuration changes made by developers and admins for every object within their customized application domain view.

Using this method, they can generate a full audit trail of all changes: who executed them and when, how they contribute to the completion of each targeted feature, and the relative success metrics and integration risk of each change on the production system.

Application Developers

Application Developers have a strong need to provision their own dev and test environments, so they can be agile and productive without provisioning bottlenecks, and deliver new functionality without worrying about the timing of production change windows.

Key considerations: Self-service provisioning and flexibility

Every bit of workflow code a developer writes for modern systems is highly dependent on component-level integration and messaging layers, therefore they need ready and functionally complete self-service environments for testing and promotion.

Developers want the permissions and empowerment to move forward using a MESH to create and consume REST-style or real-time async APIs on middleware objects, without having to wait for environment provisioning bottlenecks or lengthy change request/review cycles. A tightly granular set of access privileges to groups of objects allows them to safely work within their own application feature.

Devs can check in code without impacting the dependencies of adjacent development teams or the production system at large, while architects and managers can maintain control through system-wide monitoring and standardization of the integration object library.

Application Support

Application Support takes many different forms in an organization. There are customer-facing support teams that can resolve simple user error problems, while taking in feedback and error reports. Employee support can assist platform engineering with a help desk function that provides educational resources and keeps productivity high for all constituents of the MESH. Operations support teams need production monitoring and deeper observability into the root causes of failures and performance problems.

Key Considerations: Real-time Insight and Policy Management

Maintaining high service level objectives (SLOs) for any application stack only happens with complete real-time monitoring, early awareness and alerting of issues. Given the complexity of underlying integrations, the MESH needs to provide machine-learning based filtering routines for runtime metrics and event data that is coming in too fast for a human to process.

To stay ahead of the rate of change and focus on what is significant, support teams can group together and classify analytics and targeted searches of source data in the MESH event catalog, and create monitoring policies that address the most likely risk factors for system instability, performance hits and potential security exploits. When a high-priority alert is received, support teams can then turn over the issue to the owner of the affected application domain or integration object, with an incident report for faster remediation.

The Intellyx Take

Organizations that are seeking to close the chapter of dedicated integration teams, and the inflexibly rigid provisioning and change management standards that accompanied them, should consider moving toward a modern integration MESH platform approach like meshIQ.

As the future of enterprise software integration will only become more distributed and fragmented across hybrid cloud resources, APIs and data services, so should our stakeholder roles and responsibilities change to meet the challenges of increased complexity in deployment.

©2023 Intellyx LLC. Intellyx is editorially responsible for this document. No AI bots were used to write this content. At the time of writing, meshIQ is an Intellyx customer. Image sources: TBA; Screenshot: meshIQ dashboard (optional)