Ask the Experts & Innovators: CX and the Transition from Monitoring to AIOps & Observability

Customer experience, also known as CX, is a hot topic today.  Observability and AIOps (machine learning and artificial intelligence for IT operations), can be used for determining and linking transaction performance to your business performance, in real-time.  That makes them hot topics as well.  I recently spoke with two experts about the intersection of these topics to discuss what leading companies are doing (and evaluating) in these areas, and how they’re doing it.  The discussion includes these areas:

• AIOps
• CX (Customer Experience)
• Observability
• Advanced Troubleshooting and RCA (root cause analysis)
• Next Generation Monitoring / APM
• End-to-end Message and Transaction Tracking
• IBM MQ, Kafka, and challenges with monitoring and management of multi-middleware environments
• Build vs. Buy for Monitoring, Analytics and Situational Awareness
• Insurance, Banking, Securities, Omni-channel Retail Industry Needs
• Nastel Technologies, AppDynamics, Dynatrace, New Relic, Splunk, IBM, Apache Kafka

Link: Listen to the full interview via your browser or download it to your favorite podcast app here.

Excerpts, Key Takeaways, and More from “Ask the Experts and Innovators”:

Leigh Reed is a Service Automation and Assurance professional based in the UK. He is also an expert in service operations and APM, including messaging middleware and full stack performance improvement and troubleshooting.  As someone focused on service assurance and continual improvement in application performance, and everything else that impacts existing and prospective customers’ experience, I asked him what he’s been exploring lately.

LEIGH: I’ve been looking into a number of things, primarily I’ve been researching the latest in APM, transaction tracking, AIOPs and different methods to gather data for analytics with machine learning.

LEIGH: With my background I’ve learned a lot about what can be done with the best monitoring tools to detect and quickly respond to issues and bigger problems.  What’s exciting to me is what can be done with all that data that we can now collect from the applications, and even the messaging middleware layer and within containers, to dramatically improve operations.   Being able to query data pools and use machine learning-powered analytics to plot norms, detect pattern and anomalies, and drive decisions is very exciting to me.

Leigh Reed and I were also joined by David Liff, an expert in business marketing, customer experience, or CX), and the customer buying journey.  Mr. Liff is also a prolific writer on IT trends and innovation, and he’s the most customer-centric IT solutions professional I’ve met in my 25 years in the industry.  David is very familiar with operations’ challenges related to the messaging middleware layer of the IT stack at companies today… IBM MQ, IBM Integration Bus, TIBCO EMS, and of course Kafka, to name the big ones.   I asked David why this part of the IT Stack so important for customer experience-focused businesses, and why so many organizations struggle with that part of the stack, despite many of them already having Dynatrace, AppD, Splunk, New Relic and other tools already in-house.

DAVID:  Messaging middleware is part of the glue that links together all the elements of the enterprise. Classic thinking has always been to treat messaging middleware systems as just a bunch of servers, and to monitor the usual speeds and feeds, how fast, how much, how many-type of metrics, but if you look at messaging middleware as a source of knowledge about how your environment is connected, and even consider the contents of messages, you can learn an amazing amount about your business and technology. And historically, very few companies have thought that way. But when you overlay that messaging middleware context, on top of what you’ve already learned for messages with relevant performance data, such as the classic monitoring, you’d have an entirely additional dimension of understanding and this can change everything, in very important ways.

STEVE: So that’s why so many IBM MQ, IIB, Kafka and TIBCO EMS shops still have major monitoring and transaction tracing “blind spots” despite having Dynatrace, AppD, Splunk, New Relic and other tools?

DAVID: Yes, that’s it. Many companies are looking to dump their classic monitoring type legacy tools such as BMC and Tivoli, and they’re looking at new monitoring tools, the so-called “observability” tools.  And it’s only then that they find out that they still have a black hole through the messaging middleware layer. That’s why folks like Leigh have had additional reasons to use Nastel to gain visibility and insights into that layer and complete the whole picture. When you have a visual map of each user’s engagement, or transaction’ that you can compare to the historical record, what you get to is a place where you can have an intelligent approach to how you consider messaging middleware, you can reduce the time it takes to identify issues and solve those issues. Dramatically faster.

Leigh and I sat in on a walk-through with Nastel’s Scott Corrigan for the latest release of the Nastel AIOps and transaction tracking platform, “XRay” and here were Leigh’s first impressions.

LEIGH: Number one was how powerful, but easy-to-use it is, right out of the box.  My understanding of other AIOps tools is that they require extensive services and customization and have a long, expensive time-to-value.  Number two was the end-to-end transaction tracking that also included a topology view (to visualize message flow) and machine learning-powered insights.  Everything was clickable too, so drilling down into anything to learn more was simple.  The variety of data collector types with Nastel also really stands out. 

STEVE: Leigh, do you see the Nastel XRay platform as a viable alternative to Dynatrace, Splunk, New Relic, and AppDynamics?

LEIGH: Interesting question… Nastel was already addressing a big gap in the MQ monitoring space with AutoPilot and I know first-hand that in addition to being the only viable way to troubleshoot complex MQ and multiple middleware environments, it can monitor virtually anything in the entire legacy-to-Cloud and microservices stack.  Pushing all that data into XRay for analysis sure looks like a viable “observability” solution to me, especially if you’ve still got any legacy systems in your stack and rely on MQ, Kafka or other messaging middleware for customer transactions.  I’d put Nastel on the short list any new APM, AIOps, or Observability project aimed at addressing customer experience, business decision support or improving root cause analysis and SLAs.

STEVE: What about you Dave?  Where do you see a platform designed like Nastel XRay fitting into today’s market?

DAVE: Look, Nastel XRay provides a truly unique way of understanding the subtleties of user experience, performance and availability. Modern environments don’t fail because the system fails, they fail because a dozen different systems are all reaching different thresholds at around the same time. Classic thinking does not allow these kinds of events to be identified, while XRay does.   And this provides the business with the ability to project manage and solve this kind of business, technical, even security issues quickly.  Issues that would take classic processes 1000s of man hours to deal with. We’ve seen XRay help companies in a multitude of industries, including banking, finance, FinTech, insurance, retail and utility services. every vendor seems to have their own definition today for “Observability,” and even AIOps, and they are often far too self-serving definitions. We think the definition should be simple. And to us, it revolves around being able to discover and visualize each user’s experience through the entire application stack, and to be to be able to deeply delve into that data to resolve problems as quickly as possible.

DAVE: In one case, a top insurance company had invested literally 10s of years and countless hours to build truly complete transaction tracking. With their complex, old and new IT stack, now further complicated with microservices at the edge, their customer service reps,  (CSRs), and the teams that support them, consistently reported missing SLA targets and customer service objectives, customers and prospects waiting and waiting for their CSRs to find delayed and even lost transactions was totally unacceptable. In this ultra competitive insurance industry, that equates to dissatisfied customers, lost revenue, and ultimately missed growth targets for management… so, very clearly defined KPIs that they were missing.  They were motivated to look outside for help and Nastel got to go head-to-head with other APM and Observability leaders.  We won it.

 STEVE:  Wow. Okay, so Nastel did go head-to-head with the “big boys” that we were talking about, the big boys in “Observability,” so what happened there?

DAVID: Well, Nastel actually are the big boys in Observability.  But it turns out that transaction tracking in the industry in the insurance industry requires extracting data and deep insights from the messaging middleware level, combing it with the performance data from the rest of the stack, and understanding the transaction flow. Several vendors were asked how they would solve this issue, and only Nastel was able to solve it. We proved it and we delivered it for them.

Interested to hear what else Leigh and David had to say in this episode of “Ask the Experts and Innovators?”  Listen to the entire discussion via your browser or download it to your favorite podcast app here.

About Ask the Experts and Innovators

Ask the Experts and Innovators is a new content series which explores real stories of dramatic improvement in CX (customer experience), application performance, IT Operations, and DevOps as well as the cross-functional conundrums of root cause analysis (RCA) and problem resolution.  Ask the Experts and Innovators is produced by Nastel Technologies.

Ask the Experts and Innovators is Hosted by Nastel Technologies’ Head of Product Management and Advisory Board Director, Steven Menges.

About Leigh Reed: Leigh is a Service Automation and Assurance Analyst, who is also an expert in service operations and APM, including messaging middleware and full stack performance monitoring.  Mr. Reed has a great background, which includes him having been tasked to ensure that the customer experience (CX) is excellent across all customer-facing channels of a global, multi-channel retail business with an increasingly complex IT infrastructure.    In case you missed it, you can listen to our 1st interview with Mr. Reed.   He talks about how he and his omni-channel retailer service assurance and architect teams solved some complex service assurance and operations problems spanning Cloud and legacy infrastuctures across multiple operating systems for their IBM, BMC, and Oracle-centered organization.  He is based in the UK.

About David Liff:  Mr. Liff is a customer-centric IT business and marketing leader known for using the broadest range of innovative techniques to understand and engage buyers best match the buying community at hand.  David has over 27 years experience of global technology “business to business” and “business to consumer” channel and direct marketing at a multitude of well-known corporations including HP (Hewlett Packard), Nimsoft, ArisGlobal, CA Technologies (Computer Associates), and Nastel Technologies.  A published author, Mr. Liff is a dual US and European Citizen and is originally from the UK where he graduated from Westminster University. He lives in New York City.