Originally published in Forbes.com
Digital-first businesses are striving for service assurance, which has become the lifeblood for their businesses processes. But they are increasingly getting complex across legacy and cloud-native applications, multi-cloud distributed services, with the rise of edge and when leveraged with Kubernetes and microservices architectures. Service assurance needs full-stack observability; however, customers need an approach to tame the data deluge while enabling actionable insights. This is where observability needs to be augmented by AIOps to contextualize, automate, scale and instantly remediate the root cause while delivering business outcomes.
Observability is the ability to determine the internal state of a system based on externally available outputs (e.g., observable artifacts like metrics, events, logs and traces) and thereby understand the state of individual hardware, software components and systems. However, just collecting MELT is not enough. Observability needs to be combined with AIOps and DataOps strategies to enhance service delivery and accelerate digital innovation to help IT more effectively support business objectives. In fact, MarketsandMarkets estimates that AIOps’ global market size will grow from $2.55 billion in 2018 to $11.02 billion by 2023 at 34% CAGR.
Robotic Data Automation Fabric (RDAF)
RDAF infuses AIOps and robotic data workflow automation across the customer’s distributed edge using next-generation technologies like NLP (natural language processing), NLG (natural language generation), distributed messaging and IDEs (integrated development environment) for building customized, on-the-fly, instantaneous pipelines using low-code databots.
In recent months, the term “data fabric” has garnered a lot of traction within the data management and analytics circle. In fact, Gartner recently identified “data fabric” as one of the top 10 data and analytics trends for 2021 and 2022, as well as in the much-talked-about data mesh architecture laid forth by Zhamak Dehghani and recently analyzed by Wikibon.
RDAF should provide a secure, multi-tenant environment to design, explore, experiment, test, collaborate, reiterate, observe and finally deploy pipelines in production in a SaaS consumption model. RDAF comprises the following building blocks:
• RDA data studio (AIOps studio): Automating data and ML pipelines for AIOps and observability.
• RDA fabric (global databots network): Highly secure and distributed data analytics fabric and orchestration:
• RDA app studio: Low-code edge data analytics applications development
Every databot is a containerized atomic unit, which has access to its own data store and messaging APIs, is programmable with a low-code approach using IDE and implements DataOps and AIOps — intelligent AI/ML algorithms, neural nets, using unsupervised, federated and reinforcement learning approaches. Due to containerization, a databot can run at the edge, at the core or in a public or private, multi-cloud network forming an automated data fabric. Each databot provides pipeline tracing, data lineage and accounting, as well as data governance.
RDAF should eliminate data silos with the mission to deliver “data as a product.” RDAF does this by democratizing and self-serving data insights to multiple personas — citizen developers, CloudOps and SecOps teams, Splunk, Elastic admins, IT engineers, SREs, and DevOps, DataOps teams. No-code platforms enable companies and business professionals with minimal or no coding experience to build apps and fill the talent gaps in their organization.
RDAF Use Cases
RDAF has very broad use cases, but to provide some specific examples for AIOps, the following are more relevant:
• Optimizing log analytics for operational intelligence and SIEM.
• ITSM — incident management and data automation.
• Asset data discovery, data lineage and metadata management.
• Edge AI and industrial IoT observability.
• Data reliability automation.
• Autonomous cloud observability and automation.
• DevOps and ML pipeline observability and automation.
Additionally, there are other hyperautomation use cases where RDAF can complement RPA (robotic process automation) and data integration.
RDAF Enables The Three Cs
• Consolidate using observability: This gives you the ability to observe a number of diverse data sources across the full stack. You can connect to these different data sources with different data formats and collect and unify data and metadata to understand lineage.
• Converge using AIOps and DataOps: Contextualize, correlate, transform and leverage AI to find patterns and anomalies using technologies like NLP and no-code automation data bots.
• Conclude with hyperautomation: Formulate automation pipelines using LCAP and intelligently route data to enable reactive and proactive service management.
Data-driven organizations should explore, evaluate and implement RDAF for faster innovation, faster time to value, meet SLAs and SLOs, and excel at customer experiences. Some innovative vendors are effectively implementing RDAF/data mesh today like CloudFabrixf or AIOps and Starburst.io for healthcare and lifesciences.
