Open Telemetry
Abstract
This project is evaluating the adoption of OpenTelemetry as a standardized observability framework within the MOC clusters. As the MOC gets more customers and our clusters grow in scale and complexity, we need a more robust solution for collecting not just metrics, but traces, and logs. Currently, observability is primarily driven by metrics, providing visibility into system health and performance, but lacking insight into end to end request behavior and the relationship between different components. This project aims to determine how OpenTelemetry can expand upon our current telemetry by correlating metrics with trace context, allowing greater insight into our distributed OpenShift environments. Sally O’Malley from ET has experience with OpenTelemetry and we are working with her to expand upon what she has learned and transfer that knowledge to the MOC.
Goals of this project:
- Establishing a standardized approach to gathering metrics, traces, and logs using OpenTelemetry
- Enabling distributed tracing to observe end to end requests across different components such as pods, services, and cluster infrastructure
- Improve root cause analysis by correlating metric anomalies with traces
Learn more
- https://github.com/OCP-on-NERC/nerc-ocp-config/tree/main/opentelemetry
- https://github.com/nerc-images/ai-telemetry
Core Project Team
- Isaiah Stapleton, Red Hat Research
- Chris Tate, Red Hat Research
- Thorsten Schwesig, Red Hat Research