If you are interested in learning more about DevOps observability, here are the top five books that I recommend:

1. "Site Reliability Engineering: How Google Runs Production Systems" by Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy: This book provides an in-depth look at Google's approach to managing large-scale systems and how they use observability to monitor and troubleshoot their production environments. It covers topics such as monitoring, logging, tracing, and alerting, and provides practical advice for building highly reliable and scalable systems.

2. "Monitoring and Observability" by Cindy Sridharan: This book offers a comprehensive overview of observability, its key concepts, and how it can be applied to monitoring and troubleshooting distributed systems. It covers topics such as logging, metrics, tracing, and alerting, and provides practical advice for building effective observability systems.

3. "Observability Engineering" by Charity Majors and Lindsey Thorne: This book provides a practical guide to building and managing highly observable systems, with a focus on logging, metrics, tracing, and alerting. It covers topics such as service-level objectives (SLOs), error budgets, and chaos engineering, and provides practical advice for building and scaling observability systems.

4. "Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale" by Jennifer Davis and Katherine Daniels: While not exclusively focused on observability, this book covers all aspects of DevOps, including observability, monitoring, and logging. It provides a comprehensive overview of the DevOps culture, mindset, and practices, and provides practical advice for building and scaling effective DevOps teams.

5. "Distributed Systems Observability" by Liran Haimovitch: This book focuses on observability in distributed systems and how to apply it to detect and diagnose problems in complex, multi-layered architectures. It covers topics such as distributed tracing, log aggregation, and anomaly detection, and provides practical advice for building effective observability systems.

I recommend these books to anyone who wants to learn more about DevOps observability and build more reliable, scalable, and resilient systems. These books offer practical advice, real-world examples, and best practices that can help you improve your observability and DevOps practices.

What books have you read that should be on this list?

In the post, I explain how we were able to leverage the deep synergies between Loki and Prometheus to monitor the actual performance of jobs, allowing us to reduce cycle time for loads from 20 minutes to less than six minutes. By combining metrics with logs information, we were able to deeply understand where compute and memory were being efficiently used and where it was being wasted. This unlocked 40% savings on the cost of the cloud infrastructure supporting these stream jobs.

I go deeper into these two use cases and also highlight the job_exporter we built that leverages the symbiosis between Prometheus and Loki to break the high cardinality barrier. Our job monitoring journey started by implementing one-off solutions for Databricks and for Azure Data Factory. The learnings from these two implementations — and the need to extend to more platforms — led us to build a generic job_exporter that is easily extensible.

I hope my experience inspires you to look for what you can achieve by breaking the high cardinality barrier. Check out the full blog post on the Grafana blog to learn more:
https://grafana.com/blog/2022/12/02/monitoring-high-cardinality-jobs-with-grafana-grafana-loki-and-prometheus/