Awesome NomadMonitoring and Logging

seatgeek/nomad-firehose

⭐ 119 Go repository created 2017-07-12

The project "nomad-firehose" by seatgeek is a specialized tool designed to facilitate real-time monitoring and event streaming of HashiCorp Nomad cluster changes. It captures all changes related to Nomad jobs, allocations, nodes, and evaluations and streams these events to various sinks such as RabbitMQ, Kinesis, Kafka, Redis, MongoDB, NSQ, SQS, EventBridge, syslog, or simply outputs them to stdout for debugging purposes. This tool is particularly useful for teams that want to build custom logic or automation around Nomad task events without directly interfacing with the Nomad API. Nomad-firehose supports multiple platforms including Linux, Darwin, and Windows, and offers a Docker container for easy deployment. It leverages Consul for leader election and state management, ensuring that only one instance of the firehose is active at a time, which prevents duplicate event processing. The tool also stores the last processed event timestamp in Consul, allowing it to resume from the last known state after restarts, thus avoiding reprocessing old events. Configuration is flexible and environment variable-driven, supporting all native Nomad and Consul CLI environment variables. It also supports secure connections and authentication for Kafka and other sinks. The tool provides several subcommands to monitor different Nomad entities such as allocations, nodes, evaluations, jobs, job list stubs, and deployments. Each subcommand emits events in a structured JSON format that corresponds closely to the official Nomad API structures but optimized for event streaming. Overall, nomad-firehose is a powerful utility for organizations using HashiCorp Nomad who need to integrate Nomad event data into their existing messaging or event processing systems, enabling enhanced observability, automation, and operational workflows.

https://github.com/seatgeek/nomad-firehose

automationconsulevent-processingevent-streamingeventbridgefirehosehashicorphashicorp-nomadkafkakinesisleader-electionmongodbnomadnomad-allocationsnomad-evaluationsnomad-eventsnomad-job-monitoringnomad-nodesnsqobservabilityrabbitmqredissqsstdoutsyslog

Also in Monitoring and Logging

mr-karan/nomad-vector-logger

Nomad-vector-logger is a daemon that monitors Nomad cluster jobs and generates Vector configuration files to collect logs enriched with Nomad metadata for improved centralized logging and observability.

mr-karan/nomad-monitoring

A collection of jobspecs and Grafana dashboards for comprehensive end-to-end monitoring of HashiCorp Nomad clusters, utilizing Prometheus metrics and VictoriaMetrics for efficient data collection and storage.

sepulworld/deadman-check

Deadman Check is a monitoring tool for HashiCorp Nomad periodic jobs that alerts users if jobs fail to run at their expected intervals, using Consul for tracking and supporting Slack and AWS SNS for notifications.

mr-karan/nomad-events-sink

Nomad Events Sink is an event collection agent that processes HashiCorp Nomad events and forwards them to external sink providers like HTTP for monitoring, alerting, and analysis.

jrasell/nomad-toast

Nomad Toast is an open-source tool that provides real-time notifications based on HashiCorp Nomad events, enhancing observability and operational awareness through Slack integration.

kangaroot/rootstack-devfactory

Rootstacks Devfactory is a solution that provides a secure, observable, and integrated runtime environment for containers, native applications, and VMs using HashiCorp Nomad and related tools.

burdandrei/nomad-monitoring

A project for monitoring HashiCorp Nomad clusters using InfluxDB, Telegraf, and Grafana with Prometheus metrics telemetry.