●● IRC: #techbytes @ Techrights IRC Network: Monday, October 30, 2023 ●● ● Oct 30 [02:40] *jacobk has quit (Ping timeout: 2m30s) ● Oct 30 [03:59] *jacobk (~quassel@32hz32it3ih2k.irc) has joined #techbytes ● Oct 30 [08:32] schestowitz
  • [08:32] schestowitz
    Alerting on sticky configuration reload failures for Prometheus
    [08:32] schestowitz
    [08:32] schestowitz

    Recently, I discovered that 'promtool check config' doesn't always fail if you have Prometheus configuration errors (which may be a bug). Fortunately this was only a mild issue because quite some time ago I added an alert for a sticky configuration reload failure. When this alert fired soon after I thought my configuration was good and my reload had succeeded, it was pretty straightforward to work out [08:32] schestowitz that 'promtool check config' was more or less lying to me (especially since the Prometheus server did log information about the problem).

    [08:32] schestowitz
    [08:32] schestowitz
    • [08:32] schestowitz [08:32] -TechBytesBot/#techbytes-utcc.utoronto.ca | Chris's Wiki :: blog/sysadmin/PrometheusAlertingOnBadReload ● Oct 30 [09:11] schestowitz x https://timesofindia.indiatimes.com/world/europe/west-main-culprit-behind-gaza-massacre-recep-tayyip-erdogan/articleshow/104792586.cms [09:11] -TechBytesBot/#techbytes-timesofindia.indiatimes.com | West 'main culprit' behind Gaza 'massacre': Recep Tayyip Erdogan - Times of India [09:15] *psydroid2 (~psydroid@u8ftxtfux23wk.irc) has joined #techbytes ● Oct 30 [12:31] *jacobk has quit (Ping timeout: 2m30s) [12:32] *MinceR gives voice to psydroid2 ● Oct 30 [13:03] *jacobk (~quassel@99ed6ukzxymmc.irc) has joined #techbytes ● Oct 30 [14:09] *jacobk has quit (Ping timeout: 2m30s) ● Oct 30 [15:36] *jacobk (~quassel@6wygwq2t5e2hw.irc) has joined #techbytes ● Oct 30 [23:10] *MinceR gives voice to jacobk [23:20] *psydroid2 has quit (Quit: KVIrc 5.0.0 Aria http://www.kvirc.net/) [23:37] *jacobk has quit (Ping timeout: 2m30s)