May 24 09:31:21 agmaster-icinga-1a kernel: icinga2[531]: segfault at 48 ip 000000000062a617 sp 00007fbd824b2990 error 4 in icinga2[400000+a0c000]
We are stuck with this error for almost a week now. Don’t see any errors when restarting Icinga2. Only when we check the status of Icinga2 and other logs we see this.
We are afraid that we will have to re do the entire work all again which would take another few days of our efforts.
I ran icinga2 daemon -x debug, didn’t get any errors or warnings. However went through each conf and other files icinga2 going through. I noticed that [2020-05-28 16:55:43 +0000] information/ConfigObject: Restoring program state from file '/var/lib/icinga2/icinga2.state'
The file was last updated on May 21st 2020, and Icinga2 is still trying to reference that. I emptied the file, restarted Icinga2 and it has started successfully. Let me know of your views on this fix.
We were unable to generate the core dump. The backup has been removed after a few days of the fix. We will try to have dumps generated if we encounter similar problem in future.