# IcingaDB 'max\_allowed\_packet' exceeded

**URL:** <https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414>\
**Category:** Icinga DB\
**Created:** [December 23, 2024, 2:19pm UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414 "2024-12-23T14:19:52Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [December 23, 2024, 2:19pm UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/1 "2024-12-23T14:19:52Z")

</div>

Hello

Since the update to IcingaDB v1.2.1, we are experiencing Daemon failures in the IcingaDB every 5 minutes.

```auto
Dec 22 10:13:58 skinner2.backbone.admin icingadb[1507116]: Error 1105 (HY000): Parameter of prepared statement which is set through mysql_send_long_data() is longer than 'max_allowed_packet' bytes
                                                           can't perform "INSERT INTO \"state_history\" (\"state_type\", \"hard_state\", \"previous_soft_state\", \"check_attempt\", \"soft_state\", \"long_output\", \"object_type\", \"host_id\", \"check_source\", \"endpoint_id\", \"service_id\", \"event_time\", \"previous_hard_state\", \"output\", \"max_check_attempts\", \"id\", \"scheduling_source\", \"environment_id\") VALUES (:state_type,:hard_state,:previous_soft_state,:check_attempt,:soft_state,:long_output,:object_type,:host_id,:check_source,:endpoint_id,:service_id,:event_time,:previous_hard_state,:output,:max_check_attempts,:id,:scheduling_source,:environment_id) ON DUPLICATE KEY UPDATE \"id\" = VALUES(\"id\")"
                                                           github.com/icinga/icinga-go-library/database.CantPerformQuery
                                                                   github.com/icinga/icinga-go-library@v0.4.0/database/utils.go:16
                                                           github.com/icinga/icinga-go-library/database.(*DB).NamedBulkExec.func1.(*DB).NamedBulkExec.func1.1.2.1
                                                                   github.com/icinga/icinga-go-library@v0.4.0/database/db.go:535
                                                           github.com/icinga/icinga-go-library/retry.WithBackoff
                                                                   github.com/icinga/icinga-go-library@v0.4.0/retry/retry.go:65
                                                           github.com/icinga/icinga-go-library/database.(*DB).NamedBulkExec.func1.(*DB).NamedBulkExec.func1.1.2
                                                                   github.com/icinga/icinga-go-library@v0.4.0/database/db.go:530
                                                           golang.org/x/sync/errgroup.(*Group).Go.func1
                                                                   golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78
                                                           runtime.goexit
                                                                   runtime/asm_amd64.s:1700
                                                           retry deadline exceeded
                                                           github.com/icinga/icinga-go-library/retry.WithBackoff
                                                                   github.com/icinga/icinga-go-library@v0.4.0/retry/retry.go:100
                                                           github.com/icinga/icinga-go-library/database.(*DB).NamedBulkExec.func1.(*DB).NamedBulkExec.func1.1.2
                                                                   github.com/icinga/icinga-go-library@v0.4.0/database/db.go:530
                                                           golang.org/x/sync/errgroup.(*Group).Go.func1
                                                                   golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78
                                                           runtime.goexit
                                                                   runtime/asm_amd64.s:1700

```

On our server side, we already set ‘max\_allowed\_packet’ to 64MB and on the client side, we are at 64MB as well.

- Icinga DB Web version: 2.12.2
- Icinga Web 2 version: 1.1.13
- Web browser: Firefox 128.0.5esr
- Icinga 2 version: r2.14.3-1
- Icinga DB version: v1.2.1
- PHP version used: 8.1.2-1ubuntu2.20
- Server operating system and version: Ubuntu 22.04.5 LTS

How shall we proceed?

---

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [December 24, 2024, 9:20am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/2 "2024-12-24T09:20:01Z")

</div>

FTR: our icingadb-redis-server had a memory usage of around ~300MB.

```auto
root@skinner1:~# redis-cli -p 6380 info | grep used_memory_human
used_memory_human:304.15M

```

After flushing out the Redis Store, our system no longer crashes.

```auto
redis-cli -p 6380 flushall

```

I’m assuming that during the upgrade the Redis Server was running on, while the IcingaDB was down and stored up too much data in the “buffer” and IcingaDB was no longer capable of syncing this data.

However, issue resolved.

---

<div class="post-metadata">

**Author:** ![apenning](https://community.icinga.com/user_avatar/community.icinga.com/apenning/32/7669_2.png) [@apenning](https://community.icinga.com/u/apenning)\
**Post date:** [January 7, 2025, 10:51am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/3 "2025-01-07T10:51:53Z")

</div>

Thanks for both reporting and solving this issue.  
As a crash is undesired, I have taken another look at this issue.

Icinga DB always sets `max_allowed_packet` to 64 MiB, as this is what the underlying MySQL client library does, [https://github.com/go-sql-driver/mysql/blob/4395c45fd098a81c5251667cda111f94c693ab14/dsn.go#L88](https://github.com/go-sql-driver/mysql/blob/4395c45fd098a81c5251667cda111f94c693ab14/dsn.go#L88). However, if I read the following MySQL bug - [https://bugs.mysql.com/bug.php?id=83958](https://bugs.mysql.com/bug.php?id=83958) - correctly, prepared statements may easily exceed this limit, as it seems to be the case for your crash.

Due to the temporary unavailability of Icinga DB, the backlog stored in the Redis outgrew this limit, resulting in a later crash. I created an issue for this, as I think we should be able to mitigate this bug within our code base.

> <https://github.com/Icinga/icinga-go-library/issues/108>
>
> An Icinga DB crash was reported in the Icinga Community Forum, \<https://communit…y.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414\>. After a temporary Icinga DB downtime due to a version update, the Redis contained too much data for the \`state\_history\` table, exceeding MySQL's \`max\_allowed\_packet\` value of 64 MiB.
> 
> \`\`\`
> Dec 22 10:13:58 skinner2.backbone.admin icingadb\[1507116\]: Error 1105 (HY000): Parameter of prepared statement which is set through mysql\_send\_long\_data() is longer than 'max\_allowed\_packet' bytes
> can't perform "INSERT INTO \\"state\_history\\" (\\"state\_type\\", \\"hard\_state\\", \\"previous\_soft\_state\\", \\"check\_attempt\\", \\"soft\_state\\", \\"long\_output\\", \\"object\_type\\", \\"host\_id\\", \\"check\_source\\", \\"endpoint\_id\\", \\"service\_id\\", \\"event\_time\\", \\"previous\_hard\_state\\", \\"output\\", \\"max\_check\_attempts\\", \\"id\\", \\"scheduling\_source\\", \\"environment\_id\\") VALUES (:state\_type,:hard\_state,:previous\_soft\_state,:check\_attempt,:soft\_state,:long\_output,:object\_type,:host\_id,:check\_source,:endpoint\_id,:service\_id,:event\_time,:previous\_hard\_state,:output,:max\_check\_attempts,:id,:scheduling\_source,:environment\_id) ON DUPLICATE KEY UPDATE \\"id\\" = VALUES(\\"id\\")"
> github.com/icinga/icinga-go-library/database.CantPerformQuery
> github.com/icinga/icinga-go-library@v0.4.0/database/utils.go:16
> github.com/icinga/icinga-go-library/database.(\*DB).NamedBulkExec.func1.(\*DB).NamedBulkExec.func1.1.2.1
> github.com/icinga/icinga-go-library@v0.4.0/database/db.go:535
> github.com/icinga/icinga-go-library/retry.WithBackoff
> github.com/icinga/icinga-go-library@v0.4.0/retry/retry.go:65
> github.com/icinga/icinga-go-library/database.(\*DB).NamedBulkExec.func1.(\*DB).NamedBulkExec.func1.1.2
> github.com/icinga/icinga-go-library@v0.4.0/database/db.go:530
> golang.org/x/sync/errgroup.(\*Group).Go.func1
> golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78
> runtime.goexit
> runtime/asm\_amd64.s:1700
> retry deadline exceeded
> github.com/icinga/icinga-go-library/retry.WithBackoff
> github.com/icinga/icinga-go-library@v0.4.0/retry/retry.go:100
> github.com/icinga/icinga-go-library/database.(\*DB).NamedBulkExec.func1.(\*DB).NamedBulkExec.func1.1.2
> github.com/icinga/icinga-go-library@v0.4.0/database/db.go:530
> golang.org/x/sync/errgroup.(\*Group).Go.func1
> golang.org/x/sync@v0.10.0/errgroup/errgroup.go:78
> runtime.goexit
> runtime/asm\_amd64.s:1700
> \`\`\`
> 
> The reporter solved this issue by performing a \`FLUSHALL\` in the Redis.
> 
> However, as I wrote in the thread, "Icinga DB always sets \`max\_allowed\_packet\` to 64 MiB, as this is what the underlying MySQL client library does, \<https://github.com/go-sql-driver/mysql/blob/4395c45fd098a81c5251667cda111f94c693ab14/dsn.go#L88\>. However, if I read the following MySQL bug - https://bugs.mysql.com/bug.php?id=83958 - correctly, prepared statements may easily exceed this limit, as it seems to be the case for your crash."
> 
> Without having looked too closely, the \`DB.NamedBulkExec\` method allows bulking. Maybe we can adjust something over there. I have mostly created this issue here to not lose this over in the community forum.

---

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [September 1, 2026, 8:07am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/4 "2026-09-01T08:07:10Z")

</div>

Hello @apenning, we are now experiencing the same issue as from 2 years ago. Flushing the Redis DB is no longer working as expected. AFAICS there is no solution for this bug currently? Whatever shall we do?

---

<div class="post-metadata">

**Author:** ![apenning](https://community.icinga.com/user_avatar/community.icinga.com/apenning/32/7669_2.png) [@apenning](https://community.icinga.com/u/apenning)\
**Post date:** [September 1, 2026, 8:26am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/5 "2026-09-01T08:26:29Z")

</div>

I am sorry hearing that, @meuthak. To allow pinning this down, please share a few information about your setup with us: the log showing the crash, which Icinga components and DBMS you are using and in which version.

> [@meuthak](#):
>
> Flushing the Redis DB is no longer working as expected.

Could you elaborate? How are you flushing Redis and what does not work as expected anymore? Is it filling up again and crashes after some time?

---

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [September 1, 2026, 8:45am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/6 "2026-09-01T08:45:52Z")

</div>

We are having a problem, that some history sync cannot be written into icingadb.

`Error 1105 (HY000): Parameter of prepared statement which is set`  
`through mysql_send_long_data() is longer than 'max_allowed_packet' bytes"`

When we look at the Redis DB, it is quite large:

```auto
redis-cli -p 6380 info | grep used_memory_human
used_memory_human:304.15M

```

After flushing it, icingadb can start up again and the sync is no longer an issue.

```auto
redis-cli -p 6380 flushall

```

We are using these icinga components:

```auto
icinga-l10n/icinga-jammy,now 1.4.0-1+ubuntu22.04 all [installed,automatic]
icinga-php-library/icinga-jammy,now 0.15.2-1+ubuntu22.04 all [installed]
icinga-php-thirdparty/icinga-jammy,now 0.12.1-1+ubuntu22.04 all [installed]
icinga2/icinga-jammy,now 2.16.5-1+ubuntu22.04 amd64 [installed]
icinga2-bin/icinga-jammy,now 2.16.5-1+ubuntu22.04 amd64 [installed]
icinga2-common/icinga-jammy,now 2.16.5-1+ubuntu22.04 all [installed]
icingacli/icinga-jammy,now 2.12.7-1+ubuntu22.04 all [installed]
icingadb/icinga-jammy,now 1.5.1-8+ubuntu22.04 amd64 [installed]
icingadb-redis/icinga-jammy,now 8.2.9-1+ubuntu22.04 amd64 [installed]
icingadb-web/icinga-jammy,now 1.1.4-1+ubuntu22.04 all [installed]
icingaweb2/icinga-jammy,now 2.12.7-1+ubuntu22.04 all [installed]
icingaweb2-common/icinga-jammy,now 2.12.7-1+ubuntu22.04 all [installed]
icingaweb2-module-monitoring/icinga-jammy,now 2.12.6-2+ubuntu22.04 all [installed,automatic]
monitoring-plugins/jammy,jammy,now 2.3.1-1ubuntu4 all [installed]
monitoring-plugins-basic/jammy,now 2.3.1-1ubuntu4 amd64 [installed]
monitoring-plugins-common/jammy,now 2.3.1-1ubuntu4 amd64 [installed]
monitoring-plugins-standard/jammy,now 2.3.1-1ubuntu4 amd64 [installed]
php-icinga/icinga-jammy,now 2.12.7-1+ubuntu22.04 all [installed]

```

We are aware, that icingadb-web is OutOfDate. However i do no think the issue lies there.

For our DB we are using a Galera Cluster, which is proxied through a Proxysql server. Do you need the versions of these?

Log message:

```auto
Sep 1 09:40:15 skinner1 icingadb[2362625]: database: Can't execute query. Retrying#011error="can't perform \"INSERT INTO \\\"service_state\\\" (\\\"service_id\\\", \\\"is_flapping\\\", \\\"
last_update\\\", \\\"severity\\\", \\\"normalized_performance_data\\\", \\\"previous_soft_state\\\", \\\"check_attempt\\\", \\\"is_handled\\\", \\\"state_type\\\", \\\"is_acknowledged\\\", \
\\"long_output\\\", \\\"check_timeout\\\", \\\"last_state_change\\\", \\\"check_commandline\\\", \\\"scheduling_source\\\", \\\"next_update\\\", \\\"last_comment_id\\\", \\\"execution_time\\
\", \\\"is_reachable\\\", \\\"performance_data\\\", \\\"is_problem\\\", \\\"latency\\\", \\\"is_sticky_acknowledgement\\\", \\\"host_id\\\", \\\"affects_children\\\", \\\"check_source\\\", \
\\"in_downtime\\\", \\\"output\\\", \\\"previous_hard_state\\\", \\\"acknowledgement_comment_id\\\", \\\"next_check\\\", \\\"properties_checksum\\\", \\\"environment_id\\\", \\\"id\\\", \\\"
hard_state\\\", \\\"soft_state\\\") VALUES (:service_id,:is_flapping,:last_update,:severity,:normalized_performance_data,:previous_soft_state,:check_attempt,:is_handled,:state_type,:is_ackno
wledged,:long_output,:check_timeout,:last_state_change,:check_commandline,:scheduling_source,:next_update,:last_comment_id,:execution_time,:is_reachable,:performance_data,:is_problem,:latenc
y,:is_sticky_acknowledgement,:host_id,:affects_children,:check_source,:in_downtime,:output,:previous_hard_state,:acknowledgement_comment_id,:next_check,:properties_checksum,:environment_id,:
id,:hard_state,:soft_state) ON DUPLICATE KEY UPDATE \\\"service_id\\\" = VALUES(\\\"service_id\\\"),\\\"is_flapping\\\" = VALUES(\\\"is_flapping\\\"),\\\"last_update\\\" = VALUES(\\\"last_up
date\\\"),\\\"severity\\\" = VALUES(\\\"severity\\\"),\\\"normalized_performance_data\\\" = VALUES(\\\"normalized_performance_data\\\"),\\\"previous_soft_state\\\" = VALUES(\\\"previous_soft
_state\\\"),\\\"check_attempt\\\" = VALUES(\\\"check_attempt\\\"),\\\"is_handled\\\" = VALUES(\\\"is_handled\\\"),\\\"state_type\\\" = VALUES(\\\"state_type\\\"),\\\"is_acknowledged\\\" = VA
LUES(\\\"is_acknowledged\\\"),\\\"long_output\\\" = VALUES(\\\"long_output\\\"),\\\"check_timeout\\\" = VALUES(\\\"check_timeout\\\"),\\\"last_state_change\\\" = VALUES(\\\"last_state_change
\\\"),\\\"check_commandline\\\" = VALUES(\\\"check_commandline\\\"),\\\"scheduling_source\\\" = VALUES(\\\"scheduling_source\\\"),\\\"next_update\\\" = VALUES(\\\"next_update\\\"),\\\"last_c
omment_id\\\" = VALUES(\\\"last_comment_id\\\"),\\\"execution_time\\\" = VALUES(\\\"execution_time\\\"),\\\"is_reachable\\\" = VALUES(\\\"is_reachable\\\"),\\\"performance_data\\\" = VALUES(
\\\"performance_data\\\"),\\\"is_problem\\\" = VALUES(\\\"is_problem\\\"),\\\"latency\\\" = VALUES(\\\"latency\\\"),\\\"is_sticky_acknowledgement\\\" = VALUES(\\\"is_sticky_acknowledgement\\
\"),\\\"host_id\\\" = VALUES(\\\"host_id\\\"),\\\"affects_children\\\" = VALUES(\\\"affects_children\\\"),\\\"check_source\\\" = VALUES(\\\"check_source\\\"),\\\"in_downtime\\\" = VALUES(\\\
"in_downtime\\\"),\\\"output\\\" = VALUES(\\\"output\\\"),\\\"previous_hard_state\\\" = VALUES(\\\"previous_hard_state\\\"),\\\"acknowledgement_comment_id\\\" = VALUES(\\\"acknowledgement_co
mment_id\\\"),\\\"next_check\\\" = VALUES(\\\"next_check\\\"),\\\"properties_checksum\\\" = VALUES(\\\"properties_checksum\\\"),\\\"environment_id\\\" = VALUES(\\\"environment_id\\\"),\\\"id
\\\" = VALUES(\\\"id\\\"),\\\"hard_state\\\" = VALUES(\\\"hard_state\\\"),\\\"soft_state\\\" = VALUES(\\\"soft_state\\\")\": Error 1105 (HY000): Parameter of prepared statement which is set 
through mysql_send_long_data() is longer than 'max_allowed_packet' bytes"
Sep 1 09:40:25 skinner1 icingadb[2362625]: config-sync: Aborted initial state sync after 5m2.602835192s
Sep 1 09:40:25 skinner1 icingadb[2362625]: Error 1105 (HY000): Parameter of prepared statement which is set through mysql_send_long_data() is longer than 'max_allowed_packet' bytes#012can't perform "INSERT INTO \"service_state\" (\"service_id\", \"is_flapping\", \"last_update\", \"severity\", \"normalized_performance_data\", \"previous_soft_state\", \"check_attempt\", \"is_handled\", \"state_type\", \"is_acknowledged\", \"long_output\", \"check_timeout\", \"last_state_change\", \"check_commandline\", \"scheduling_source\", \"next_update\", \"last_comment_id\", \"execution_time\", \"is_reachable\", \"performance_data\", \"is_problem\", \"latency\", \"is_sticky_acknowledgement\", \"host_id\", \"affects_children\", \"check_source\", \"in_downtime\", \"output\", \"previous_hard_state\", \"acknowledgement_comment_id\", \"next_check\", \"properties_checksum\", \"environment_id\", \"id\", \"hard_state\", \"soft_state\") VALUES (:service_id,:is_flapping,:last_update,:severity,:normalized_performance_data,:previous_soft_state,:check_attempt,:is_handled,:state_type,:is_acknowledged,:long_output,:check_timeout,:last_state_change,:check_commandline,:scheduling_source,:next_update,:last_comment_id,:execution_time,:is_reachable,:performance_data,:is_problem,:latency,:is_sticky_acknowledgement,:host_id,:affects_children,:check_source,:in_downtime,:output,:previous_hard_state,:acknowledgement_comment_id,:next_check,:properties_checksum,:environment_id,:id,:hard_state,:soft_state) ON DUPLICATE KEY UPDATE \"service_id\" = VALUES(\"service_id\"),\"is_flapping\" = VALUES(\"is_flapping\"),\"last_update\" = VALUES(\"last_update\"),\"severity\" = VALUES(\"severity\"),\"normalized_performance_data\" = VALUES(\"normalized_performance_data\"),\"previous_soft_state\" = VALUES(\"previous_soft_state\"),\"check_attempt\" = VALUES(\"check_attempt\"),\"is_handled\" = VALUES(\"is_handled\"),\"state_type\" = VALUES(\"state_type\"),\"is_acknowledged\" = VALUES(\"is_acknowledged\"),\"long_output\" = VALUES(\"long_output\"),\"check_timeout\" = VALUES(\"check_timeout\"),\"last_state_change\" = VALUES(\"last_state_change\"),\"check_commandline\" = VALUES(\"check_commandline\"),\"scheduling_source\" = VALUES(\"scheduling_source\"),\"next_update\" = VALUES(\"next_update\"),\"last_comment_id\" = VALUES(\"last_comment_id\"),\"execution_time\" = VALUES(\"execution_time\"),\"is_reachable\" = VALUES(\"is_reachable\"),\"performance_data\" = VALUES(\"performance_data\"),\"is_problem\" = VALUES(\"is_problem\"),\"latency\" = VALUES(\"latency\"),\"is_sticky_acknowledgement\" = VALUES(\"is_sticky_acknowledgement\"),\"host_id\" = VALUES(\"host_id\"),\"affects_children\" = VALUES(\"affects_children\"),\"check_source\" = VALUES(\"check_source\"),\"in_downtime\" = VALUES(\"in_downtime\"),\"output\" = VALUES(\"output\"),\"previous_hard_state\" = VALUES(\"previous_hard_state\"),\"acknowledgement_comment_id\" = VALUES(\"acknowledgement_comment_id\"),\"next_check\" = VALUES(\"next_check\"),\"properties_checksum\" = VALUES(\"properties_checksum\"),\"environment_id\" = VALUES(\"environment_id\"),\"id\" = VALUES(\"id\"),\"hard_state\" = VALUES(\"hard_state\"),\"soft_state\" = VALUES(\"soft_state\")"#012github.com/icinga/icinga-go-library/database.CantPerformQuery#012#011github.com/icinga/icinga-go-library@v0.8.2/database/utils.go:19#012github.com/icinga/icinga-go-library/database.(*DB).NamedBulkExec.func1.1.1.1#012#011github.com/icinga/icinga-go-library@v0.8.2/database/db.go:535#012github.com/icinga/icinga-go-library/retry.WithBackoff#012#011github.com/icinga/icinga-go-library@v0.8.2/retry/retry.go:65#012github.com/icinga/icinga-go-library/database.(*DB).NamedBulkExec.func1.1.1#012#011github.com/icinga/icinga-go-library@v0.8.2/database/db.go:530#012golang.org/x/sync/errgroup.(*Group).Go.func1#012#011golang.org/x/sync@v0.19.0/errgroup/errgroup.go:93#012runtime.goexit#012#011runtime/asm_amd64.s:1264#012retry deadline exceeded#012github.com/icinga/icinga-go-library/retry.WithBackoff#012#011github.com/icinga/icinga-go-library@v0.8.2/retry/retry.go:100#012github.com/icinga/icinga-go-library/database.(*DB).NamedBulkExec.func1.1.1#012#011github.com/icinga/icinga-go-library@v0.8.2/database/db.go:530#012golang.org/x/sync/errgroup.(*Group).Go.func1#012#011golang.org/x/sync@v0.19.0/errgroup/errgroup.go:93#012runtime.goexit#012#011runtime/asm_amd64.s:1264

```

---

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [September 1, 2026, 10:18am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/7 "2026-09-01T10:18:26Z")

</div>

We found out, that this is due to a check result being around 100 MB. The question is now, do we have to ensure that no result like this is possible across all check scripts, or are we missing a dial / switch somewhere?

---

<div class="post-metadata">

**Author:** ![apenning](https://community.icinga.com/user_avatar/community.icinga.com/apenning/32/7669_2.png) [@apenning](https://community.icinga.com/u/apenning)\
**Post date:** [September 1, 2026, 1:03pm UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/8 "2026-09-01T13:03:33Z")

</div>

Thanks for your detailed answer and pinning this down to an huge check result output. Honestly, even a two digit MB-check output is quite out of scope. Would it be possible to reduce the output?

---

<div class="post-metadata">

**Author:** ![moreamazingnick](https://community.icinga.com/user_avatar/community.icinga.com/moreamazingnick/32/7725_2.png) [@moreamazingnick](https://community.icinga.com/u/moreamazingnick)\
**Post date:** [September 1, 2026, 1:57pm UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/9 "2026-09-01T13:57:47Z")

</div>

just some information if you run into another problem

> [@meuthak](#):
>
> ```auto
> icingadb/icinga-jammy,now 1.5.1-8+ubuntu22.04 amd64 [installed]
> icingadb-redis/icinga-jammy,now 8.2.9-1+ubuntu22.04 amd64 [installed]
> icingadb-web/icinga-jammy,now 1.1.4-1+ubuntu22.04 all [installed]
> icingaweb2/icinga-jammy,now 2.12.7-1+ubuntu22.04 all [installed]
> 
> ```

be aware that your packages are not the latest.  
This is caused by the fact that icinga only supports the latest ubuntu LTS completely:

> - The latest version of each component receives full updates (new features, bug fixes, and security fixes).
> - The previous version of each component receives security updates only.
> - Older versions are no longer maintained.  
> [https://icinga.com/products/product-support-lifecycle/](https://icinga.com/products/product-support-lifecycle/)

your version of icingadb-web is 1.1.4 vs:

> **[Release Icinga DB Web v1.4.0 · Icinga/icingadb-web](https://github.com/Icinga/icingadb-web/releases/tag/v1.4.0)**
>
> All included changes can be found on the milestone.
> Breaking Changes
> 
> Raise minimum required PHP version to 8.2
> Change license to GPL-3.0-only #1338
> 
> New Features
> 
> Add support for PHP 8.5 (#1315)
> E...

currently ubuntu 24.04 packages are still up to date

---

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [September 2, 2026, 5:22am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/10 "2026-09-02T05:22:48Z")

</div>

Hello

@apenning it certainly is possible to reduce output size in this check script. We are just wondering if we have to ensure that no result bigger than the `max_allowed_packet` is ever created, or if there is somehow an auto truncate or something. Because not all our scripts are on a shared codebase, and maybe we are missing one which could resurface this problem again.

@moreamazingnick yeah, we are running a bit behind on upgrading all our servers, it’s in the pipeline, we haven’t got around it just yet.

Thanks for your answers 😁

---

<div class="post-metadata">

**Author:** ![bberg](https://community.icinga.com/letter_avatar_proxy/v4/letter/b/b782af/32.png) [@bberg](https://community.icinga.com/u/bberg)\
**Post date:** [September 3, 2026, 8:21am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/11 "2026-09-03T08:21:11Z")

</div>

@meuthak btw, i am curious. What kind of a check are you running that returns such a large plugin output?

---

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [September 3, 2026, 8:38am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/12 "2026-09-03T08:38:45Z")

</div>

It’s a Firewall Block Check, which just returns a condensed list of blocks in a timerange.

Large amount of blocks = Large Check Output.

---

<div class="post-metadata">

**Author:** ![apenning](https://community.icinga.com/user_avatar/community.icinga.com/apenning/32/7669_2.png) [@apenning](https://community.icinga.com/u/apenning)\
**Post date:** [September 3, 2026, 8:44am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/13 "2026-09-03T08:44:19Z")

</div>

> [@meuthak](#):
>
> We are just wondering if we have to ensure that no result bigger than the `max_allowed_packet` is ever created, or if there is somehow an auto truncate or something.

The `max_allowed_packet` is MySQL’s upper limit and you cannot exceed it. Of course, you can try to tweak this setting on your machines, but I am unsure how well this will play out. Please note that replicating huge columns will get expensive.

Further reading: [https://dev.mysql.com/doc/refman/8.4/en/replication-features-max-allowed-packet.html](https://dev.mysql.com/doc/refman/8.4/en/replication-features-max-allowed-packet.html)

Again, I would recommend checking your check plugins to ensure that the output is reasonable short or, at least, does not goes in the MBs. Icinga DB Web also truncates the output to ensure that the browser, or at least the tab, does not crash.

---

<div class="post-metadata">

**Author:** ![bberg](https://community.icinga.com/letter_avatar_proxy/v4/letter/b/b782af/32.png) [@bberg](https://community.icinga.com/u/bberg)\
**Post date:** [September 3, 2026, 8:51am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/14 "2026-09-03T08:51:53Z")

</div>

> [@apenning](#):
>
> Icinga DB Web also truncates the output to ensure that the browser, or at least the tab, does not crash.

For future reference: This can be configured and behaves as documented in [Configuration - Icinga DB Web](https://icinga.com/docs/icinga-db-web/latest/doc/03-Configuration/#available-settings-and-defaults)

`plugin_output_character_limit` defaults to 10000. There are sometimes good reasons to change it, some checks return valid HTML to the plugin output, which actually gets rendered then. And for this it might make sense to increase that value.

---

<div class="post-metadata">

**Author:** ![meuthak](https://community.icinga.com/letter_avatar_proxy/v4/letter/m/bc8723/32.png) [@meuthak](https://community.icinga.com/u/meuthak)\
**Post date:** [September 3, 2026, 8:58am UTC](https://community.icinga.com/t/icingadb-max-allowed-packet-exceeded/14414/15 "2026-09-03T08:58:34Z")

</div>

Clean summary of this topic:

- During sync from icingadb-redis into configured DB backend in icingadb, a crash CAN occur, when a statement or query exceeds the `max_allowed_packet` option from MySQL.
  - In my case, both times, icingadb had this error message:  
`Error 1105 (HY000): Parameter of prepared statement which is set`  
`through mysql_send_long_data() is longer than 'max_allowed_packet' bytes"`

- The query is normally retried a few times, before the crash occurs. Good indicator for an issue of this Redis to DB sync, is the Redis DB size on your Icinga Node.
  - 

```auto
redis-cli -p 6380 info | grep used_memory_human

```

  - To just get rid of the large data, which can not be synced, flush the Redis DB on your Icinga Node. This will cause a few missed events, because Icinga DB missed to sync them.  
`redis-cli -p 6380 flushall`

- To prevent this, make sure your plugin / check output are limited in size. There is no default way, to truncate or abbreviate a check result by icinga2.
  - Icinga DB Web truncates the output, but only when displaying on Icinga Web. Data is already stored in the Database at this moment.

I will remark this as the solution and close this discussion. Thank you for your answers.
