Node no longer syncs --Failed to query m_blocks: MDB_BAD_TXN--
Dear all,

my node v0.18.2.2 on FreeBSD no longer syncs from block 2890276.

Last good log messages are


<pre>
2023-05-31 13:00:40.589 I SYNCHRONIZATION started
2023-05-31 13:00:42.269 I Synced 2890196/2897900 (99%, 7704 left)
2023-05-31 13:00:49.001 I Synced 2890216/2897900 (99%, 7684 left)
2023-05-31 13:01:21.513 I Synced 2890236/2897900 (99%, 7664 left)
2023-05-31 13:01:42.105 I Synced 2890256/2897900 (99%, 7644 left)
2023-05-31 13:02:06.275 I Synced 2890276/2897900 (99%, 7624 left)
</pre>

and then come error messages:
<pre>
2023-05-31 13:02:13.651 W Failed to query m_blocks: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
2023-05-31 13:02:13.652 E Error adding transaction to txpool: Error adding txpool tx metadata to db transaction: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
2023-05-31 13:02:13.652 W Failed to query m_blocks: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
2023-05-31 13:02:13.652 E Exception at [core::handle_incoming_txs()], what=Failed to query m_blocks: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
2023-05-31 13:02:13.653 W Failed to commit a transaction to the db: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
2023-05-31 13:02:13.653 E Exception in cleanup_handle_incoming_blocks: Failed to commit a transaction to the db: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
</pre>

I can rollback a zfs-snapshot to a previous, well-known state. And when restarting monerod, it fails at same point.

Here is some debug output with loglevel=2:

<pre>
2023-05-31 12:03:03.768 I Synced 2890276/2897866 (99%, 7590 left) (17.778047 sec, 1.124983 blocks/sec), 90.812469 MB queued in 63 spans, stripe 2 -> 2: [mooooooooooooooooooo
oooooooooo.ooooooooooooooooo.oooooooooooo.ooo.o.....]
2023-05-31 12:03:03.768 D [130.61.138.150:18080 OUT]  next span in the queue has blocks 2890276-2890295, we need 2890276
2023-05-31 12:03:03.768 D block <68cccc801b0e41d0a53c82c61178b4d8172512dc97c2b7a1f8659e2e0db1b7ac> found in main chain
2023-05-31 12:03:03.768 D [get_estimated_batch_size] m_height: 2890276  block_start: 2889776  block_stop: 2890275
2023-05-31 12:03:03.768 D estimated average block size for batch: 79408
2023-05-31 12:03:03.768 D calculated batch size: 1786680064
2023-05-31 12:03:03.768 D increase size: 1786680064
2023-05-31 12:03:03.768 D DB map size:     69434211328
2023-05-31 12:03:03.768 D Space used:      60983902208
2023-05-31 12:03:03.768 D Space remaining: 8450309120
2023-05-31 12:03:03.768 D Size threshold:  1786680064
2023-05-31 12:03:03.768 D Percent used: 87.8298  Percent threshold: 90.0000
2023-05-31 12:03:03.769 D block_batches: 20

...

2023-05-31 12:03:10.394 I +++++ BLOCK SUCCESSFULLY ADDED
2023-05-31 12:03:10.394 I id:   <2a397749c384e45f64963face6d03981a716106a3f03e1100992c2d03cf95ecf>
2023-05-31 12:03:10.394 I PoW:  <0c86e813ba99b070a4a26bf257e3ac0ac9d428589939fba464a9120200000000>
2023-05-31 12:03:10.394 I HEIGHT 2890282, difficulty:   310732759729
2023-05-31 12:03:10.394 I block reward: 0.603569520000(0.600000000000 + 0.003569520000), coinbase_weight: 102, cumulative weight: 139070, 27(0/22)ms
2023-05-31 12:03:10.395 D Invalidating block template cache
2023-05-31 12:03:10.729 D Mixin: 15-15
2023-05-31 12:03:10.731 D RCT cache: tx <7a25f171e8ee00bd506eb26439ca688bca34425091787ae89f7f9c8574d93b90> missed
2023-05-31 12:03:10.736 I Transaction added to pool: txid <7a25f171e8ee00bd506eb26439ca688bca34425091787ae89f7f9c8574d93b90> weight: 1529 fee/byte: 4e+06
2023-05-31 12:03:10.736 D tx added: <7a25f171e8ee00bd506eb26439ca688bca34425091787ae89f7f9c8574d93b90>

...

2023-05-31 12:03:11.638 D RCT cache: tx <534310b1487e11afd1194fd26a68b464e904eba7869e09a8f9e4db92ceb78df6> missed
2023-05-31 12:03:11.643 I Transaction added to pool: txid <534310b1487e11afd1194fd26a68b464e904eba7869e09a8f9e4db92ceb78df6> weight: 1527 fee/byte: 20000
2023-05-31 12:03:11.643 D tx added: <534310b1487e11afd1194fd26a68b464e904eba7869e09a8f9e4db92ceb78df6>
2023-05-31 12:03:11.643 D Mixin: 15-15
2023-05-31 12:03:11.643 D RCT cache: tx <1699466d89f5ca0ce4f1c334ccbbdd409dd1fca7e033acedd4e35823f84d8689> missed
2023-05-31 12:03:11.648 I Transaction added to pool: txid <1699466d89f5ca0ce4f1c334ccbbdd409dd1fca7e033acedd4e35823f84d8689> weight: 1533 fee/byte: 20000
2023-05-31 12:03:11.648 D tx added: <1699466d89f5ca0ce4f1c334ccbbdd409dd1fca7e033acedd4e35823f84d8689>
2023-05-31 12:03:11.648 D Mixin: 15-15
2023-05-31 12:03:11.649 W Failed to query m_blocks: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
2023-05-31 12:03:11.649 E Output does not exist! amount = 0, absolute_offset = 37199366
2023-05-31 12:03:11.649 E Failed to get output keys for tx with amount = 0.000000000000 and count indexes 16
2023-05-31 12:03:11.649 E Failed to check ring signature for tx <399cb54881df31a8037fce32751ba4e67232cd39602e17491ab5d9655672089c>  vin key with k_image: <b53a615b53c85bb7ff
b1f5d354b7902a653abb9c8835a6886ed315a735b3bedd>  sig_index: 0
2023-05-31 12:03:11.649 E   *pmax_used_block_height: 0
2023-05-31 12:03:11.649 I Error adding txpool tx metadata to db transaction: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
2023-05-31 12:03:11.649 E Error adding transaction to txpool: Error adding txpool tx metadata to db transaction: MDB_BAD_TXN: Transaction must abort, has a child, or is invalid
</pre>

Any advice on how to work around this would be appreciated.

Thanks


BR

rothisbi