I have been running my own Monero Node (Monerod) for some weeks now and it worked perfectly. However, today my Node get stuck in the constant sync limbo at exactly 99.9%. It was always 2 blocks behind the network:
status
Height: 2246208/2246210 (99.9%) on mainnet, not mining, net hash 1.64 GH/s, v14, 12(out)+0(in) connections, uptime 0d 9h 12m 6s
I am running a little bit of selfish Node for my own needs with no incoming connections allowed. Run attributes for my Monerod are as following:
--restricted-rpc --rpc-login <DummyText>:<DummyText> --log-level=1 --enforce-dns-checkpointing --no-zmq --no-igd --hide-my-port --in-peers=0 --check-updates disabled --max-concurrency 2
While investigating an issue I have noticed that my Node is surprisingly always and exactly 2 blocks behind due to some subset of peers - lets call them "+2 peers" for the clarity:
sync_info
Height: 2246208, target: 2246210 (99.9999%)
Downloading at 32 kB/s
12 peers
148.251.195.244:18080 04988397a2eac5ca normal 0 2246208 0 kB/s, 0 blocks / 0 MB queued
192.99.154.164:17085 fa9df573098b9b52 synchronizing 0 2246210 16 kB/s, 0 blocks / 0 MB queued
167.172.150.102:18080 90034b2baf0e81c6 normal 0 2246208 0 kB/s, 0 blocks / 0 MB queued
144.217.241.28:18080 f76d3b057c41fc4e synchronizing 0 2246210 16 kB/s, 0 blocks / 0 MB queued
97.126.77.35:18080 ab9fbcf12b6fa412 normal 0 1 0 kB/s, 0 blocks / 0 MB queued
94.23.23.52:18080 e0214630fbeebc70 normal 0 2246208 0 kB/s, 0 blocks / 0 MB queued
144.91.122.162:18080 e74c7910ba360ee4 normal 0 2246208 0 kB/s, 0 blocks / 0 MB queued
51.79.52.228:9505 9dc3992c92fd4c83 synchronizing 0 2246210 0 kB/s, 0 blocks / 0 MB queued
37.191.201.68:18080 e40ee8258f2bed16 normal 0 2246208 0 kB/s, 0 blocks / 0 MB queued
216.130.231.18:18080 c9954b06032f233a normal 0 2246208 0 kB/s, 0 blocks / 0 MB queued
54.39.75.65:4698 2b932a39cfb437aa synchronizing 0 2246210 0 kB/s, 0 blocks / 0 MB queued
83.212.105.141:18080 9a4db716d08250b9 normal 0 2246208 0 kB/s, 0 blocks / 0 MB queued
0 spans, 0 MB
[]
In this output, majority of nodes are on block height: 2246208, while a subset is at block height: 2246210. The problems is that Nodes with +2 block height (2246210) are not synchronizing their blocks to the rest. I have checked the Network status with https://xmrchain.net/ and see that most likely "+2 peers" are not in consensus with the rest. In other words block height of 2246208 was the valid block height visible on XMR Chain Block Explorer as well.
I tried to ban the "+2 peers" and after an hour or so, I got the following results:
bans
137.74.189.88 banned for 79234 seconds
142.44.144.243 banned for 79417 seconds
193.70.78.196 banned for 85348 seconds
51.178.251.7 banned for 83963 seconds
51.210.96.211 banned for 82505 seconds
51.79.11.199 banned for 82797 seconds
51.79.49.41 banned for 84239 seconds
51.79.50.44 banned for 85744 seconds
51.79.54.168 banned for 80803 seconds
54.39.75.73 banned for 85133 seconds
87.98.229.114 banned for 84704 seconds
94.23.147.238 banned for 79261 seconds
After banning those nodes, everything went back to normal. I was able to properly synchronize Wallets and even make test Tx without any problems. However, those "+2 peers" which do not synchronize their extra 2 blocks were keep coming back and I had to keep banning them to make it work again. As you can see there is a lot of 51.79.XX.XX IP addresses in the problematic group.
I just do not know what the heck is happening. What is the problem here and how to solve it without constant banning?
Thank in advance for your help!
