1

I'm syncing a node right now from scratch. It seems to be going slower than I expected. CPU/Memory/IO/Network all do not seem fully utilized. What is the limiting factor here?

My assumption is it might be it's doing a lot of random IO which is a lot slower than sequential IO.

  • I am seeing the same thing. Network is well below 1Mb/s of 96 Mb/s (per speedtest-cli), Disk is ~300 KB/s of 17.1 MB/s (using dd), CPU ~25% (renice -10), RAM ~25%. Peers: 36 (47 connections). Only thing I can think of is that peers are just very slow? – Jonathan Cross Oct 12 at 16:18
  • It was significantly faster with a NVME ssd. So highly likely to be random IO bottlenecked. – Chris Stryczynski Oct 12 at 17:35
  • Yes, this was mentioned by selsta and others. – Jonathan Cross Oct 12 at 21:25
0

Possible causes explained to me in #monero:matrix.org:

  1. Verification tends to be limiting factor.
  2. Using an HDD is known to be significantly slower than SSD/NVMe.
  3. Apparently HDD seek time is a significant bottleneck.
  4. Verification operations are not very parallelizable therefore if a single CPU core is spiked to 100%, that could be limiting factor, even if others are idle. Monero code doesn't seem to properly utilize multiple CPU cores.
  5. A single slow peer might be able to slow things down overall.
  6. Wallet syncing at the same time as blockchain syncing -> very slow.

Notes:

  • I was just syncing the most recent few thousand blocks.
  • I was using an HDD rather than SSD/NVMe, so my bottleneck may not apply to you.

[1] Per selsta: "yes, because bottleneck is verification not network speed"

[2] Per selsta: "yes, HDD will always be extremely slow during verification."

[3] Per moneromooo: "It's seek time that matters, not contiguous r/w speed."

[4] Per Inge-: "disk queue length is indicative (while transfers/sec will be very low for random reads on a HDD) and if any single cpu core is pinned at 100% (overall CPU utilization could be very low as there is a limit to how parallelizable the workload is))" and "the number of cores that can be utilized are limited. I would expect parallel verification of transactions is possible, while parallel validation of blocks is more difficult"

Q: Wouldn't users complain about multiple cores not being used?
A: hyc: "you probably would never have noticed. I have 8 cores and pretty much never see more than 4 cores used
there's definitely something artificially limiting throughput
I just don't know how to pinpoint what it is"

[5] Per moneromooo: "If none is used, it's possibly waiting on data from a peer. If that peer's busy, it might wait a bit before sending."

[6] Per moneromooo.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.