27
I wasn't around when the decision to use LMDB was made, but I can present some obvious reasons why such a decision makes sense:
1) at the time they made their decision, there were only 2 embedded key value stores out there with ACID transaction support: BerkeleyDB and LMDB. (RocksDB has recently added it. I haven't tested it to see if it's actually usable. ...
15
LMDB is a fast database that is a lot less brittle than the typical-within-cryptocurrencies leveldb. The only drawback which called for a Berkeley DB fallback was its patchy 32 bit support, but that has now been fixed. Moreover, hyc (LMDB author) is working with the Monero team to ensure LMDB is used the best way. I believe it was Monero that found hyc first,...
15
You are probably looking at a database file with the blockchain in it. The main variation cause is that the database library Monero uses, LMDB, preallocates some space for writing. Every so often, when that space falls too low, LMDB will resize the file to get more breathing space.
Some OSes do not support sparse files - that is, files with "holes without ...
12
Monero transaction confirmations are very quick on modern computers. With LMDB estimates from developers Smooth and NoodleDoodle are that Monero can handle 1700 TPS (up from 1600 TPS pre LMDB).
Confirmation time is definitely not the limiting factor. Many nodes do not have the bandwidth to support 1700 TPS based on current transaction sizes.
Monero can ...
10
RingCT
RingCT was developed by Monero Research Labs specifically for coins with ring signatures (and even more specifically Monero). Monero is by far the most common of these, and Monero is the first to include this in their code. Keep in mind that RingCT was based on Confidential Transactions for Bitcoin, so a coin like Bitcoin would implement CT instead.
...
9
As of v0.10.0, yes the LMDB files are cross-compatible between 32 and 64bit architectures. They have always been cross-compatible between OSs. They are still byte-order dependent but almost everyone uses little-endian CPUs these days so it's not much of an issue.
8
In general, LMDB files are architecture-dependent and cannot be moved between different architectures.
In Monero, all LMDB structures are 64-bit clean, so the files are portable between 32 and 64 bit architectures. But they are still endian-dependent, and are not portable across endianness. This hasn't been a concern since the two dominant architectures ...
8
If you're using python, the py-lmdb module includes a basic CLI. http://lmdb.readthedocs.io/en/release/
Keep in mind that you'll be getting raw data back, not anything in a human-readable format.
8
For performance reasons, LMDB asks the OS to preallocate its storage space in large chunks, instead of growing incrementally as new records are added. Periodically this space must be increased as the actual usage increases.
8
This will not work. VirtualBox shared folders don't support mmap, which is required for LMDB (and BDB).
7
Security wise, it's the same, assuming you do not disable verification on the conversion process (which speeds up a lot, but then if there are problems, they won't be detected).
Speedwise, it varies. Typically, syncing is faster, as this can do some things in parallel. However, if your internet download speed is pretty slow, or if you can't download much (e....
7
The previous answers are correct, but there are plenty of other reasons for size variations. The timing and order in which records are written will also affect the way space is allocated in the database, and if the two nodes didn't run in lockstep then they'll get batches of blocks at different times as well as alternate blocks at different times.
7
Probably because Windows, by default, reports file sizes in KiB/MiB/GiB/TiB (1024 bytes in a kilobyte, 1024 kilobytes in a megabyte, etc.), while Ubuntu, by default reports file sizes using the "proper" SI definition of those prefixes (1000 bytes in a kilobyte, 1000 kilobytes in a megabyte, etc.).
Here's some website I Googled, with pictures of the two OSes ...
6
rsync copies what's changed:
rsync ~/.bitmonero/lmdb/data.mdb /mnt/wherever/monero-blockchain/data.mdb
This should be done when nothing is writing to the blockchain database file. If you need to copy while it is being written to, use mdb_copy:
mdb_copy ~/.bitmonero/lmdb /mnt/wherever/monero-blockchain/
Note how mdb_copy wants the directory, and not ...
6
I personally use an external hard drive to store the blockchain and get virtual box to capture the device so it can be recognised by the VB OS. It works very well as an alternative to storing the blockchain on the virtual OS itself.
6
Adding to the previous answer, those TPS estimates assume an Intel i7 2600k quad core processor. So roughly 400 TPS per core. That processor is from 2011, so I assume that this figure will improve with newer processors, as well scale with more cores. Note that these are just tx verification times, bandwidth is a whole different issue.
I would say that ...
6
It appears that you're using a 32bit build. This was a known bug in the 32bit LMDB code, Issue #855, fixed on August 11 in commit 7442dd084acfcef571c51f53277247c050d1c98b.
5
Bandwidth is currently the limiting factor.
Assuming an average transaction size of around 15 KiB, nodes can handle ~8 tx/s with every Mbit/s connection speed.
There is no real answer to how many transactions per second the network can "theoretically" handle though. A higher transaction rate means higher requirements, whether it's bandwidth, CPU or disk ...
5
I don't think there is an equivalent of the mysql or psql commands for lmdb, therefore to make custom queries to the database you must write a program that accesses it directly.
There are some examples at github.com/moneroexamples:
github.com/moneroexamples/lmdbcpp-monero
github.com/moneroexamples/access-blockchain-in-cpp
github.com/moneroexamples/finding-...
5
If your daemon was interrupted by a power failure or OS crash, and it was still trying to catch up to the network (as opposed to already being synchronized), there's a chance the DB got corrupted. If you get a DB error after starting after a crash, you should try using this flag. You should only use this flag after you've already seen a DB error that causes ...
5
The use case I had in mind when I wrote this code is to run daemons on different network interfaces, or on different network protocols, using the same database. E.g., if you want to use different bandwidth limits on one interface vs another, or if you want to run a daemon on TOR simultaneously with a daemon on clearnet. Or, you want to run a public node (...
4
It is likely that this means blockchain database (LMDB) corruption.
The easiest way to fix it is to rename your corrupt lmdb folder to something else, for example:
Rename C:\ProgramData\bitmonero\lmdb to C:\ProgramData\bitmonero\lmdb-old and run monerod again.
It should start without errors and it will again create a fresh copy of C:\ProgramData\bitmonero\...
4
Understanding the structure of Monero's LMDB and how explore its contents using mdb_stat
Firstly, mdb_stat is a tool to get status information about the database environment, not to explore its contents. Quoting the description directly from the mdb_stat manual:
The mdb_stat utility displays the status of an LMDB environment.
I am assuming querying it ...
3
There was a bug in the ARMv7-specific source code, which was triggered when using newer versions of GCC to compile the code.
https://github.com/monero-project/monero/issues/1991
It has recently been fixed.
3
The .raw file is "raw" as the name implies, meaning it should be platform-agnostic, ie, the data inside is the bare minimum, unoptimized, blockchain data, as it flies over the network, and nothing else. Think of it as just copying and pasting block data one on top of another and saving this into a file. The .raw file, then, is just a collection of all the ...
3
It might work. Edit: As per hyc's comment, it will work fine.
You can safely try as it would properly trigger an error.
Here is a detailled answer.
3
While getting a bad block on top should not be possible in the first place, it could happen in case of crashes, etc, on some systems.
Popping one or more blocks can be done with blockchain_import, eg: blockchain_import --pop-blocks 1 will pop the last block. However, if the blockchain got corrupted somehow, this might not work.
Last, note that the message ...
2
Unless you had --fast-block-sync 0 as a command-line flag, it should've already been skipping both POW checks and ring signature checks. With the transaction rates we've had in much of the coin's history, checking POW actually takes longer than signatures.
You might also try bumping --db-sync-mode arg (=fast:async:1000) to something higher (10000?), and ...
2
The default ntfs driver in Linux doesn't support writes. NTFS support on Linux is generally pretty awful.
2
With 0.11 (and possibly 0.10.3.1, though I'm not 100% certain this was ready then), there is no particular precautions to take in order to share a database between two daemons. Pointing the daemons to the same directory using --data-dir should be safe. I'm not sure this has been tested a lot, however, so if anything like corruption should happen, please ...
Only top voted, non community-wiki answers of a minimum length are eligible
