Why/how does monero generate public ed25519 keys without using the standard public key generation provided by ed25519 libraries?

Question

Why/how does monero generate public ed25519 keys without using the standard public key derivation provided by ed25519 libraries?

Inspecting the code for mininero (https://github.com/monero-project/mininero/blob/master/mininero.py), it seems that it doesn't call the ed25519 publickey method to generate public keys. Instead, it defines its own public_key method that does this:

ed25519.encodepoint(ed25519.scalarmultbase(sk))

Mininero's copy of the ed25519 library (from http://ed25519.cr.yp.to/software.html) provides a function called "publickey", which appears to be identical in purpose to the python pynacl and warner/python-ed25519 libraries. All three produce the same resulting public keys. Neither of those other libraries appear to provide an alternative mechanism to derive a public key the way mininero does, nor do they provide any methods to mirror the functionality of mininero's ed25519.encodepoint. This leads me to believe that minero is deriving the public spendkey in an alternative way.

Why doesn't it just use the normal public key methods provided by these libraries? What is the reason for deriving the public key this way?

My hunch is that there is a reason to use a different public key generation system because the intent is to use ed25519 for more than just signing something. For background, I'm writing some monero address generation code as a learning project and I noticed that with the same seed, my public keys are not being generated the same way as valid monero code. My code takes the seed:

awning ramped obedient frown vaults voice dash sunken talent myriad soggy pumpkins buffet vigilant yields foggy wayside rabbits unplugs sarcasm behind lopped tycoon uttered frown

Which produces a spendkey of

77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09

My code uses python libraries to generate an ed25519 public spend key. I wrote the code to use either https://github.com/pyca/pynacl or https://github.com/warner/python-ed25519 , which both produce the same result. For my spendkey above:

9235e2ecf938d462e79fe993bfdb1717353c87a0b0ed7cb1d58c9692035336d0

This doesn't match what the mininero code and https://xmr.llcoins.net/addresstests.html produce for the same spendkey. They both produce 0f3b913371411b27e646b537e888f685bf929ea7aab93c950ed84433f064480d

Here is a snippet of code that shows the 9235e2ecf938d462e79fe993bfdb1717353c87a0b0ed7cb1d58c9692035336d0 generated public key (ed25519 from warner/python-ed25519):

import ed25519
import binascii
spendkey_hex = b'77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09'
sk = binascii.unhexlify(spendkey_hex)

def sc_reduce32(n):
    n = int.from_bytes(n, byteorder='little')
    l = (2**252 + 27742317777372353535851937790883648493)
    reduced = n % l
    newbytes = reduced.to_bytes(32, 'little')
    return newbytes


reduced_sk = sc_reduce32(sk)
sec = ed25519.SigningKey(reduced_sk)
pub = sec.get_verifying_key()

print('pub key: ' + str(pub.to_ascii(encoding="hex")))

Here is a snipped of code that produces 9235e2ecf938d462e79fe993bfdb1717353c87a0b0ed7cb1d58c9692035336d0 using ed25519 from the mininero copy of ed25519.py

import ed25519 as ed25519
import binascii
spendkey_hex = b'77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09'
mininero_pub_spend = ed25519.publickey(binascii.unhexlify(spendkey_hex))
print('pub key: ' + str(binascii.hexlify(mininero_pub_spend)))

Answer 1

Monero doesn't use EdDSA, which all of those libraries are specifically set up for. We don't use SHA512 at all, but rather Keccak (~SHA3). We don't use secret keys as seeds like EdDSA does, but rather as scalars. If you look at ed25519.py on L63, you can see what I'm talking about.

Change the function to look like this:

def publickey(sk):
  a = decodeint(sk)
  A = scalarmult(B,a)
  return encodepoint(A)

Now you'll get the correct result: 0f3b913371411b27e646b537e888f685bf929ea7aab93c950ed84433f064480d. I use binascii.hexlify to get the output converted to hex (and also unhexlify to use the secret key as input to publickey(sk)).

Answer 2

Worth pointing out these command level piped facts with the context up top:

1% echo 77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09 | ./25519 9235e2ecf938d462e79fe993bfdb1717353c87a0b0ed7cb1d58c9692035336d0

2% echo 77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09 | bx sha512 | cut -c 1-64 | ./clamp | ./sc_reduce32 | ./secret_key_to_public_key 9235e2ecf938d462e79fe993bfdb1717353c87a0b0ed7cb1d58c9692035336d0

3% echo 77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09 | bx sha512 | cut -c 1-64

b359d96c1864b19ba68bb72f474b459294fa4938c3f03cd5260141ddd7349915

4% echo 77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09 | bx sha512 | cut -c 1-64 | ./clamp

b059d96c1864b19ba68bb72f474b459294fa4938c3f03cd5260141ddd7349955

Is how Monero converts a private key to a public key:

5% echo b059d96c1864b19ba68bb72f474b459294fa4938c3f03cd5260141ddd7349955 | ./sc_reduce32 | ./secret_key_to_public_key

9235e2ecf938d462e79fe993bfdb1717353c87a0b0ed7cb1d58c9692035336d0

6% echo 77fadbe52830d30438ff68036374c0e3fb755d0d983743bcbfb6a45962f50a09 | ./sc_reduce32 | ./secret_key_to_public_key

0f3b913371411b27e646b537e888f685bf929ea7aab93c950ed84433f064480d

Additional Points:

1. ./25519 utilizes libsodium's crypto_sign_ed25519_sk_to_seed() and crypto_sign_seed_keypair() functions.
2. ./sc_reduce32 utilizes libsodium's crypto_core_ed25519_scalar_reduce() function.
3. ./secret_key_to_public_key utilizes libsodium's crypto_scalarmult_ed25519_base_noclamp() function.
4. ./clamp performs this:

  // Big Endian Clamping
  // https://quirks.ed25519.info/basics/
  // % echo 7ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff8 | ./endian32
  //        f8ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f  
  // https://moderncrypto.org/mail-archive/curves/2017/000861.html; 64 = 0x40

  output[0]  &= (unsigned char)0xf8;
  output[31] &= (unsigned char)0x7f;
  output[31] |= (unsigned char)0x40;

5. % echo 9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60 | ./25519 d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a matches results from https://www.rfc-editor.org/rfc/rfc8032#section-7.1

Everything looks very Kosher using independent sources, and realizing that Monero was designed to utilize Keccak hashing instead of SHA2-256/512 in case SHA2 bugs are discovered that will impact BTC. That is a reason why a RFC 8032 form of ed25519 is not used directly.