I know that it's important to run sc_reduce() first to filter some invalid scalar when I need a private key.
How about public keys in Monero? How can I ensure its format is right?
Please help to answer it, many thanks!
sc_reduce just takes a 256 bit number and reduces it mod 𝑙 to yield a scalar of the correct curve order. Nothing special. Any number between 0 and 𝑙 is a valid scalar.
For a point (which is what a public key is) however, the bytes need to first decode to a point and this can vary based on what format has been used to store the point. Assuming it's a compressed 32 byte point, this will consist of the
y coordinate (a scalar) combined with a 1 bit sign (used during recovery). Thus after taking the binary of the
y coordinate, one must recover the
x coordinate. This is a little more involved but covered perfectly well in [ref]. Having recovered
x one can then check the resulting
x,y point is actually on the curve with a check such as:
(-x*x + y*y - 1 - d*x*x*y*y) % q == 0. In practice, you will be using a tried-and-tested library to do any decoding / point operations.