How to do sign or hash larger messages (greater than 32 bytes)?

[ebuchman]

The example in the playground and all examples I've looked at both for signing and hashing assume the message being signed or hashed is a single frontend.Variable and thus 32 bytes or less.

How can the example be extended to support a message longer than 32 bytes?

For signing it seems the msg being signed must be 32 bytes anyways, and so it should be hashed first, but that still leaves the question of how to handle a pre-image larger than 32 bytes?

There's a cryptic comment on the Sign method about using fr.Hash first, but I couldn't find an example or figure out how that was supposed to work: https://github.com/Consensys/gnark-crypto/blob/da0317fd013308db6ce847bc9c3d506a2a3ae0ff/ecc/bn254/twistededwards/eddsa/eddsa.go#L118-L121

So how best to get a pre-image larger than 32-bytes into the witness?

It seems I might be able to use a fixed length array, ie. Message [1000]frontend.Variable, but all elements need to be populated and this seems wasteful if the message is much smaller than that. Another possibility might be using [1000]uints.U8, but I couldn't get that working either.

Is there an example of how best to do this somewhere? Apologies if I've missed something obvious.

Thanks!

[ivokub]

So, there is essentially two options:

1. decompose the input to be hashed base-modulus representation (input = \sum input_i modulus^i). Then your input would be defined as [maxLength]frontend.Variable
2. decompose the input to be hashed into byte representation (input = \sum inputByte_i (2^8)^i). Then your input would be defined as [maxLength]uints.U8

Now, depending on which approach you take, you have different options for hashing the input. For 1) you can use algebraic hash function a la MiMC as it internally works on field elements. The advantage of this approach is that the hashing is quite efficient (a few hundred constraints in R1CS for every input).
For 2) you can use a binary hash function a la sha2. It is quite inefficient (several tens of thousands of constraints per sha2 block 64 bytes), but allows to have quite good compatibility with out-circuit hash functions.

My recommendation, if it works for you, is to use MiMC and base-modulus representation due to the efficiency.

And as you said, we need to allocate the space for maximal possible input size and then zero-pad unused inputs. This is a deficiency of a SNARK as the circuit is fixed for a proving/verification key and there is now good way around it. By using PLONK recursion, we can mitigate the issue a bit (we have several hashing circuits for inputs of different lengths and then in the second layer of recursion switch between the corresponding verification key), but as this has quite a lot of overhead, then it makes only sense if the rest of circuit logic is complex enough that recursive PLONK verification is cheap compared to performing no-op hashing.