Comment 30 for bug 312536

Responding to a lot of things in this bug...

1) Is Mozilla willing to disable MD5 cert signatures and "break the web"
unilaterally, even if it drives users to IE?

Johnathan, as Mozilla's representative to CABForum, have you broached this
subject with the other browser representatives there?

I have so many things on my plate now that I must prioritize my work
carefully. Until I see that either
a) the browsers are willing to act in concert on this, and collectively
announce a date on which they will desupport it, or
b) someone speaking for Mozilla says here that Mozilla is willing to
unilaterally "break the web",
I don't think this bug is really urgent.

2) Error code

Whatever error code is set by CERT_VerifySignedDataWithPublicKey is likely
to be overridden by the caller, (see examples below) so I wouldn't fret too
much about which error code it is.
http://mxr.mozilla.org/security/source/security/nss/lib/certdb/crl.c#1565
http://mxr.mozilla.org/security/source/security/nss/lib/certhigh/certvfy.c#180
http://mxr.mozilla.org/security/source/security/nss/lib/certhigh/certvfy.c#635
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11nobj.c#793

3) hash configuration API design

If we're going to make this configurable, then clearly we don't want to
have MD5 be a special case. We want it to be possible to configure all
the hashes. The configuration API needs to allow the caller to specify
which hash it wants to enable/disable.

Then there's the matter of the level of granularity of functional uses
that will be controlled by this API. Presently, we're proposing to be
able to control the acceptability of hashes in the context of verifying
signatures on certificates and CRLs (and probably OCSP responses).

Are those the only operations for which we will ever want to exert such control?

What about:
- SMIME signatures?
- signatures on JAR and XPI files?
- disabling MD5 alltogether?

Here's a first draft proposal for an API.

Two functions, a set and a get. The hash algorithm is specified using a
SECOidTag (effectively an enum, whose value maps to/from OIDs). The data
associated with each SecOidTag is a 32-bit value treated as a bit array.
Initially, only 1 bit is defined, bit 0 (value 0x00000001) which specifies
the use in signatures of certs and CRLs. The API is extensible in that
we have 31 more bits to define as we think of uses for them.

#define NSS_ALG_USABLE_IN_CERT_SIGNATURE 0x00000001

extern SECStatus
NSS_SetAlgorithmPolicy(SECOidTag tag, PRUint32 mask, PRUint32 value);

extern SECStatus
NSS_GetAlgorithmPolicy(SECOidTag tag, PRUint32 *pValue);

Either function may return SECSuccess or SECFailure with the error code
set to SEC_ERROR_UNKNOWN_OBJECT_TYPE if the SECOidTag is out of range.

The Get function outputs the 32-bit value associated with the SECOidTag.
The Set function modifies the stored value according to the following
algorithm:
   policy[tag] = (policy[tag] & mask) | value;

Default value for any policy would be 0xffffffff (enabled for all purposes).

How's that sound?