PCKS11 security provider not working

should work when configured with --enable-nss, however the tests never did succeed. Seen this forever on every Debian/Ubuntu build. However keytool is able to import the certificate with SHA384withECDSA signatures (see bug #356).

FAILED: com/sun/crypto/provider/KeyFactory/TestProviderLeak.java
FAILED: java/security/KeyPairGenerator/Failover.java
FAILED: sun/security/pkcs11/ec/ReadCertificates.java
FAILED: sun/security/pkcs11/ec/ReadPKCS12.java
FAILED: sun/security/pkcs11/ec/TestCurves.java
FAILED: sun/security/pkcs11/ec/TestECDH.java
FAILED: sun/security/pkcs11/ec/TestECDSA.java
FAILED: sun/security/pkcs11/ec/TestECGenSpec.java
FAILED: sun/security/pkcs11/ec/TestKeyFactory.java
FAILED: sun/security/pkcs11/sslecc/ClientJSSEServerJSSE.java
FAILED: sun/security/pkcs11/tls/TestPRF.java
FAILED: sun/security/ssl/javax/net/ssl/NewAPIs/SSLEngine/TestAllSuites.java
FAILED: sun/security/ssl/sanity/ciphersuites/CheckCipherSuites.java
FAILED: sun/security/ssl/sanity/interop/ClientJSSEServerJSSE.java

just turning off security.provider.9 in java.security lets the sun/security/ssl/ tests succeed.

Created attachment 325
jtr files

Replicated here.

With some more debugging on the ReadCertificates test:

Loading sunlabscerts.pem...
----------System.err:(49/3120)----------
encodedPoint: [4, 41, 4, 74, 38, 59, 63, 127, -83, 45, 42, -32, -28, -123, -38, 19, -10, -34, 31, 2, -95, -72, -70, -99, -5, 101, \
62, 91, -32, -87, 87, 35, -89, -21, -25, -119, -58, -70, -63, 118, 124, 77, -125]
encodedParams: [6, 5, 43, -127, 4, 0, 8]
java.security.cert.CertificateParsingException: java.io.IOException: subject key, Could not create EC public key
        at sun.security.x509.X509CertInfo.<init>(X509CertInfo.java:171)
        at sun.security.x509.X509CertImpl.parse(X509CertImpl.java:1747)
        at sun.security.x509.X509CertImpl.<init>(X509CertImpl.java:320)
        at sun.security.provider.X509Factory.parseX509orPKCS7Cert(X509Factory.java:550)
        at sun.security.provider.X509Factory.engineGenerateCertificates(X509Factory.java:434)
        at java.security.cert.CertificateFactory.generateCertificates(CertificateFactory.java:444)
        at ReadCertificates.readCertificates(ReadCertificates.java:51)
        at ReadCertificates.main(ReadCertificates.java:86)
        at PKCS11Test.premain(PKCS11Test.java:79)
        at PKCS11Test.testDefault(PKCS11Test.java:113)
        at PKCS11Test.main(PKCS11Test.java:86)
 at ReadCertificates.main(ReadCertificates.java:57)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at com.sun.javatest.regtest.MainAction$SameVMThread.run(MainAction.java:595)
        at java.lang.Thread.run(Thread.java:636)
Caused by: java.io.IOException: subject key, Could not create EC public key
        at sun.security.x509.X509Key.parse(X509Key.java:174)
        at sun.security.x509.CertificateX509Key.<init>(CertificateX509Key.java:75)
        at sun.security.x509.X509CertInfo.parse(X509CertInfo.java:705)
        at sun.security.x509.X509CertInfo.<init>(X509CertInfo.java:169)
        ... 17 more
Caused by: java.security.InvalidKeyException: Could not create EC public key
        at sun.security.x509.X509Key.buildX509Key(X509Key.java:227)
        at sun.security.x509.X509Key.parse(X509Key.java:170)
        ... 20 more
Caused by: java.security.spec.InvalidKeySpecException: Could not create EC public key
        at sun.security.pkcs11.P11ECKeyFactory.engineGeneratePublic(P11ECKeyFactory.java:154)
        at java.security.KeyFactory.generatePublic(KeyFactory.java:321)
        at sun.security.x509.X509Key.buildX509Key(X509Key.java:223)
        ... 21 more
Caused by: java.security.InvalidKeyException: Could not create EC public key
        at sun.security.pkcs11.P11ECKeyFactory.implTranslatePublicKey(P11ECKeyFactory.java:117)
        at sun.security.pkcs11.P11ECKeyFactory.engineGeneratePublic(P11ECKeyFactory.java:152)
        ... 23 more
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_DOMAIN_PARAMS_INVALID
        at sun.security.pkcs11.wrapper.PKCS11.C_CreateObject(Native Method)
        at sun.security.pkcs11....

The improved stack trace requires this patch: http://mail.openjdk.java.net/pipermail/security-dev/2010-April/001771.html

        if (EC_FillParams(arena, &pubKey->u.ec.ecParams.DEREncoding,
                    &pubKey->u.ec.ecParams) != SECSuccess) {
            crv = CKR_DOMAIN_PARAMS_INVALID;
            break;
        }

from pkcs11.c in NSS, 1629-1634 in GetPubKey

which returns

cleanup:
    if (!params->cofactor) {
        PORT_SetError(SEC_ERROR_UNSUPPORTED_ELLIPTIC_CURVE);
#if EC_DEBUG
 printf("Unrecognized curve, returning NULL params\n");
#endif
    }

This looks like a valid error. NSS does not support the curve requested:

Breakpoint 2, gf_populate_params (name=ECCurve_SECG_PRIME_112R1, field_type=ec_field_GFp, params=0x7fffd800e180) at ecdecode.c:145

    curveParams = ecCurve_map[params->name];
    CHECK_OK(curveParams);

That's from TestCurves. TestECDH wants ECCurve_NIST_P192. Both are NULL in nss-3.12.6/mozilla/security/nss/lib/freebl/ecl/ecl-curve.h

http://hg.mozilla.org/mozilla-central/file/8526e9e6c9ed/security/nss/lib/freebl/ecl/ecl-curve.h is the NSS version.

http://hg.openjdk.java.net/jdk7/jdk7/jdk/file/b50cfd4479fa/src/share/native/sun/security/ec/impl/ecl-curve.h is the version Sun imported into JDK7.

Needs new tests; the current ones tests algorithms unavailable to FOSS distros.