The openssl s390x assembly pack is not used

------- Comment From <email address hidden> 2016-07-13 08:02 EDT-------
---Problem Description---
The openssl s390x assembly code is not available, causing a fallback to openssl internal C-code. The performance degradation is up to a factor 4 for asymmetric (RSA, ...) and a factor >10 for symmetric cipher like SHA & AES.

Contact Information = <email address hidden>

---uname output---
4.4.0-28-generic #47-Ubuntu SMP Fri Jun 24 10:14:29 UTC 2016 s390x s390x s390x GNU/Linux

Machine Type = 2964, 701

---Debugger---
A debugger is not configured

---Steps to Reproduce---
#Run a benchmark
openssl speed sha1

#Check if CPACF was used
cpacfstatsd
cpacfstats

Userspace tool common name: OpenSSL 1.0.2g-fips 1 Mar 2016

The userspace tool has the following bit modes: 64-bit

Userspace rpm: OpenSSL 1.0.2g-fips 1 Mar 2016

Userspace tool obtained from project website: na

*Additional Instructions for <email address hidden>:
-Attach ltrace and strace of userspace application.

------- Comment From <email address hidden> 2016-07-13 08:04 EDT-------
To me it seems the following compiler flags

-DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DAES_CTR_ASM -DAES_XTS_ASM -DGHASH_ASM

are completely missing. Probably openssl was configured with the 'no-asm' option. This option, for example, is applied for s390 as defined in the openssl Configure file.

"debian-s390","gcc:-DB_ENDIAN ${debian_cflags}::-D_REENTRANT::-ldl:RC4_CHAR RC4_CHUNK DES_INT DES_UNROLL:${no_asm}:dlfcn:linux-shared:-fPIC::.so.\.\",

"debian-s390x","gcc:-DB_ENDIAN ${debian_cflags}::-D_REENTRANT::-ldl:SIXTY_FOUR_BIT_LONG RC4_CHAR RC4_CHUNK DES_INT DES_UNROLL:${no_asm}:dlfcn:linux-shared:-fPIC::.so.\.\",
-------
Its the same code as in bug IBM140645(LP1601836), right, where I report performance improvements using modified s390x code already upstream on openssl github. Tested by a locally installed openssl source package.

The issue reported here says that the s390x CPACF assembly code is not configured into the openssl build of Ubuntu at all and thus is not available in openssl client/server workloads; a fallback to very slow C-code is the result.

Probably, just some compiler flags are missing ...

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1602655/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

------- Comment From <email address hidden> 2016-07-13 09:03 EDT-------
dpkg -s openssl
Package: openssl
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 967
Maintainer: Ubuntu Developers <email address hidden>
Architecture: s390x
Version: 1.0.2g-1ubuntu4.1
Depends: libc6 (>= 2.15), libssl1.0.0 (>= 1.0.2g)
Suggests: ca-certificates
Conffiles:
/etc/ssl/openssl.cnf 7df26c55291b33344dc15e3935dabaf3
Description: Secure Sockets Layer toolkit - cryptographic utility
This package is part of the OpenSSL project's implementation of the SSL
and TLS cryptographic protocols for secure communication over the
Internet.
.
It contains the general-purpose command line binary /usr/bin/openssl,
useful for cryptographic operations such as:
* creating RSA, DH, and DSA key parameters;
* creating X.509 certificates, CSRs, and CRLs;
* calculating message digests;
* encrypting and decrypting with ciphers;
* testing SSL/TLS clients and servers;
* handling S/MIME signed or encrypted mail.
Original-Maintainer: Debian OpenSSL Team <email address hidden>

This ticket is related to https://bugs.launchpad.net/ubuntu-z-systems/+bug/1601836

Hi Bastian,
wasn't up until recently the pure SW slow anyway and only added about 2-3 years ago in that crazy perl generated code?
I remember we wondered why SW mode was suddenly as fast as HW mode.

Up until that time to exploit HW like cpacf one had to install and configure ibmca, which should would work today as well.
Is the test with HW-nocard as fast as it should be in general (excluding for bug 1601836)?

This doesn't mean it shouldn't be added, but in terms of severity and urgency that would mean: a proper setup works and thereby a workaround is available right?

Please remove target 16.04.1.

This feature must first land in Yakkety Yak, before being eligible for SRU into Xenial. SRU cycle is one week minimum, if and when features are ready for SRU.

Furthermore, this bug report does not affect installation media, and thus is only needed to land in -updates pocket.

------- Comment From <email address hidden> 2016-07-14 09:08 EDT-------
Hi,

that's right !
A workaround would be to configure the IBMCA engine into openssl (openssl.cnf).
However, as the Polyakov code directly make use of the CPACF instructions we will end up with about
50 % performance degradation compared to the 'longer' way through the libica API for small payloads.

For the asymmetric case, when no CEX5 cards are available, we will still have a factor ~4. You remember the 'bn_mult_add_words' improvements, right ?

------- Comment From <email address hidden> 2016-07-22 07:52 EDT-------
Changed distro target to 16.10 . But this fix should also be SRUed to 16.04.1 due to the performance impacts !

Some updates on this ticket: openssl-1.1 in debian experimental has asm optimisations enabled for the debian-s390x build configuration. Enabling similar configuration change in 1.0.2 results in the package failing to build from source - shared libraries appear to be missing for an unknown reason. Investigating further.

This bug was fixed in the package openssl - 1.0.2g-1ubuntu6

---------------
openssl (1.0.2g-1ubuntu6) yakkety; urgency=medium

  * Enable asm optimisations on s390x. LP: #1602655.

 -- Dimitri John Ledkov <email address hidden> Thu, 28 Jul 2016 15:37:07 +0300

Hello bugproxy, or anyone else affected,

Accepted openssl into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/openssl/1.0.2g-1ubuntu4.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

------- Comment From <email address hidden> 2016-08-12 06:48 EDT-------
I tested for:

1) Ubuntu Yakkety Yak (development branch) (GNU/Linux 4.4.0-33-generic s390x)
Version: 1.0.2g-1ubuntu7

2)Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-18-generic s390x)
Version: 1.0.2g-1ubuntu4.2

The BUG is fixed in both cases.

------- Comment From <email address hidden> 2016-08-12 07:54 EDT-------
Due to SRU.... to 16.04.1 changing target to reflect the inclusion to the LTS release....

16041 installation media milestone makes no sense.... because it has shipped, and this bug is not about installation media / does not affect nor prevents installation from succeeding.

In general, following targets make sense:

16.04-updates -> for bugfixes to ship in xenial series (aka xenial-updates pocket). This is an open-ended target for the lifetime of xenial (5 years)

16041 -> for bugfixes that must ship on the point release installation media, and thus affect / prevent installation from happening. This milestone has closed since 16.04.1 release has shipped.

16042 -> currently open milestone, for the next point release which will happen approximately 6 months after .1 / ~3 months after 16.10 ships. It's not scheduled yet, but should be sometime in January 2017. Only bugs that affect / prevent installation from succeeding may be targetted at 16.04.2 milestone.

16043, 16044, 16045 -> future point release installation media milestones.

This bug was fixed in the package openssl - 1.0.2g-1ubuntu4.2

---------------
openssl (1.0.2g-1ubuntu4.2) xenial; urgency=medium

  * Cherry-pick s390x assembly pack bugfix to cache capability query
    results for improved performance. LP: #1601836.
  * Enable asm optimisations on s390x. LP: #1602655.

 -- Dimitri John Ledkov <email address hidden> Thu, 28 Jul 2016 15:37:07 +0300

The verification of the Stable Release Update for openssl has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.