OpenSSL is fun (not)

Today I decided to finally upgrade two of my older boxes to the most recent OpenSSL version (openssl-0.9.8e) as that change has been on my ToDo list for ages. Both hosts are ancient Pentium I based Linux servers, so compiling OpenSSL was great fun and took hours.
Read on if you are interested in never-ending compile sessions, SSHd segfaults, issues with bn_mul_add_words() functions, no-sse2 settings, VIA Padlock crypto engines or other OpenSSL fun. Oh, and don’t ask me anything about OpenSSL in the next weeks, right?! ๐Ÿ˜‰

First I forgot to add “shared” to the config command so I had to do the whole thing again to get shared libraries. Then the newly built OpenSSH server just segfaulted on start – gdb quickly told me that it was crashing in an OpenSSL routine called bn_mul_add_words(). A quick “googling” later (link) I knew that the config command of OpenSSL wants the “no-sse2” option, otherwise it builds libraries that make linked binaries segfault on start – at least (obviously) on my good old P1 cpu. Great. Another never-ending recompile session.
Finally I had an OpenSSL version and a new SSHd that did, indeed, work. But that was just one out of two hosts.

On the other server OpenSSL failed to build with some errors specific to VIA Padlock crypto-engines:

gcc -I.. -I../.. -I../../include -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -Wall -DOPENSSL_BN_ASM_PART_WORDS -DSHA1_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM -c -o eng_padlock.o eng_padlock.c
eng_padlock.c: In function padlock_available':
eng_padlock.c:348: output operand constraint contains
+’
eng_padlock.c:355: output operand constraint contains +'
eng_padlock.c:362: output operand constraint contains
+’
eng_padlock.c: In function padlock_bswapl':
eng_padlock.c:380: output operand constraint contains
+’
eng_padlock.c: In function padlock_verify_context':
eng_padlock.c:419: output operand constraint contains
+’
make[2]: *** [eng_padlock.o] Error 1

Hey, this is a stone-age Pentium I, go away with VIA crypto! Feeding Google with this error message didn’t help much, but after a quick look at the source I added -DOPENSSL_NO_HW_PADLOCK to the main Makefile. Voilรƒยก, now the OpenSSL compile worked. But that wasn’t all, “make install” failed with some errors because it couldn’t copy some shared libraries out of the “engines” subdirectory. As I only needed OpenSSL to build a new OpenSSH version I didn’t care much about these engines, so I removed the “install” section of the Makefile in the “engines” subdir.
That helped, and I finally had a new (more or less) working OpenSSL installation. At least it was sufficient to build a (working) SSHd ๐Ÿ˜‰

Note: I’m not responsible if you destroy your system that way. But if you need these tips you certainly should consider upgrading. ๐Ÿ™‚

Comments are closed.