[SRU]: Cyrus linked against db4.4 compiled against 4.3 ?

Bug #67111 reported by Sander van Boxtel
56
Affects Status Importance Assigned to Milestone
cyrus-imapd-2.2 (Ubuntu)
Fix Released
Undecided
Unassigned
Nominated for Feisty by Michael Poole
Edgy
Fix Released
Undecided
Steve Kowalik

Bug Description

Binary package hint: cyrus-common-2.2

Hi,

In Edgy, my Cyrus2.2 install is broken.

/var/log/mail.err says :

Oct 19 12:07:51 cyrus/master[4689]: process 21024 exited, signaled to death by 11
Oct 19 12:07:51 cyrus/master[4689]: process 21025 exited, signaled to death by 11
Oct 19 12:07:51 cyrus/notify[21026]: incorrect version of Berkeley db: compiled against 4.3.29, linked against 4.4.20
Oct 19 12:07:51 cyrus/notify[21026]: Fatal error: wrong db version

Revision history for this message
Daniel Givens (rugmonster) wrote :

I experienced this as well.

Fixed by downloading source package and changing libdb4.3-dev to libdb4.4-dev as a Build-Depends in cyrus-imapd-2.2-2.2.13/debian/control. Rebuilt the package, reinstalled new debs and things worked.

Changed in cyrus-imapd-2.2:
status: Unconfirmed → Confirmed
Revision history for this message
WillyTP (willytp) wrote :

I confirm this bug too.
Moreover, on AMD64 as I execute cyrus it dies due to a segfault.

Revision history for this message
rleibman (roberto-freezlets) wrote :

Bug confirmed AND fix confirmed (hey cool, I learned something new, how to build a package from source). Thanks a lot!

I am running an AMD64 version of Ubuntu, for what it's worth.

Furthermore, upon compiling and installing I got bad error messages from cyrus, I imagine i could not read the db4.3 database with a db4.4 library, however, I followed <a href='http://www.irbs.net/internet/info-cyrus/0503/0248.html'>these instructions</a> and it worked (I'm pasting them here in case the provided link dies)

# /etc/init.d/cyrus stop
.
.
$ sudo -u cyrus bash
$ cd /var/imap/
$ rm db/*
$ rm db.backup?/*
$ rm deliver.db
$ rm tls_sessions.db
$ /usr/lib/cyrus/ctl_mboxlist -d > mailboxes.txt
$ mv mailboxes.db mailboxes.db.old
$ /usr/lib/cyrus/ctl_mboxlist -u < mailboxes.txt
$ diff mailboxes.db*
Files mailboxes.db and mailboxes.db.old differ
.
.
# /etc/init.d/cyrus start

Revision history for this message
Martin Marcher (martinmarcher) wrote :

That doesn't work for me:

root@vatikanstadt:/var/lib/cyrus# sudo -u cyrus /usr/sbin/ctl_mboxlist -d > mailboxes.txt
fatal error: wrong db version
Segmentation fault

The fix I'd expect would be to resolve those issues and have a clean compiled version of cyrus in the repository. I think it's not a good thing to install ubuntu-minimal apt-get install cyrus + postfix and have cyrus immediately segfaulting....

Revision history for this message
Michael Heimann (michael-heimann) wrote :

This also happens to me here and by the way makes my 1Gig P3 have a load of 3.5 ! since this: (cat /var/log/syslog:)

Nov 21 22:17:18 mail cyrus/master[3934]: process 17590 exited, signaled to death by 11
Nov 21 22:17:19 mail cyrus/master[3934]: service notify pid 17590 in READY state: terminated abnormally
Nov 21 22:17:19 mail cyrus/master[17593]: about to exec /usr/lib/cyrus/bin/notifyd
Nov 21 22:17:19 mail cyrus/notify[17593]: incorrect version of Berkeley db: compiled against 4.3.29, linked against 4.4.20
Nov 21 22:17:19 mail cyrus/notify[17593]: Fatal error: wrong db version
Nov 21 22:17:19 mail cyrus/master[3934]: process 17593 exited, signaled to death by 11

happens over and over and over.

I don't want to learn how to build debs - I prefer compiling over building debs myself. Although it would be nice to have cyrus installed just with synaptic (should be possible, or not ? )

Revision history for this message
Martin Marcher (martinmarcher) wrote :

I agree,

I chose ubuntu because of it's release cycles (used to have gentoo but it is too maintenance intensive and before that debian but it is too slow/old in it's stable tree).

Installing a package and having it immediately segfault (especially something like cyrus) is kind of a show stopper for.

Hope this will be fixed ASAP as I really like the rest of the ubuntu infrastructure but packages like these should really work out of the box with a default config...

Revision history for this message
Daniel Givens (rugmonster) wrote :

I can make working packages, but I need to know who I need to get in contact with to get them submitted to the official repository. Please contact me if you need either x86 or AMD64 packages.

Revision history for this message
Martin Marcher (martinmarcher) wrote :

I'm on a ubuntu-minimal amd64 with edgy here and get exactly that error

Changed in cyrus-imapd-2.2:
status: Unconfirmed → Confirmed
Revision history for this message
Daniel Givens (rugmonster) wrote :

Since I don't know the proper way to submit a change, I have attached the new control file. If you wish to build your own package from source, you simply need to do the following:

1. Download the attached control patch file which simply changes the build-depends from libdb4.3-dev to libdb4.4-dev.

2. Get the cyrus-imapd-2.2 source

# apt-get source cyrus-imapd-2.2

The source will be downloaded and expanded in your current working directory

3. Get the build dependencies

# apt-get build-dep cyrus-imapd-2.2
# apt-get install libdb4.4-dev fakeroot

Since the current source lists libdb4.3-dev as a build dependency, we need to get libdb4.4-dev ourselves. Fakeroot is also needed to properly build the package.

4. Apply the control patch

# patch -i control-cyrus-imapd-2.2.diff

5. Build the new packages

# cd cyrus-imapd-2.2-2.2.13/
# dpkg-buildpackage -rfakeroot -b

Packages will be built and placed in the parent directory

6. Install the packages

# cd ..
# dpkg -i cyrus*.deb

If you don't feel like going through that, I have posted the correct packages for i386 and AMD64 on my site at http://rugmonster.org/cyrus/.

Revision history for this message
Daniel Givens (rugmonster) wrote :

Ignore the first patch. I messed it up. Get the one marked *CORRECT*.

Revision history for this message
Daniel Givens (rugmonster) wrote :

Final note, change step 4 above to

# patch -p0 < control.diff

Revision history for this message
sir-archimedes (dominik-dbedv) wrote :

When will you patch this bug in the official release? It is already known for quite a time and a fix would be easily possible. This bug already costed some errors that could have been avoided.

Regards,
sir-archimedes

Revision history for this message
Daniel Givens (rugmonster) wrote :

I emailed the creator, Matthias Klose. Hopefully, he can suggest someone to get in touch with to get this fixed officially.

Revision history for this message
Matthias Klose (doko) wrote :

[away until January 2007]

- we need to figure out why the build dependency on 4.3
  leads to a dependency on 4.4 (dapper didn't have this).

- upgrade scenaries become complicated; dapper has
  the dependency on 4.3, edgy and feisty on 4.4. is
  an existing database correctly upgraded?

Revision history for this message
Daniel Givens (rugmonster) wrote :

Thanks, Matthias. Good points.

I'm not entirely sure how the database works when moving from 4.3 to 4.4. I want to say that when I upgraded from dapper, they just picked right up once cyrus started. I can't be quoted on that for sure, though. Maybe I can get some time to setup a test environment with dapper and then do an upgrade to see exactly what happens.

Revision history for this message
Michael Poole (mdpoole) wrote :

On the question of why the library dependency on libdb4.3-dev leads to linking against libdb4.4: My guess is that the package's Makefiles link using -ldb, rather than -ldb-3. When libdb4.4-dev (or libdb4.5-dev on Feisty) is installed, /usr/lib/libdb.so points to the newer version. My further hypothesis is that the build machines for Dapper didn't (or don't) have those newer versions installed, but they do for Edgy and Feisty.

Revision history for this message
Michael Poole (mdpoole) wrote :

I was clearly not thinking clearly when I posted my previous comment. The actual culprit seems to be the CYRUS_BERKELEY_DB_CHK_LIB autoconf helper in cyrus-imapd-2.2*/cmulocal/berkdb.m4, which explicitly looks for versioned shared libraries (-ldb-4.4, -ldb4.4, -ldb44, -ldb-4.3, etc, in that order) before linking against plain libdb. My package patching foo is not up to snuff to verify is the only problem, but it looks like a place to start.

Revision history for this message
triplem (triplem) wrote :

Is this bug resolved in feisty?

Revision history for this message
Matthias Klose (doko) wrote : Re: [Bug 67111] Re: Cyrus linked against db4.4 compiled against 4.3 ?

triplem schrieb:
> Is this bug resolved in feisty?

so how can you find out?

- apt-cache showsrc shows you the build dependencies
- apt-cache showsrc shows you the names of the binary packages
- apt-cache show <binary pkg> shows you the dependencies

As an alternative, you can find all this information at packages.ubuntu.com

Revision history for this message
jollyroger (jeffrey-crawford) wrote : Re: Cyrus linked against db4.4 compiled against 4.3 ?

Is there a way to increase the version of the self build binaries until the fix is provided? Every update I perform it wants to overwrite the version I have built with the broken version from the repository. I would be nice to increase it just a little to prevent that.

Revision history for this message
Steve Kowalik (stevenk) wrote :

cyrus-imapd-2.2 (2.2.13-10ubuntu2) feisty; urgency=low

  * Add missing semi-colon and correct a const in 61-64bit-quota.
  * Add a patch to drop libdb4.4 detection, as it will always use that
    if it's available. (LP: #67111)
  * Update configure patching in 99-update-autoconf.
  * Correct Maintainer fields.

 -- Steve Kowalik <email address hidden> Fri, 23 Feb 2007 19:36:54 +1100

Changed in cyrus-imapd-2.2:
status: Confirmed → Fix Committed
Revision history for this message
Steve Kowalik (stevenk) wrote :

Attached is a debdiff for uploading to edgy-proposed. This bug is a regression from Dapper, and is actually quite nasty. The init script fires off a master process which spawns off an imapd and/or pop3d process. They die horribly with a SEGV due to this libdb brain damage, are immediately re-spawned by the master process, and proceed to die again. Rinse and repeat.

The debdiff fixes the bug by patching configure (which the Debian build does anyway) to not include db4.4 in it's list of libdb versions.

Revision history for this message
StefanPotyra (sistpoty) wrote :

+1

Revision history for this message
Daniel T Chen (crimsun) wrote :

Looks sane, +1.

Revision history for this message
Reinhard Tartler (siretart) wrote :

+1 as well

Revision history for this message
Steve Kowalik (stevenk) wrote :

Uploaded to edgy-proposed.

Changed in cyrus-imapd-2.2:
assignee: nobody → stevenk
Revision history for this message
Shuvam (shuvam) wrote : Re: [Bug 67111] Re: Cyrus linked against db4.4 compiled against 4.3 ?

Am a little new to Debian and Ubuntu, hence don't know how
this bugfix process works. Am I reading you right if I conclude
that you are releasing the new, bugfixed version of Cyrus 2.2
only on Fiesty, and the Edgy release will not contain the bug-fix?
If I'm right, I'll have to download the Cyrus source from the Edgy
source archives and apply this Debdiff and recompile?

Please can you confirm/deny?

Shuvam

On 2/23/07, Steve Kowalik <email address hidden> wrote:
> Attached is a debdiff for uploading to edgy-proposed. This bug is a
> regression from Dapper, and is actually quite nasty. The init script
> fires off a master process which spawns off an imapd and/or pop3d
> process. They die horribly with a SEGV due to this libdb brain damage,
> are immediately re-spawned by the master process, and proceed to die
> again. Rinse and repeat.
>
> The debdiff fixes the bug by patching configure (which the Debian build
> does anyway) to not include db4.4 in it's list of libdb versions.
>
> ** Attachment added: "debdiff"
> http://librarian.launchpad.net/6516492/cyrus-edgy-proposed.debdiff
>
> --
> Cyrus linked against db4.4 compiled against 4.3 ?
> https://launchpad.net/bugs/67111
>

Revision history for this message
StefanPotyra (sistpoty) wrote :

@Shuvam: currently the fix for edgy is on its way. It will however need to go to the edgy-proposed repository first so that it can get a wider testing before it can finally go to edgy-updates.

You can find the exact details of the process at https://wiki.ubuntu.com/MOTU/SRU.

Revision history for this message
Steve Kowalik (stevenk) wrote :

2.2.13-10ubuntu2 has built on all arches, so I'm closing the Feisty task.

Changed in cyrus-imapd-2.2:
status: Fix Committed → Fix Released
Revision history for this message
Shuvam (shuvam) wrote : Re: [Bug 67111] Re: [SRU]: Cyrus linked against db4.4 compiled against 4.3 ?

> @Shuvam: currently the fix for edgy is on its way. It will however need
> to go to the edgy-proposed repository first so that it can get a wider
> testing before it can finally go to edgy-updates.
>
> You can find the exact details of the process at
> https://wiki.ubuntu.com/MOTU/SRU.

Thanks. Will look. :)

Shuvam

Revision history for this message
Martin Pitt (pitti) wrote :

Accepted into edgy-proposed, please go ahead with QA testing.

Changed in cyrus-imapd-2.2:
status: Confirmed → Fix Committed
Revision history for this message
Steve Kowalik (stevenk) wrote :

I'd like to get people to test my update in edgy-proposed so it can either be fixed properly, or get signed off and propagate to edgy-updates.

The steps to do this depend of if you used Daniel Givens' test packages or not.

If you did:

You need to downgrade your Cyrus databases from libdb 4.4 back down to 4.3. You can do this by running the following commands *before* installing the Cyrus package in edgy-proposed:
# /etc/init.d/cyrus stop
$ sudo -u cyrus bash
$ cd /var/imap/
$ rm db/*
$ rm db.backup?/*
$ rm deliver.db
$ rm tls_sessions.db
$ /usr/lib/cyrus/ctl_mboxlist -d > mailboxes.txt
$ mv mailboxes.db mailboxes.db.old

You can now upgrade to the packages in edgy-propsed, and run:
$ sudo -u cyrus bash # if required again
$ /usr/lib/cyrus/ctl_mboxlist -u < mailboxes.txt
# /etc/init.d/cyrus start

If you didn't use Daniel's packages, you should still be using libdb 4.3 databases, and so just need to upgrade to the packages in edgy-proposed and see if they work.

Revision history for this message
Shuvam (shuvam) wrote :

> If you didn't use Daniel's packages, you should still be using libdb 4.3
> databases, and so just need to upgrade to the packages in edgy-proposed
> and see if they work.

I didn't use Daniel's test packages.

Do I just add a line replacing 'edgy' with 'edgy-proposed' in
/etc/apt/sources.list
and apt-get install this package? Which packages exactly need to be installed?
I am asking because there are many packages which make up Cyrus 2.2.

Sorry about the confusion, but I'm new to this bug-tracking and updating
thing.

Shuvam

Revision history for this message
Egon Frerich (e-frerich) wrote :

Hello Steve,

Steve Kowalik schrieb:
> I'd like to get people to test my update in edgy-proposed so it can
> either be fixed properly, or get signed off and propagate to edgy-
> updates.
>
> The steps to do this depend of if you used Daniel Givens' test packages
> or not.
>

Yes, I've them installed.
> If you did:
>
> You need to downgrade your Cyrus databases from libdb 4.4 back down to 4.3.

My db's are filled with mails so I cannot simple remove them. But I
tried to find the files.
> You can do this by running the following commands *before* installing the Cyrus package in edgy-proposed:
> # /etc/init.d/cyrus stop
>
It is cyrus2.2
> $ sudo -u cyrus bash
> $ cd /var/imap/
>
Db's I found in /var/lib/cyrus
> $ rm db/*
> $ rm db.backup?/*
> $ rm deliver.db
> $ rm tls_sessions.db
> $ /usr/lib/cyrus/ctl_mboxlist -d > mailboxes.txt
>
ctl_mboxlist is not in /usr/lib/cyrus but cyrus-db-types.txt and
cyrus-db-types.active. Here I found "DBENGINE BerkeleyDB4.3". Not 4.4 !!

Maybe it helps.

Egon
> $ mv mailboxes.db mailboxes.db.old
>
> You can now upgrade to the packages in edgy-propsed, and run:
> $ sudo -u cyrus bash # if required again
> $ /usr/lib/cyrus/ctl_mboxlist -u < mailboxes.txt
> # /etc/init.d/cyrus start
>
> If you didn't use Daniel's packages, you should still be using libdb 4.3
> databases, and so just need to upgrade to the packages in edgy-proposed
> and see if they work.
>
>

Revision history for this message
hgg (helder) wrote :

Steve Kowalik wrote:
> I'd like to get people to test my update in edgy-proposed so it can
> either be fixed properly, or get signed off and propagate to edgy-
> updates.

I've made the update and so far so good!

Thanks,
--
Helder Guerreiro
http://www.paxjulia.com

Quidquid latine dictum sit, altum sonatur.

Revision history for this message
Daniel Givens (rugmonster) wrote :

Steve,

Great work! I've had it running today without any problems.

I would like to offer some corrections on your downgrade process. I (obviously) had to do it myself this morning when I installed the new packages.

For those that don't know, if you want to add the edgy-proposed repo without installing everything sitting in there each time, you can setup a preferences file for apt. Here is mine (/etc/apt/preferences)

Package: *
Pin: release a=edgy
Pin-Priority: 700

Package: *
Pin: release a=edgy-proposed
Pin-Priority: 600

The higher the number, the higher the priority. I'm not sure what the default is.

Once you have that, do an "apt-get update" to update your package list.

Next, stop your cyrus daemon.
$ sudo /etc/init.d/cyrus-2.2 stop

Make sure it's stopped or you'll get a lot of nasty errors in your logs.
$ ps ax | grep cyrmaster

If you get anything back other than your grep, kill it.
$ sudo pkill /usr/sbin/cyrmaster

Become cyrus and go to the appropriate directory to find the db files
$ sudo -u cyrus bash
<as cyrus now>
$ cd /var/lib/cyrus
$ rm db/*
$ rm db.backup?/*
$ rm deliver.db
$ rm tls_sessions.db

Convert the mailboxes.db to a flat file.
$ /usr/sbin/ctl_mboxlist -d > mailboxes.txt

Backup the mailboxes.db, just in case
$ mv mailboxes.db mailboxes.db.orig

Exit out of the cyrus user.
$ exit

Install the new packages.
$ sudo apt-get -t edgy-proposed install cyrus-imapd-2.2

Next, stop your cyrus daemon...it starts after install and your logs are filling up at this moment:
$ sudo /etc/init.d/cyrus-2.2 stop

Make sure it's stopped or you'll get a lot more nasty errors in your logs.
$ ps ax | grep cyrmaster

If you get anything back other than your grep, kill it.
$ sudo pkill /usr/sbin/cyrmaster

Become cyrus again, like you did before.
$ sudo -u cyrus bash

Convert the flat mailboxes file back to bdb 4.3 format.
$ /usr/sbin/ctl_mboxlist -u < mailboxes.txt

Exit the shell running as cyrus and start up the cyrus master daemon and test.
$ exit
$ sudo /etc/init.d/cyrus-2.2 start

As you can see, I based this on Steve's write up above. I just wrote it all out for clarity. Thanks to Steve for the packages and for the initial write up. Without the write up, I would have been lost. All I did was correct some path mistakes. I'm sure on a source install, that is where those things reside. Debian and Ubuntu have their own methods and policies on where stuff goes, and <diety> bless them for it.

Revision history for this message
StefanPotyra (sistpoty) wrote :

2 works for me -> please upload to -updates, thanks.

Revision history for this message
Martin Pitt (pitti) wrote :

Accepted into edgy-updates.

Changed in cyrus-imapd-2.2:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.