Panic: file sieve-cmu.c: line 88 (unfold_header): assertion failed: (str[i] == ' ' || str[i] == '\t')

Hi Matt,

Can you please provide your dovecot.conf and your sieve.conf?

Thanks
chuck

I've reproduced the bugs with a minimal dovecot.conf and sieve.conf as follows:

dovecot.conf:
# Base directory where to store runtime data.
base_dir = /tmp/dovecot-264306
protocols =
mail_location = maildir:/tmp/dovecot-264306/mail

protocol lda {
    postmaster_address =
    log_path = /tmp/dovecot-264306/mail.log
 mail_plugins = cmusieve
}

plugin {
 sieve = /tmp/dovecot-264306/sieve.conf
}

sieve.conf:
if header :contains "Subject" "blahblahblah" { discard; }

I think the problem is that the Subject headers of these messages contain MIME-encoded newlines:

Subject: =?windows-1255?Q?=E0=E9=EE=E5=EF_=E0=E9=F9=E9_=EC=E9=F8=E9=E3=E4_=E1=E8=E5=E7=E4_=E1=EE=F9=F7=EC=0D=0A?=
Subject: =?windows-1255?Q?=E0=FA=F8_=E4=EE=EB=F8=E6=E9=ED_=E4=EE=E5=E1=E9=EC_=E1=EE=E3=E9=F0=E4=0D=0A?=
Subject: =?windows-1255?Q?=E0=FA=F8_=E4=EE=EB=F8=E6=E9=ED_=E4=EE=E5=E1=E9=EC_=E1=EE=E3=E9=F0=E4=0D=0A?=
Subject: =?windows-1255?Q?=E0=FA=F8_=EE=EB=F8=E6=E9=ED__=EE=E5=E1=E9=EC_=E1=EE=E3=E9=F0=E4=0D=0A?=

This bug was fixed in the package dovecot - 1:1.1.2-1ubuntu7

---------------
dovecot (1:1.1.2-1ubuntu7) intrepid; urgency=low

  * debian/patches/fix-dovecot-sieve.dpatch: Fixes assertion error
    when a header string ends with a LF (LP: #264306)

 -- Chuck Short <email address hidden> Wed, 03 Sep 2008 11:20:31 -0400

I'm seeing this again with 1:1.1.4-0ubuntu1.2:

deliver(mdz): Nov 25 12:07:45 Panic: file sieve-cmu.c: line 90 (unfold_header): assertion failed: (str[i] == ' ' || str[i] == '\t')
deliver(mdz): Nov 25 12:07:45 Error: Raw backtrace: /usr/lib/dovecot/deliver [0xb7f1ce01] -> /usr/lib/dovecot/deliver(default_fatal_handler+0x4c) [0xb7f1cf4c] -> /usr/lib/dovecot/deliver [0xb7f1c695] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so [0xb7cd769f] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so [0xb7ce20b9] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so [0xb7ce1d6b] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so(sieve_eval_bc+0x47e) [0xb7ce333e] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so(sieve_execute_bytecode+0x125) [0xb7ce94d5] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so(cmu_sieve_run+0x325) [0xb7cd8785] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so [0xb7cd6456] -> /usr/lib/dovecot/deliver(main+0x1229) [0xb7ea6c39] -> /lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe5) [0xb7d09685] -> /usr/lib/dovecot/deliver [0xb7ea4531]

This bug was fixed in the package dovecot - 1:1.1.11-0ubuntu1

---------------
dovecot (1:1.1.11-0ubuntu1) jaunty; urgency=low

  [ Ante Karamatic ]
    Add new binary pkg dovecot-postfix that integrates postfix and dovecot
    automatically: (LP: #164837)
    - debian/control:
      + add new binary with short description.
    - debian/dovecot-postfix.postinst:
      + create initial certificate symlinks to snakeoil.
      + set up postfix with postconf to:
        - use Maildir/ as the default mailbox.
        - use dovecot as the sasl authentication server.
        - use dovecot LDA (deliver).
        - use tls for smtp{d} services.
      + restart postfix and dovecot.
    - debian/dovecot-postfix.postrm:
      + remove all dovecot related configuration from postfix.
      + restart postfix and dovecot.
    - debian/dovecot-common.init:
      + check if /etc/dovecot/dovecot-postfix.conf exists and use it
        as the configuration file if so.
    - debian/patches/warning-ubuntu-postfix.dpatch
      + add warning about dovecot-postfix.conf in dovecot default
        configuration file.
    - debian/patches/dovecot-postfix.conf.diff:
      + Ubuntu server custom changes to the default dovecot configuration for
        better integration with postfix:
        - enable imap, pop3, imaps, pop3s and managesieve by default.
        - enable dovecot LDA (deliver).
        - enable SASL auth socket in postfix private directory.
    - debian/rules:
      + copy, patch and install dovecot-postfix.conf in /etc/dovecot/.

  [ Mathias Gug ]
  * New upstream release:
  * Update dovecot-managesieve to 0.10.5. Fixes:
    - check if names of sieve scripts contain '/' (LP: #307291)
  * Update dovecot-managesieve patch for 1.1.11 and 0.10.5.
  * Update dovecot-sieve plugin to 1.1.6.
  * Merge from debian experimental, remaining changes:
    - Use Snakeoil SSL certificates by default.
      + debian/control: Depend on ssl-cert
      + debian/paptches/ssl-cert-snakeoil.dpatch: Change default SSL cert
        paths to snakeoil.
      + debian/dovecot-common.postinst: Relax grep for SSL_* a bit.
    - Add autopkgtest in debian/tests/*.
    - debian/dovecot-common.init: Check to see if there is an /etc/inetd.conf.
      (LP: #208411)
    - Fast TearDown: Update lsb init header to not stop in level 6.
    - Add status action to the init script:
      + debian/control: Depend on lsb >= 3.2.12ubuntu3.
      + debian/dovecot-common-init: Add the 'status' action (LP: #247096).
    - debian/rules:
      - Copy config.{guess,sub} after running libtoolize.
      - Clean dovecot-managesieve directory.
    - Add ufw integration:
      - Created debian/dovecot-common.ufw.profile
      - debian/rules:
        + install profile
      - debian/control
        + Suggest ufw
    - debian/{control,rules}: enable PIE hardening.
    - Updated dovecot.common.README.Debian with information on what has changed
      between 1.0 and 1.1.1. Fixes (LP: #257625)
    - dovecot-imapd, dovecot-pop3: Replaces dovecot-common (<< 1:1.1). LP: #254721.
    - debian/control:
      + Update Vcs-* headers.
  * debian/rules:
    - Create emtpy stamp.h.in files in dovecot-sieve/ and dovecot-managesi...

Reopening as I have seen this as recently as yesterday:

deliver(mdz): Apr 16 11:49:56 Panic: file sieve-cmu.c: line 90 (unfold_header): assertion failed: (str[i] == ' ' || str[i] == '\t')

Here are some more messages which trigger the bug.

How to reproduce:

Install dovecot-common and procmail packages.

mathiaz@t-dovecot$ mkdir /tmp/dovecot-264306; cd /tmp/dovecot-264306/

mathiaz@t-dovecot:/tmp/dovecot-264306$ cat dovecot.conf
# Base directory where to store runtime data.
base_dir = /tmp/dovecot-264306
protocols =
mail_location = maildir:/tmp/dovecot-264306/mail

protocol lda {
    postmaster_address =
    log_path = /tmp/dovecot-264306/mail.log
 mail_plugins = cmusieve
}

plugin {
 sieve = /tmp/dovecot-264306/sieve.conf
}

mathiaz@t-dovecot:/tmp/dovecot-264306$ cat sieve.conf
if header :contains "Subject" "blahblahblah" { discard; }

mathiaz@t-dovecot:/tmp/dovecot-264306$ formail -s /usr/lib/dovecot/deliver -c /tmp/dovecot-264306/dovecot.conf < spam2.mbox

Matt, I've tried to process the three mbox files you've attached to this bug. Both the 1st and 3rd one don't produce any errors.

However the 2nd one produces the following error:
deliver(mathiaz): Apr 18 14:57:53 Panic: file sieve-cmu.c: line 90 (unfold_header): assertion failed: (str[i] == ' ' || str[i] == '\t')
deliver(mathiaz): Apr 18 14:57:53 Error: Raw backtrace: /usr/lib/dovecot/deliver [0x7f06f76ebab2] -> /usr/lib/dovecot/deliver(default_fatal_handler+0x37) [0x7f06f76ebbc7] -> /usr/lib/dovecot/deliver [0x7f06f76eb1e8] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so [0x7f06f6cb0170] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so [0x7f06f6cba387] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so(sieve_eval_bc+0x47d) [0x7f06f6cbb4ad] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so(sieve_execute_bytecode+0xf6) [0x7f06f6cc1116] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so(cmu_sieve_run+0x318) [0x7f06f6cb1188] -> /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so [0x7f06f6caf0eb] -> /usr/lib/dovecot/deliver(main+0x1032) [0x7f06f76803a2] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x7f06f6ee95a6] -> /usr/lib/dovecot/deliver [0x7f06f767dfb9]

The reason the bug was closed the second time is that a changelog mentioned this bug number again after this bug was reopened:

  * Dropped:
    - debian/patches/fix-message-parser.dpatch: Parsing an invalid message
      address like "From: (" caused an assert-crash. (LP: #290901).
      (CVE-2008-4907 - fixed in 1.1.6)
    - debian/patches/login-max-process-count-warning.dpatch: Tell the user
      that they have reached the maximum number of processes count.
      (LP: #189616) - Different implementation from upstream.
    - debian/patches/fix-dovecot-sieve.dpatch: Fixes assertion error
      when a header string ends with a LF (LP: #264306). Implemented upstream.

So it seems that the upstream patch provided for the first issue isn't enough.

I've attached a debdiff with a patch taken from upstream (http://hg.dovecot.org/dovecot-sieve-1.1/raw-rev/b9567e94b897).

Processing the second mbox file:
 * without the patch dovecot LDA crashes and doesn't deliver the email (ie email is lost).
 * with the patch applied, dovecot LDA is able to deliver the email, with the following messages written to the log file:

deliver(mathiaz): Apr 18 19:03:06 Error: Corrupted index cache file (in-memory index).cache: Broken fields for mail UID 1
deliver(mathiaz): Apr 18 19:03:06 Error: Couldn't fix broken header unfolding
deliver(mathiaz): Apr 18 19:03:06 Info: msgid=<01c9548a$f1666dfe$6c822ecf@pillarsx6>: saved mail to INBOX
deliver(mathiaz): Apr 18 19:03:06 Error: Corrupted index cache file (in-memory index).cache: Broken fields for mail UID 1
deliver(mathiaz): Apr 18 19:03:06 Error: Couldn't fix broken header unfolding
deliver(mathiaz): Apr 18 19:03:06 Info: msgid=<email address hidden>: saved mail to INBOX
deliver(mathiaz): Apr 18 19:03:06 Error: Corrupted index cache file (in-memory index).cache: Broken fields for mail UID 1
deliver(mathiaz): Apr 18 19:03:06 Error: Couldn't fix broken header unfolding
deliver(mathiaz): Apr 18 19:03:06 Info: msgid=<01c95592$4014eb4e$6c822ecf@forborne>: saved mail to INBOX
deliver(mathiaz): Apr 18 19:03:06 Error: Corrupted index cache file (in-memory index).cache: Broken fields for mail UID 1
deliver(mathiaz): Apr 18 19:03:06 Error: Couldn't fix broken header unfolding
deliver(mathiaz): Apr 18 19:03:06 Info: msgid=<email address hidden>: saved mail to INBOX

Asked upstream how serious the Error messages were:

http://dovecot.org/pipermail/dovecot/2009-April/039014.html

Reading through the code, the cache is marked invalid and the cache file is deleted. The message is still delivered to the mail box. The patch has been in the upstream repository for six months and hasn't seen any activity related to it. I'll apply the patch to karmic in the next upload.

This bug was fixed in the package dovecot - 1:1.1.11-0ubuntu8

---------------
dovecot (1:1.1.11-0ubuntu8) karmic; urgency=low

  * dovecot-sieve: (LP: #264306)
    - Fix an assertion failure on messages that have a bad header.

 -- Mathias Gug <email address hidden> Wed, 09 Sep 2009 21:31:20 -0400