mysql refuses to start because missing !includedir configuration directory

Bug #305107 reported by Henning Sprang
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
mysql-dfsg-5.1 (Ubuntu)
Triaged
Low
Unassigned

Bug Description

root@xnote:~# lsb_release -rd
Description: Ubuntu 8.04
Release: 8.04

henning@xnote:~$ apt-cache policy mysql-server-5.0
mysql-server-5.0:
  Installed: 5.0.51a-3ubuntu5.4
  Candidate: 5.0.51a-3ubuntu5.4
  Version table:
 *** 5.0.51a-3ubuntu5.4 0
        500 file: hardy-security/main Packages
        100 /var/lib/dpkg/status
     5.0.51a-3ubuntu5 0
        500 file: hardy/main Packages

Corecting name and adding the state of things at the beginning:

mysqld is not starting when /etc/mysql/conf.d is not empty, even though it contains only optional information.

In conjunction with bug 305107 this leads to mysql failing to start after a security upgrade from the default version in the normal apt repository to the newest security update version.

More explanation below:

---

I just did a security upgrade for my Hardy system, and after the reboot, mysql refuses to start!

Errors already appear at system start, here the output of the command when I try to start the server manually after system start(errors are the same):

/etc/init.d/mysql restart
open: Permission denied
 * Stopping MySQL database server mysqld /usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
open: Permission denied
                                                                         [ OK ]
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
open: Permission denied
 * Starting MySQL database server mysqld /usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
/usr/sbin/mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 2)
Fatal error in defaults handling. Program aborted
open: Permission denied
                                                                         [fail]

root@xnote:~# lsb_release -rd
Description: Ubuntu 8.04
Release: 8.04

root@xnote:~# dpkg -l mysql*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Installed/Config-f/Unpacked/Failed-cfg/Half-inst/t-aWait/T-pend
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name Version Description
+++-=================================-=================================-==================================================================================
ii mysql-admin 5.0~rc12-2ubuntu1 GUI tool for intuitive MySQL administration
un mysql-admin-common <none> (no description available)
ii mysql-client 5.0.51a-3ubuntu5.4 MySQL database client (meta package depending on the latest version)
un mysql-client-4.1 <none> (no description available)
ii mysql-client-5.0 5.0.51a-3ubuntu5.4 MySQL database client binaries
ii mysql-common 5.0.51a-3ubuntu5.4 MySQL database common files
un mysql-community-client-5.0 <none> (no description available)
un mysql-community-server-5.0 <none> (no description available)
un mysql-doc-5.0 <none> (no description available)
un mysql-enterprise-client-5.0 <none> (no description available)
un mysql-enterprise-server-5.0 <none> (no description available)
un mysql-gpl-client <none> (no description available)
ii mysql-gui-tools-common 5.0~rc12-2ubuntu1 Architecture independent files for MySQL GUI Tools
un mysql-navigator <none> (no description available)
un mysql-proxy <none> (no description available)
ii mysql-query-browser 5.0~rc12-2ubuntu1 Official GUI tool to query MySQL database
un mysql-query-browser-common <none> (no description available)
ii mysql-server 5.0.51a-3ubuntu5.4 MySQL database server (meta package depending on the latest version)
un mysql-server-4.1 <none> (no description available)
ii mysql-server-5.0 5.0.51a-3ubuntu5.4 MySQL database server binaries
un mysqltcl <none> (no description available)

Revision history for this message
Henning Sprang (henning) wrote :

Hmm, simple workaround:

mkdir /etc/mysql/conf.d

helps!

Revision history for this message
Augusto Cezar Amaral (acacs) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better. This bug did not have a package associated with it, which is important for ensuring that it gets looked at by the proper developers. You can learn more about finding the right package at https://wiki.ubuntu.com/Bugs/FindRightPackage . I have classified this bug as a bug in mysql-dfsg-5.0.
For future reference you might be interested to know that a lot of applications have bug reporting functionality built in to them. This can be accessed via the Report a Problem option in the Help menu for the application with which you are having an issue. You can learn more about this feature at https://wiki.ubuntu.com/ReportingBugs.

Revision history for this message
Augusto Cezar Amaral (acacs) wrote :

Could you also, please, answer these questions:

* Is this reproducible?
* If so, what specific steps should we take to recreate this bug?

This will help us to find and resolve the problem. Thank you again.

Revision history for this message
Henning Sprang (henning) wrote : Re: [Bug 305107] Re: mysql refuses to start after security update

On Thu, Dec 4, 2008 at 4:22 PM, Augusto Cezar Amaral
<email address hidden> wrote:
> Thank you for taking the time to report this bug and helping to make Ubuntu better. This bug did not have a package associated
> with it, which is important for ensuring that it gets looked at by the proper developers.

oops, sorry, I obviously forgot that...

Maybe launchpad should get an extra field for this, and ask people one
extra time if they forget to fill it in (but still let them save if
they, for some reason, don't know which package it is.

> You can learn more about finding the right package at https://wiki.ubuntu.com/Bugs/FindRightPackage . I have classified this
> bug
> as a bug in mysql-dfsg-5.0.

seems correct:

The script in error seems to be /usr/sbin/mysqld

dpkg -S /usr/sbin/mysqld
mysql-server-5.0: /usr/sbin/mysqld

source of this is the package you mentioned.

Thanks for setting it right!

> For future reference you might be interested to know that a lot of applications have bug reporting functionality built in to them.
>This can be accessed via the Report a Problem option in the Help menu for the application with which you are having an issue.
> You can learn more about this feature at https://wiki.ubuntu.com/ReportingBugs.

Yeah, I think I already saw that somewhere.

But now I searched for two hours for the help menu of
mysql-server.5.0, and I cannot find it - do you have an idea?
No, just joking, I think that hint doesn't work for server daemons :)

Anyway, hope that thing get's fixed soon - I believe that is a very
grave error to happen - a security update that breaks the abilty of a
server daemon to start is no fun at all!
I'm happy that I don't use Ubuntu on servers, this would have been a
major catastrophe.

Revision history for this message
Henning Sprang (henning) wrote :

On Thu, Dec 4, 2008 at 4:43 PM, Augusto Cezar Amaral
<email address hidden> wrote:
> Could you also, please, answer these questions:
>
> * Is this reproducible?

I don't know. Reproducing an error that only occurs after a system
upgrade take some more time than reproducing something that occurs at
a click in a GUI application :)

> * If so, what specific steps should we take to recreate this bug?

As I wrote - the problem is caused by a security update.

I have no time to check that now(maybe sometime these days), but I
guess it can be reproduced like that:

* install Ubuntu 8.04 from an old cd - and do NOT include the apt
repository for security updates
* install the mysql-server-5.0 package
* check if the is a directory /etc/mysql/conf.d - it should NOT be
there - if it is there, the error will not be reproducable, and this
directory has been deleted from my system for some strange reason
* check that the mysql daemon is running
* add the apt repository for security updates
* run an upgrade
* restart the system and/or try to restart the mysql daemon

Hope that helps

Henning

Revision history for this message
Henning Sprang (henning) wrote : Re: mysql refuses to start after security update

I'm trying to start up a VM installation and see if I can reproduce the problem!

(maybe it even works from the live cd?!)

Revision history for this message
Mathias Gug (mathiaz) wrote : Re: [Bug 305107] Re: mysql refuses to start after security update

On Thu, Dec 04, 2008 at 04:02:35PM -0000, Henning Sprang wrote:
> * check if the is a directory /etc/mysql/conf.d - it should NOT be
> there - if it is there, the error will not be reproducable, and this
> directory has been deleted from my system for some strange reason

Are you able to start mysql-server if you re-create /etc/mysql/conf.d/
as an empty directory?

  status incomplete

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

Revision history for this message
Henning Sprang (henning) wrote :

On Thu, Dec 4, 2008 at 5:32 PM, Mathias Gug <email address hidden> wrote:
> On Thu, Dec 04, 2008 at 04:02:35PM -0000, Henning Sprang wrote:
>> * check if the is a directory /etc/mysql/conf.d - it should NOT be
>> there - if it is there, the error will not be reproducable, and this
>> directory has been deleted from my system for some strange reason
>
> Are you able to start mysql-server if you re-create /etc/mysql/conf.d/
> as an empty directory?

yes, see my second post :)

Strange, I installed the mysql-server-5.0 package in a hardy live cd,
but here, the directory /etc/mysql/conf.d is created and it contains
the file old_passwirds.conf. I have no idea how this file could be
removed from my system.

Still, a daemon should definitely not fail to start because a
directory that contains optional, additional configs is not there.

I'm checkin further to find out
* whether maybe even the package install script itself is responsible
for removing the file/directory
* whether previous versions of the mysql package also are unable to
start when the directory is missing.

and get back with more info...

Revision history for this message
Henning Sprang (henning) wrote : Re: mysql refuses to start after security update

O.k. I checked this again.

I went through the steps mentioned before to reproduce the problem.

After installation of mysql-server-5.0 from the apt repository _without_ security updates, the directory conf.d and a file in it, old_passwords.cnf EXISTS!

Adding the security repository and then upgrading the package, makes the directory conf.d be DELETED, seemingly be the upgrade/postinstall script, and then, the server fails to start!

Nnot only is it wrong in my opinion that a sertver fails to start when an optional directory is missing, that the upgrade process even deletes this directory, without which the server cannot start... strange.

Additional info: I made a default desktop install from live cd, not a server install. marking the bug confirmed, as I thoroughly tested it on a clean install.

The simplest fix I can imagine would be to change the mysqld start script to ignore the optional directory if not there.

Changed in mysql-dfsg-5.0:
status: Incomplete → Confirmed
Revision history for this message
Henning Sprang (henning) wrote :

setting status to confirmed, as I don't see what more info could be provided, and it's thoroughly tested now.

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

I can't reproduce this issue. Here's what I've done:

- Installed Hardy
- Disabled -updates and -security in /etc/apt/sources.list
- sudo apt-get install mysql-server-5.0
- Directory /etc/mysql/conf.d exists with old_passwords.cnf in it
- Enabled -updates and -security in /etc/apt/sources.list
- sudo apt-get dist-upgrade
- Directory /etc/mysql/conf.d still exists (with no files in it)
- Mysql starts fine

Also, I took a look at all the scripts in the mysql package, and I don't see where any of them could remove that directory.

I am not sure why you're getting this issue. Is there anything else you are installing or doing to reproduce it?

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

Also, in your first comment, you are getting "open: Permission denied". What user are you using to restart the mysql daemon?

Revision history for this message
Henning Sprang (henning) wrote : Re: [Bug 305107] Re: mysql refuses to start after security update

On Fri, Dec 5, 2008 at 10:20 PM, Marc Deslauriers
<email address hidden> wrote:
> Also, in your first comment, you are getting "open: Permission denied".
> What user are you using to restart the mysql daemon?

the script "/etc/init.d/mysql restart"

I'm checking everything again...

Revision history for this message
Henning Sprang (henning) wrote :

On Fri, Dec 5, 2008 at 10:15 PM, Marc Deslauriers
<email address hidden> wrote:
> I am not sure why you're getting this issue. Is there anything else you
> are installing or doing to reproduce it?

Strange.
I'm checking again - it will take a while.

About the only thing that I did additionally before installing, to
better track changes, is that I installed etckeeper and did

git init-db
git add .
git commit -m "init"

in /etc

Revision history for this message
Henning Sprang (henning) wrote :

On Fri, Dec 5, 2008 at 10:15 PM, Marc Deslauriers
<email address hidden> wrote:
> You received this bug notification because you are a direct subscriber
> of the bug.

You're right!
When I don't do any git / etckeeper stuff, the empty directory remains.
When I do

apt-get install etckeeper
etckeeper init

the conf.d directory is deleted and mysql cannot start.
I did not think that could be causing the error

So we have really two bugs:

1) for etckeeper: it should not delete empty directories after a
system upgrade (in general, don't apply any changes on upgrades,
unless being explicitly told to clean something up)
2) In any case, the issue of the daemon not starting just because a
missing optional directory remains here for mysql

Revision history for this message
Henning Sprang (henning) wrote : Re: mysql refuses to start after security update

reported for etckeeper as bug 305642

Henning Sprang (henning)
description: updated
Mathias Gug (mathiaz)
Changed in mysql-dfsg-5.0:
importance: Undecided → Low
Mathias Gug (mathiaz)
Changed in mysql-dfsg-5.0:
status: Confirmed → Triaged
Chuck Short (zulcss)
affects: mysql-dfsg-5.0 (Ubuntu) → etckeeper (Ubuntu)
Revision history for this message
Henning Sprang (henning) wrote : Re: [Bug 305107] Re: mysql refuses to start because missing !includedir configuration directory

Chuck, why is that changed this way?

This problem is two bugs!

* etckepper shouldn't remove empty directories - that is handled, and
fixed, with a bugreport on it's own as can be seen in the history here!

* a program not starting because a directory missing that _might_
contain _optional_ config files is a bug too, IMHO - why should it worry
about a missing directory, when it's contents are optional, and it can
very well run without any information in that directory, instead
insisting the empty directory must be there?

So, please move this bug back so it can be fixed where it belongs - thanks.

Or close it, if you think a program should be allowed to insist whetever
empty directory to exist so it can start, even if it'll never need that
directory.

Henning

On 2010-04-23 16:13, Chuck Short wrote:
> ** Package changed: mysql-dfsg-5.0 (Ubuntu) => etckeeper (Ubuntu)
>

--
Henning Sprang EDV Schulung und Beratung
Phone: +49 176 82188257
WWW: http://www.sprang.de

Thierry Carrez (ttx)
affects: etckeeper (Ubuntu) → mysql-dfsg-5.1 (Ubuntu)
Revision history for this message
Micah W. (mmw-canada) wrote :

Ok, I think I ran into the same bug upgrading yesterday from 10.10 to 15.04.

I was able continue by doing three steps:

1. Adding the mysql Repos(may not be necessary)
https://dev.mysql.com/downloads/repo/

2. Tried doing apt-get install mysql-server and got this error:
"Setting up mysql-community-server (5.6.26-1ubuntu15.04) ...
mysqld: Can't read dir of '/etc/mysql/mysql.conf.d/' (Errcode: 13 - Permission denied)"

3. Removed apparmor
apt-get remove apparmor apparmor-utils

4. Did apt-get install mysql-server -> This time it worked!

Revision history for this message
Robie Basak (racb) wrote :

@Micah

It sounds like you had a different issue - bug 1435368. This is handled in 15.04 correctly now, but may not be in Oracle's packaging - but you'll need to file a bug with them to get that fixed as it isn't part of Ubuntu and you chose to install the package from there.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.