ssh is started before hostkey is installed/regenerated

Bug #512377 reported by Thierry Carrez
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ec2-init (Ubuntu)
Fix Released
Medium
Scott Moser

Bug Description

Binary package hint: ec2-init

In cloud images, there is a race between ssh starting and hostkey generation/installation. That can create a (short) window where ssh uses the image built-in hostkeys, before it is restarted to make use of the new ones.

Revision history for this message
Thierry Carrez (ttx) wrote :

This can be easily reproduced on UEC instances (where you tend to know the IP in advance and can start trying to ssh in as soon as the vm is "running" :)

Revision history for this message
Eric Hammond (esh) wrote :

Changing this to "Confirmed" since I see this behavior occasionally on EC2 when I ssh in too quickly and then get a host key changed error when I ssh in again later.

Changed in ec2-init (Ubuntu):
status: New → Confirmed
Scott Moser (smoser)
Changed in ec2-init (Ubuntu):
importance: Undecided → Medium
Revision history for this message
Scott Moser (smoser) wrote :

The solution I'm going to implement is to remove ssh keys from the image, so there will be no "built-in hostkeys".
In testing, the ssh server will then still start, but attempts to ssh to it will fail:

$ sudo rm /etc/ssh/ssh_host_rsa_key*
$ sudo stop ssh
$ sudo start ssh
$ ssh localhost; echo $?
Read from socket failed: Connection reset by peer
255
$ sudo ssh-keygen -f /etc/ssh/ssh_host_rsa_key -t rsa -N "";
Generating public/private rsa key pair.
...
$ ssh localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
RSA key fingerprint is 32:44:17:eb:21:5d:a1:d3:e9:c0:57:f9:1e:e3:ce:a8.
Are you sure you want to continue connecting (yes/no)?
...

I think this is good enough, it means that the window is still present, but
a.) in the window there will be no ssh keys reported by 'ssh' or 'ssh-keyscan'
b.) ssh host /bin/true
  will exit with non-zero
c.) when keys are available, ssh server picks them up.

Changed in ec2-init (Ubuntu):
assignee: nobody → Scott Moser (smoser)
milestone: none → lucid-alpha-3
status: Confirmed → In Progress
Revision history for this message
Scott Moser (smoser) wrote :

just to point out here, Its a goal for ec2-init to be running early enough that it could actually block the start of ssh until the keys are generated.

Revision history for this message
Scott Moser (smoser) wrote :

This should not occur with latest images. There are no keys in the images.
If you try to connect before they're generated, you should get something like:
$ ssh localhost; echo $?
Read from socket failed: Connection reset by peer
255

Changed in ec2-init (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.