Merge lp:~chris-gondolin/charms/trusty/storage/trunk into lp:charms/storage
- Trusty Tahr (14.04)
- trunk
- Merge into trunk
Proposed by
Chris Stratford
| Status: | Merged |
|---|---|
| Merged at revision: | 39 |
| Proposed branch: | lp:~chris-gondolin/charms/trusty/storage/trunk |
| Merge into: | lp:charms/storage |
| Diff against target: |
988 lines (+732/-40) 9 files modified
config.yaml (+23/-8) files/crypt_mount.sh (+19/-0) hooks/common_util.py (+72/-20) hooks/crypt_fs.py (+164/-0) hooks/storage-provider.d/partition/data-relation-changed (+31/-0) hooks/test_common_util.py (+23/-12) hooks/test_crypt_fs.py (+248/-0) tests/10-crypt-keep-keyfile (+79/-0) tests/20-crypt-lose-keyfile (+73/-0) |
| To merge this branch: | bzr merge lp:~chris-gondolin/charms/trusty/storage/trunk |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Matt Bruzek (community) | Approve | ||
| Cory Johns (community) | Needs Fixing | ||
|
Review via email:
|
|||
Commit message
Description of the change
Added encrypted filesystem support for block devices.
Introduces two new config variables:
encryption_key_map = "{postgres/0: password, postgres/2: password2}"
store_encryptio
If store_encryptio
If store_encryptio
To post a comment you must log in.
Revision history for this message
| Tim Van Steenburgh (tvansteenburgh) wrote : | # |
- 41. By Chris Stratford
-
[chriss] Added a partition provider to cope with physical disk partitions (on MaaS units for example). Added some amulet tests that should cover both the new provider and encryption.
- 42. By Chris Stratford
-
[chriss] A bit more logging
- 43. By Chris Stratford
-
[chriss] Attempt to get juju test to log useful stuff
- 44. By Chris Stratford
-
[chriss] Removed spin_up_delay, as it did not do what I had hoped. Tests *should* work once bug #1378309 is fixed
Revision history for this message
| Cory Johns (johnsca) wrote : | # |
Chris,
Thanks for adding the tests for this functionality. The new tests look good, but unfortunately I had some issues running them.
First, I'd just like to point out that Amulet has been updated to fix the issues you are working around and mention in your tests, specifically the issues with the series and local charm not being used properly, and adding relations after deployment not working correctly. So, you can change your d.add('postgresql', 'cs:trusty/
That said, with those changes I am still getting failures (http://
Additionally, `make lint` had some complaints about the new code, just some small quibbles about line length and spacing: http://
Finally, some of the existing tests in `make test` are having issues with the new code. It mostly seems like they need some additional mocks (http://
review:
Needs Fixing
- 45. By Chris Stratford
-
[chriss] PEP8 fixes. Amulet test fixes now amulet itself is fixed
Revision history for this message
| Chris Stratford (chris-gondolin) wrote : | # |
The Amulet tests should now work (d.sentry.wait() after adding the relation doesn't appear to be enough, so there's a sleep in there too now, which should fix the errors you saw)
make lint should also be happy now.
I've got to do some reading up on mocker before fixing the make test problems.
- 46. By Chris Stratford
-
[chriss] Unit test fixes
Revision history for this message
| Chris Stratford (chris-gondolin) wrote : | # |
Ok, the unit tests now pass (even the ones that weren't a result of my changes :-)
There still aren't any for the crypt_fs functions or partition storage type additions (yet).
- 47. By Chris Stratford
-
[chriss] Add unit tests for crypt_fs. Minor fixes.
Revision history for this message
| Adam Collard (adam-collard) wrote : | # |
This broke the charm :(
http://
There is nothing in the charm that attempts to install cryptsetup-bin package (which installs the cryptsetup script that this MP relies on).
Revision history for this message
| Chris Stratford (chris-gondolin) wrote : | # |
Adam, I'm curious to know what system you installed it on to get that error, as it looks like cryptsetup-bin is standard on all recent Ubuntu installs.
I'll fix the bug, but it would be good to know for future reference.
Revision history for this message
| David Britton (dpb) wrote : | # |
Hi @Chris --
Adam tested on openstack with official cloud images. Matt tested on lxc. Could be a difference between the images there. :(
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
| 1 | === modified file 'config.yaml' |
| 2 | --- config.yaml 2014-02-10 21:51:04 +0000 |
| 3 | +++ config.yaml 2014-11-07 12:56:29 +0000 |
| 4 | @@ -8,21 +8,36 @@ |
| 5 | default: local |
| 6 | description: | |
| 7 | The backend storage provider. Will be mounted at root, and |
| 8 | - can be one of, local, block-storage-broker or nfs. If you set to |
| 9 | - block-storage-broker, you should provide a suitable value for |
| 10 | - volume_size. |
| 11 | + can be one of, local, block-storage-broker, nfs or partition. |
| 12 | + If you set to block-storage-broker, you should provide a suitable |
| 13 | + value for volume_size. |
| 14 | volume_map: |
| 15 | type: string |
| 16 | default: "" |
| 17 | description: | |
| 18 | YAML map as e.g. "{postgres/0: vol-0000010, postgres/1: vol-0000016}". |
| 19 | - A service unit to specific block storage volume-id that should be |
| 20 | - attached to each unit. This requires that provider is set to |
| 21 | - block-storage-broker and that volume-ids specified match a listing |
| 22 | - from nova volume-list. If a related unit does not have a volume-id |
| 23 | - specified, a new volume of volume_size will be created for that unit. |
| 24 | + A service unit to specific volume-id that should be attached to each |
| 25 | + unit. If the provider is set to block-storage-broker, volume-ids |
| 26 | + specified should match a listing from nova volume-list. |
| 27 | + If a related unit does not have a volume-id specified, a new volume |
| 28 | + of volume_size will be created for that unit. |
| 29 | + If the provider is set to partition, volume-ids should be a disk |
| 30 | + partition name (e.g. /dev/sdb1) |
| 31 | volume_size: |
| 32 | type: int |
| 33 | description: | |
| 34 | The volume size in GB to request from the block-storage-broker. |
| 35 | default: 5 |
| 36 | + encryption_key_map: |
| 37 | + type: string |
| 38 | + default: "" |
| 39 | + description: | |
| 40 | + YAML map as e.g. "{postgres/0: password, postgres/1: password2}" |
| 41 | + If set, the unit's volume will be encrypted with the matching key |
| 42 | + (unless the volume is already formatted) |
| 43 | + store_encryption_keys: |
| 44 | + type: string |
| 45 | + default: "yes" |
| 46 | + description: | |
| 47 | + If the volume is encrypted, this will write the details to |
| 48 | + disk to allow auto-mounting. |
| 49 | |
| 50 | === added directory 'files' |
| 51 | === added file 'files/crypt_mount.sh' |
| 52 | --- files/crypt_mount.sh 1970-01-01 00:00:00 +0000 |
| 53 | +++ files/crypt_mount.sh 2014-11-07 12:56:29 +0000 |
| 54 | @@ -0,0 +1,19 @@ |
| 55 | +#!/bin/bash |
| 56 | +# |
| 57 | +# Simple scrip to assist with mounting a crypted filesystem |
| 58 | +# Assumes an fstab entry already exists |
| 59 | + |
| 60 | +scriptname=$(basename $0) |
| 61 | +dev=$1 |
| 62 | + |
| 63 | +set -eu |
| 64 | + |
| 65 | +if [ "${dev}" = "" ]; then |
| 66 | + echo "Usage: ${scriptname} <device>" |
| 67 | + echo "e.g. ${scriptname} vdc" |
| 68 | + exit 1 |
| 69 | +fi |
| 70 | + |
| 71 | +cryptsetup open --type luks /dev/${dev} ${dev} |
| 72 | +mount /dev/mapper/${dev} |
| 73 | +exit 0 |
| 74 | |
| 75 | === modified file 'hooks/common_util.py' |
| 76 | --- hooks/common_util.py 2014-08-27 11:01:14 +0000 |
| 77 | +++ hooks/common_util.py 2014-11-07 12:56:29 +0000 |
| 78 | @@ -6,6 +6,8 @@ |
| 79 | from time import sleep |
| 80 | from stat import S_ISBLK, ST_MODE |
| 81 | import json |
| 82 | +import crypt_fs |
| 83 | + |
| 84 | |
| 85 | FSTAB_FILE = "/etc/fstab" |
| 86 | EC2_METADATA_URL = "http://169.254.169.254/latest/meta-data" |
| 87 | @@ -167,38 +169,81 @@ |
| 88 | |
| 89 | def _read_partition_table(device_path): |
| 90 | """Call blockdev to read a valid partition table and return exit code""" |
| 91 | + # blockdev doesn't like loopback, so we lie |
| 92 | + if device_path.startswith("/dev/loop"): |
| 93 | + return 0 |
| 94 | return subprocess.call( |
| 95 | "blockdev --rereadpt %s" % device_path, shell=True) |
| 96 | |
| 97 | |
| 98 | def _assert_fstype(device_path, fstype): |
| 99 | """Return C{True} if a filesystem is of type C{fstype}""" |
| 100 | - command = "file -s %s | egrep -q %s" % (device_path, fstype) |
| 101 | + # /dev/mapper/blah is usually a symlink |
| 102 | + real_device_path = os.path.realpath(device_path) |
| 103 | + command = "file -s %s | egrep -q %s" % (real_device_path, fstype) |
| 104 | return subprocess.call(command, shell=True) == 0 |
| 105 | |
| 106 | |
| 107 | +def _get_mkfs_command(device_path, encryption): |
| 108 | + mkfs_command = None |
| 109 | + encrypted_dev_path = None |
| 110 | + if encryption: |
| 111 | + if crypt_fs.is_encrypted(device_path): |
| 112 | + encrypted_dev_path = crypt_fs.get_encrypted_dev_path(device_path) |
| 113 | + log("Found an encrypted device {}".format(encrypted_dev_path)) |
| 114 | + else: |
| 115 | + encrypted_dev_path = crypt_fs.encrypt_device(device_path) |
| 116 | + mkfs_command = ["mkfs.ext4", encrypted_dev_path] |
| 117 | + crypt_fs.write_encryption_details(device_path) |
| 118 | + |
| 119 | + if _assert_fstype(encrypted_dev_path, "ext4"): |
| 120 | + log("{} already encrypted and formatted - " |
| 121 | + "skipping mkfs.ext4.".format(device_path)) |
| 122 | + mkfs_command = None |
| 123 | + else: |
| 124 | + mkfs_command = ["mkfs.ext4", device_path] |
| 125 | + return mkfs_command, encrypted_dev_path |
| 126 | + |
| 127 | + |
| 128 | def _format_device(device_path): |
| 129 | """ |
| 130 | Create an ext4 partition, if needed, for C{device_path} and fsck that |
| 131 | partition. |
| 132 | """ |
| 133 | + encryption = True if crypt_fs.get_encryption_key() else False |
| 134 | + command = [] |
| 135 | + |
| 136 | result = _read_partition_table(device_path) |
| 137 | if result == 1: |
| 138 | log("INFO: %s is busy, no fsck performed. Assuming formatted." % |
| 139 | device_path) |
| 140 | + encryption = False |
| 141 | elif result == 0: |
| 142 | # Create an ext4 filesystem if NOT already present |
| 143 | # use e.g. LABEl=vol-000012345 |
| 144 | if _assert_fstype(device_path, "ext4"): |
| 145 | log("%s already formatted - skipping mkfs.ext4." % device_path) |
| 146 | - else: |
| 147 | - command = "mkfs.ext4 %s" % device_path |
| 148 | - log("NOTICE: Running: %s" % command) |
| 149 | - subprocess.check_call(command, shell=True) |
| 150 | - if subprocess.call("fsck -p %s" % device_path, shell=True) != 0: |
| 151 | + encryption = False |
| 152 | + else: |
| 153 | + mkfs_command, encrypted_dev_path = _get_mkfs_command(device_path, |
| 154 | + encryption) |
| 155 | + if mkfs_command: |
| 156 | + log("NOTICE: Running: %s" % " ".join(mkfs_command)) |
| 157 | + subprocess.check_call(mkfs_command, shell=False) |
| 158 | + |
| 159 | + if encryption: |
| 160 | + command = ["fsck", "-p", encrypted_dev_path] |
| 161 | + else: |
| 162 | + command = ["fsck", "-p", device_path] |
| 163 | + if subprocess.call(command, shell=False) != 0: |
| 164 | log("ERROR: fsck -p %s failed" % device_path) |
| 165 | sys.exit(1) |
| 166 | |
| 167 | + if encryption: |
| 168 | + return encrypted_dev_path |
| 169 | + else: |
| 170 | + return device_path |
| 171 | + |
| 172 | |
| 173 | def _get_from_relation(relation, key, default=None): |
| 174 | relids = hookenv.relation_ids(relation) |
| 175 | @@ -213,8 +258,8 @@ |
| 176 | |
| 177 | def mount_volume(device_path=None): |
| 178 | """ |
| 179 | - Mount the attached volume, nfs or block-storage-broker type, to the |
| 180 | - principal requested C{mountpoint}. |
| 181 | + Mount the attached volume, partition, nfs or block-storage-broker |
| 182 | + type, to the principal requested C{mountpoint}. |
| 183 | |
| 184 | If the C{mountpoint} required relation data does not exist, log the |
| 185 | issue and exit. Otherwise attempt to initialize and mount the blockstorage |
| 186 | @@ -255,13 +300,16 @@ |
| 187 | "nfs-relation-changed hook needs to run first.") |
| 188 | return |
| 189 | |
| 190 | - device_path = "%s:%s" % (nfs_server, nfs_path) |
| 191 | - elif provider == "block-storage-broker": |
| 192 | - if device_path is None: |
| 193 | - log("Waiting for persistent nova device provided by " |
| 194 | - "block-storage-broker.") |
| 195 | - sys.exit(0) |
| 196 | + new_device_path = "%s:%s" % (nfs_server, nfs_path) |
| 197 | + elif provider in ("block-storage-broker", "partition"): |
| 198 | + # Specific to block-storage |
| 199 | + if provider == "block-storage-broker": |
| 200 | + if device_path is None: |
| 201 | + log("Waiting for persistent nova device provided by " |
| 202 | + "block-storage-broker.") |
| 203 | + sys.exit(0) |
| 204 | |
| 205 | + # Common to any partition-like device |
| 206 | _assert_block_device(device_path) |
| 207 | if os.path.exists("%s1" % device_path): |
| 208 | # Then we are supporting upgrade-charm path for deprecated |
| 209 | @@ -271,10 +319,14 @@ |
| 210 | device_path = "%s1" % device_path |
| 211 | log("Mounting %s as the device path as the root device is already " |
| 212 | "partitioned." % device_path) |
| 213 | - _format_device(device_path) |
| 214 | - _set_label(device_path, mountpoint) |
| 215 | + # _format_device may create a new device_path if it's encrypted |
| 216 | + new_device_path = _format_device(device_path) |
| 217 | + _set_label(new_device_path, mountpoint) |
| 218 | fstype = subprocess.check_output( |
| 219 | - "blkid -o value -s TYPE %s" % device_path, shell=True).strip() |
| 220 | + "blkid -o value -s TYPE %s" % new_device_path, shell=True).strip() |
| 221 | + if crypt_fs.is_encrypted(device_path): |
| 222 | + if hookenv.config("store_encryption_keys") != "yes": |
| 223 | + options = "defaults,noauto" |
| 224 | else: |
| 225 | log("ERROR: Unknown provider %s. Cannot mount volume." % hookenv.ERROR) |
| 226 | sys.exit(1) |
| 227 | @@ -284,12 +336,12 @@ |
| 228 | |
| 229 | options_string = "" if options == "defaults" else " -o %s" % options |
| 230 | command = "mount -t %s%s %s %s" % ( |
| 231 | - fstype, options_string, device_path, mountpoint) |
| 232 | + fstype, options_string, new_device_path, mountpoint) |
| 233 | subprocess.check_call(command, shell=True) |
| 234 | log("Mount (%s) successful" % mountpoint) |
| 235 | with open(FSTAB_FILE, "a") as output_file: |
| 236 | - output_file.write( |
| 237 | - "%s %s %s %s 0 0\n" % (device_path, mountpoint, fstype, options)) |
| 238 | + output_file.write("{} {} {} {} 0 0\n".format( |
| 239 | + new_device_path, mountpoint, fstype, options)) |
| 240 | # publish changes to data relation and exit |
| 241 | for relid in hookenv.relation_ids("data"): |
| 242 | hookenv.relation_set( |
| 243 | |
| 244 | === added file 'hooks/crypt_fs.py' |
| 245 | --- hooks/crypt_fs.py 1970-01-01 00:00:00 +0000 |
| 246 | +++ hooks/crypt_fs.py 2014-11-07 12:56:29 +0000 |
| 247 | @@ -0,0 +1,164 @@ |
| 248 | +"""Common python utilities for storage providers""" |
| 249 | +from charmhelpers.core import hookenv |
| 250 | +import os |
| 251 | +import subprocess |
| 252 | +import sys |
| 253 | +import yaml |
| 254 | +from yaml.constructor import ConstructorError |
| 255 | +import shutil |
| 256 | + |
| 257 | + |
| 258 | +CRYPTTAB_FILE = "/etc/crypttab" |
| 259 | +KEYFILE_DIR = "/root" |
| 260 | +CRYPT_MOUNT_SCRIPT = "/usr/local/sbin/crypt_mount.sh" |
| 261 | + |
| 262 | + |
| 263 | +def log(message, level=None): |
| 264 | + """Quaint little log wrapper for juju logging""" |
| 265 | + hookenv.log(message, level) |
| 266 | + |
| 267 | + |
| 268 | +# Format a Luks device |
| 269 | +def encrypt_device(device_path): |
| 270 | + keyfile = _write_encryption_keyfile(device_path) |
| 271 | + # Create the encrypted volume |
| 272 | + command = ["cryptsetup", |
| 273 | + "--key-file", keyfile, |
| 274 | + "--batch-mode", |
| 275 | + "luksFormat", device_path] |
| 276 | + log("NOTICE: Running: {}".format(command)) |
| 277 | + if subprocess.call(command, shell=False) != 0: |
| 278 | + log("ERROR: {} failed".format(command)) |
| 279 | + sys.exit(1) |
| 280 | + return get_encrypted_dev_path(device_path) |
| 281 | + |
| 282 | + |
| 283 | +# Return the Luks device path if it exists |
| 284 | +# Perform a LuksOpen if it doesn't |
| 285 | +def get_encrypted_dev_path(device_path): |
| 286 | + keyfile = _write_encryption_keyfile(device_path) |
| 287 | + newname = "/dev/mapper/{}".format(os.path.basename(device_path)) |
| 288 | + if os.path.exists(newname): |
| 289 | + return newname |
| 290 | + else: |
| 291 | + basedev = os.path.basename(device_path) |
| 292 | + command = ["cryptsetup", |
| 293 | + "--key-file", keyfile, |
| 294 | + "open", |
| 295 | + "--type", "luks", |
| 296 | + device_path, |
| 297 | + basedev] |
| 298 | + log("NOTICE: Running: {}".format(command)) |
| 299 | + if subprocess.call(command, shell=False) != 0: |
| 300 | + log("ERROR: {} failed".format(command)) |
| 301 | + sys.exit(1) |
| 302 | + return newname |
| 303 | + |
| 304 | + |
| 305 | +def is_encrypted(device_path): |
| 306 | + command = ["cryptsetup", "isLuks", device_path] |
| 307 | + if subprocess.call(command, shell=False) == 0: |
| 308 | + return True |
| 309 | + else: |
| 310 | + return False |
| 311 | + |
| 312 | + |
| 313 | +def _get_encryption_uuid(device_path): |
| 314 | + command = ["cryptsetup", "luksUUID", device_path] |
| 315 | + try: |
| 316 | + uuid = subprocess.check_output(command, shell=False).rstrip() |
| 317 | + return uuid |
| 318 | + except subprocess.CalledProcessError: |
| 319 | + log("ERROR: {} failed".format(command)) |
| 320 | + return None |
| 321 | + |
| 322 | + |
| 323 | +def get_encryption_key(): |
| 324 | + # Get first unit of first data relation |
| 325 | + # (hopefully only unit of only data relation) |
| 326 | + relid = hookenv.relation_ids("data")[0] |
| 327 | + unit = hookenv.related_units(relid)[0] |
| 328 | + log("get_encryption_key() unit: {}".format(unit)) |
| 329 | + encryption_key_map = hookenv.config("encryption_key_map") |
| 330 | + encryption_key = None |
| 331 | + if encryption_key_map and unit: |
| 332 | + log("get_encryption_key() Have map and unit") |
| 333 | + try: |
| 334 | + encryption_key_map = yaml.load(encryption_key_map) |
| 335 | + if encryption_key_map: |
| 336 | + encryption_key = encryption_key_map.get(unit, None) |
| 337 | + except ConstructorError as e: |
| 338 | + log("Invalid YAML in 'encryption_key_map': {}".format(e), |
| 339 | + hookenv.WARNING) |
| 340 | + return None |
| 341 | + if not encryption_key: |
| 342 | + log("get_encryption_key() Failed to find key") |
| 343 | + return encryption_key |
| 344 | + |
| 345 | + |
| 346 | +def _get_encryption_keyfile_name(device_path): |
| 347 | + return "{}/keyfile-{}".format(KEYFILE_DIR, device_path.replace("/", "-")) |
| 348 | + |
| 349 | + |
| 350 | +# Store the encryption key, as it's needed by the Luks commands |
| 351 | +# We will delete it later if the store_encryption_keys config variable |
| 352 | +# is not "yes" |
| 353 | +def _write_encryption_keyfile(device_path): |
| 354 | + encryption_key = get_encryption_key() |
| 355 | + keyfile = _get_encryption_keyfile_name(device_path) |
| 356 | + # Write out the keyfile |
| 357 | + log("Writing keyfile: {}".format(keyfile)) |
| 358 | + with open(keyfile, "w") as f: |
| 359 | + os.chmod(keyfile, 0600) |
| 360 | + f.write(encryption_key) |
| 361 | + return keyfile |
| 362 | + |
| 363 | + |
| 364 | +def _delete_encryption_keyfile(device_path): |
| 365 | + keyfile = _get_encryption_keyfile_name(device_path) |
| 366 | + if os.path.isfile(keyfile): |
| 367 | + log("Deleting keyfile: {}".format(keyfile)) |
| 368 | + os.unlink(keyfile) |
| 369 | + |
| 370 | + |
| 371 | +# Write the encryption details to the FS to allow auto-mounting |
| 372 | +# If store_encryption_keys config option is not "yes", this will |
| 373 | +# remove the current device_path from the file |
| 374 | +def write_encryption_details(device_path): |
| 375 | + keyfile = _write_encryption_keyfile(device_path) |
| 376 | + basedev = os.path.basename(device_path) |
| 377 | + uuid = _get_encryption_uuid(device_path) |
| 378 | + newCrypttabFile = CRYPTTAB_FILE + ".new" |
| 379 | + if uuid is not None: |
| 380 | + log("Writing encryption details to {}".format(CRYPTTAB_FILE)) |
| 381 | + # Read original |
| 382 | + lines = [] |
| 383 | + if os.path.exists(CRYPTTAB_FILE): |
| 384 | + with open(CRYPTTAB_FILE, "r") as fr: |
| 385 | + lines = fr.readlines() |
| 386 | + |
| 387 | + # Write new (removing any existing lines for our device) |
| 388 | + with open(newCrypttabFile, "w") as fw: |
| 389 | + for line in lines: |
| 390 | + if not line.startswith((basedev + " ", basedev + "\t")): |
| 391 | + fw.write(line) |
| 392 | + if hookenv.config("store_encryption_keys") == "yes": |
| 393 | + fw.write("{} UUID={} {} luks\n".format(basedev, uuid, keyfile)) |
| 394 | + |
| 395 | + # Move into place |
| 396 | + if os.path.exists(CRYPTTAB_FILE): |
| 397 | + os.rename(CRYPTTAB_FILE, CRYPTTAB_FILE + ".orig") |
| 398 | + os.rename(newCrypttabFile, CRYPTTAB_FILE) |
| 399 | + |
| 400 | + # Remove the keyfile if we don't want it saved |
| 401 | + if hookenv.config("store_encryption_keys") != "yes": |
| 402 | + _delete_encryption_keyfile(device_path) |
| 403 | + else: |
| 404 | + log("Unable to write encryption details to".format(CRYPTTAB_FILE)) |
| 405 | + |
| 406 | + # Add a simple script to help mount the crypted filesystem |
| 407 | + # (only needed is not auto-mounting) |
| 408 | + src = os.path.join(hookenv.charm_dir(), "files", "crypt_mount.sh") |
| 409 | + dest = "/usr/local/sbin/crypt_mount.sh" |
| 410 | + shutil.copyfile(src, dest) |
| 411 | + os.chmod(dest, 0755) |
| 412 | |
| 413 | === added directory 'hooks/storage-provider.d/partition' |
| 414 | === added file 'hooks/storage-provider.d/partition/data-relation-changed' |
| 415 | --- hooks/storage-provider.d/partition/data-relation-changed 1970-01-01 00:00:00 +0000 |
| 416 | +++ hooks/storage-provider.d/partition/data-relation-changed 2014-11-07 12:56:29 +0000 |
| 417 | @@ -0,0 +1,31 @@ |
| 418 | +#!/usr/bin/python |
| 419 | + |
| 420 | +import sys |
| 421 | +import yaml |
| 422 | +from yaml.constructor import ConstructorError |
| 423 | + |
| 424 | +from charmhelpers.core import hookenv |
| 425 | +import common_util |
| 426 | + |
| 427 | +log = hookenv.log |
| 428 | + |
| 429 | +partition = None |
| 430 | +volume_map = hookenv.config().get("volume_map", None) |
| 431 | +relids = hookenv.relation_ids("data") |
| 432 | +for relid in relids: |
| 433 | + for unit in hookenv.related_units(relid): |
| 434 | + if volume_map is not None: |
| 435 | + try: |
| 436 | + volume_map = yaml.load(volume_map) |
| 437 | + if volume_map: |
| 438 | + partition = volume_map.get(unit, None) |
| 439 | + except ConstructorError as e: |
| 440 | + hookenv.log( |
| 441 | + "invalid YAML in 'volume-map': {}".format(e), |
| 442 | + hookenv.WARNING) |
| 443 | + |
| 444 | +if partition is None: |
| 445 | + log("No data-relation-changed hook fired. Awaiting data-relation") |
| 446 | + sys.exit(0) |
| 447 | + |
| 448 | +common_util.mount_volume(partition) |
| 449 | |
| 450 | === modified file 'hooks/test_common_util.py' |
| 451 | --- hooks/test_common_util.py 2014-03-21 22:53:22 +0000 |
| 452 | +++ hooks/test_common_util.py 2014-11-07 12:56:29 +0000 |
| 453 | @@ -1,4 +1,5 @@ |
| 454 | import common_util as util |
| 455 | +import crypt_fs |
| 456 | import json |
| 457 | import mocker |
| 458 | import os |
| 459 | @@ -13,6 +14,7 @@ |
| 460 | self.maxDiff = None |
| 461 | util.FSTAB_FILE = self.makeFile() |
| 462 | util.hookenv = TestHookenv({"provider": "nfs"}) |
| 463 | + crypt_fs.hookenv = TestHookenv() |
| 464 | self.addCleanup(setattr, os, "environ", os.environ) |
| 465 | self.charm_dir = self.makeDir() |
| 466 | os.environ = {"CHARM_DIR": self.charm_dir} |
| 467 | @@ -537,7 +539,7 @@ |
| 468 | |
| 469 | # Assert fsck is called |
| 470 | fsck = self.mocker.replace(subprocess.call) |
| 471 | - fsck("fsck -p /dev/vdz", shell=True) |
| 472 | + fsck(["fsck", "-p", "/dev/vdz"], shell=False) |
| 473 | self.mocker.result(0) # successful fsck command exit |
| 474 | self.mocker.replay() |
| 475 | |
| 476 | @@ -561,9 +563,9 @@ |
| 477 | ext4_check("/dev/vdz", "ext4") |
| 478 | self.mocker.result(False) |
| 479 | fsck = self.mocker.replace(subprocess.check_call) |
| 480 | - fsck("mkfs.ext4 /dev/vdz", shell=True) |
| 481 | + fsck(["mkfs.ext4", "/dev/vdz"], shell=False) |
| 482 | fsck = self.mocker.replace(subprocess.call) |
| 483 | - fsck("fsck -p /dev/vdz", shell=True) |
| 484 | + fsck(["fsck", "-p", "/dev/vdz"], shell=False) |
| 485 | self.mocker.result(0) # Sucessful command exit 0 |
| 486 | self.mocker.replay() |
| 487 | |
| 488 | @@ -586,9 +588,9 @@ |
| 489 | ext4_check("/dev/vdz", "ext4") |
| 490 | self.mocker.result(False) |
| 491 | fsck = self.mocker.replace(subprocess.check_call) |
| 492 | - fsck("mkfs.ext4 /dev/vdz", shell=True) |
| 493 | + fsck(["mkfs.ext4", "/dev/vdz"], shell=False) |
| 494 | fsck = self.mocker.replace(subprocess.call) |
| 495 | - fsck("fsck -p /dev/vdz", shell=True) |
| 496 | + fsck(["fsck", "-p", "/dev/vdz"], shell=False) |
| 497 | self.mocker.result(1) |
| 498 | self.mocker.replay() |
| 499 | |
| 500 | @@ -733,7 +735,7 @@ |
| 501 | util.hookenv.is_logged(message), "Not logged- %s" % message) |
| 502 | |
| 503 | # Wrote proper fstab mount info to mount on reboot |
| 504 | - fstab_content = "me.com:/nfs/server/path /mnt/this nfs someopts 0 0" |
| 505 | + fstab_content = "me.com:/nfs/server/path /mnt/this nfs someopts 0 0\n" |
| 506 | with open(util.FSTAB_FILE) as input_file: |
| 507 | self.assertEqual(input_file.read(), fstab_content) |
| 508 | |
| 509 | @@ -789,9 +791,9 @@ |
| 510 | exists = self.mocker.replace(os.path.exists) |
| 511 | exists("/dev/vdx1") |
| 512 | self.mocker.result(False) |
| 513 | - create_partition = self.mocker.replace( |
| 514 | - util._format_device) |
| 515 | + create_partition = self.mocker.replace(util._format_device) |
| 516 | create_partition("/dev/vdx") |
| 517 | + self.mocker.result("/dev/vdx") |
| 518 | |
| 519 | _set_label = self.mocker.replace(util._set_label) |
| 520 | _set_label("/dev/vdx", "/mnt/this") |
| 521 | @@ -806,6 +808,11 @@ |
| 522 | |
| 523 | mount = self.mocker.replace(subprocess.check_call) |
| 524 | mount("mount -t ext4 /dev/vdx /mnt/this", shell=True) |
| 525 | + |
| 526 | + is_encrypted = self.mocker.replace(crypt_fs.is_encrypted) |
| 527 | + is_encrypted("/dev/vdx") |
| 528 | + self.mocker.result(False) |
| 529 | + |
| 530 | self.mocker.replay() |
| 531 | |
| 532 | self.assertEqual(util.get_provider(), "block-storage-broker") |
| 533 | @@ -815,7 +822,7 @@ |
| 534 | util.hookenv.is_logged(message), "Not logged- %s" % message) |
| 535 | |
| 536 | # Wrote proper fstab mount info to mount on reboot |
| 537 | - fstab_content = "/dev/vdx /mnt/this ext4 default 0 0" |
| 538 | + fstab_content = "/dev/vdx /mnt/this ext4 defaults 0 0\n" |
| 539 | with open(util.FSTAB_FILE) as input_file: |
| 540 | self.assertEqual(input_file.read(), fstab_content) |
| 541 | |
| 542 | @@ -849,9 +856,9 @@ |
| 543 | exists = self.mocker.replace(os.path.exists) |
| 544 | exists("/dev/vdx1") |
| 545 | self.mocker.result(True) |
| 546 | - create_partition = self.mocker.replace( |
| 547 | - util._format_device) |
| 548 | + create_partition = self.mocker.replace(util._format_device) |
| 549 | create_partition("/dev/vdx1") |
| 550 | + self.mocker.result("/dev/vdx1") |
| 551 | |
| 552 | _set_label = self.mocker.replace(util._set_label) |
| 553 | _set_label("/dev/vdx1", "/mnt/this") |
| 554 | @@ -861,6 +868,10 @@ |
| 555 | get_fstype("blkid -o value -s TYPE /dev/vdx1", shell=True) |
| 556 | self.mocker.result("ext4\n") |
| 557 | |
| 558 | + is_encrypted = self.mocker.replace(crypt_fs.is_encrypted) |
| 559 | + is_encrypted("/dev/vdx1") |
| 560 | + self.mocker.result(False) |
| 561 | + |
| 562 | exists("/mnt/this") |
| 563 | self.mocker.result(True) |
| 564 | |
| 565 | @@ -878,7 +889,7 @@ |
| 566 | util.hookenv.is_logged(message), "Not logged- %s" % message) |
| 567 | |
| 568 | # Wrote proper fstab mount info to mount on reboot |
| 569 | - fstab_content = "/dev/vdx1 /mnt/this ext4 default 0 0" |
| 570 | + fstab_content = "/dev/vdx1 /mnt/this ext4 defaults 0 0\n" |
| 571 | with open(util.FSTAB_FILE) as input_file: |
| 572 | self.assertEqual(input_file.read(), fstab_content) |
| 573 | |
| 574 | |
| 575 | === added file 'hooks/test_crypt_fs.py' |
| 576 | --- hooks/test_crypt_fs.py 1970-01-01 00:00:00 +0000 |
| 577 | +++ hooks/test_crypt_fs.py 2014-11-07 12:56:29 +0000 |
| 578 | @@ -0,0 +1,248 @@ |
| 579 | +import crypt_fs |
| 580 | +import os |
| 581 | +import shutil |
| 582 | +import mocker |
| 583 | +from testing import TestHookenv |
| 584 | + |
| 585 | + |
| 586 | +class TestCryptFs(mocker.MockerTestCase): |
| 587 | + |
| 588 | + def setUp(self): |
| 589 | + self.maxDiff = None |
| 590 | + crypt_fs.hookenv = TestHookenv() |
| 591 | + crypt_fs.KEYFILE_DIR = "/tmp" |
| 592 | + crypt_fs.CRYPTTAB_FILE = "/tmp/crypttab" |
| 593 | + self.addCleanup(setattr, os, "environ", os.environ) |
| 594 | + self.charm_dir = self.makeDir() |
| 595 | + os.environ = {"CHARM_DIR": self.charm_dir} |
| 596 | + |
| 597 | + def test_get_encryption_key_if_exists(self): |
| 598 | + """ |
| 599 | + L{get_encryption_key} returns the encryption key for a |
| 600 | + specific unit |
| 601 | + """ |
| 602 | + crypt_fs.hookenv._config = ( |
| 603 | + ("encryption_key_map", "{data/0: password1}"), |
| 604 | + ) |
| 605 | + self.assertEqual(crypt_fs.get_encryption_key(), "password1") |
| 606 | + |
| 607 | + def test_get_encryption_key_if_not_exists(self): |
| 608 | + """ |
| 609 | + L{get_encryption_key} returns None when no key found |
| 610 | + """ |
| 611 | + crypt_fs.hookenv._config = ( |
| 612 | + ("encryption_key_map", "{data/1: password1}"), |
| 613 | + ) |
| 614 | + self.assertEqual(crypt_fs.get_encryption_key(), None) |
| 615 | + |
| 616 | + def test_write_encryption_keyfile(self): |
| 617 | + """ |
| 618 | + L{_write_encryption_keyfile} writes the key to disk in |
| 619 | + crypt_fs.KEYFILE_DIR |
| 620 | + """ |
| 621 | + crypt_fs.hookenv._config = ( |
| 622 | + ("encryption_key_map", "{data/0: password1}"), |
| 623 | + ) |
| 624 | + keyfile_path = os.path.join(crypt_fs.KEYFILE_DIR, |
| 625 | + "keyfile--dev-something") |
| 626 | + if os.path.exists(keyfile_path): |
| 627 | + os.unlink(keyfile_path) |
| 628 | + crypt_fs._write_encryption_keyfile("/dev/something") |
| 629 | + self.assertTrue(os.path.exists(keyfile_path)) |
| 630 | + with open(keyfile_path) as infile: |
| 631 | + key = infile.read() |
| 632 | + self.assertEqual(key, "password1") |
| 633 | + |
| 634 | + def test_delete_encryption_keyfile(self): |
| 635 | + """ |
| 636 | + L{_delete_encryption_keyfile} should remove the key from |
| 637 | + crypt_fs.KEYFILE_DIR |
| 638 | + """ |
| 639 | + keyfile_path = os.path.join(crypt_fs.KEYFILE_DIR, |
| 640 | + "keyfile--dev-something") |
| 641 | + with open(keyfile_path, "w") as outfile: |
| 642 | + outfile.write("test") |
| 643 | + self.assertTrue(os.path.exists(keyfile_path)) |
| 644 | + crypt_fs._delete_encryption_keyfile("/dev/something") |
| 645 | + self.assertFalse(os.path.exists(keyfile_path)) |
| 646 | + |
| 647 | + def test_write_encryption_details_no_uuid_no_crypttab(self): |
| 648 | + """ |
| 649 | + L{write_encryption_details} Adds a crypttab entry for |
| 650 | + the newly created filesystem. |
| 651 | + It should leave it well alone for this test |
| 652 | + """ |
| 653 | + crypt_fs.hookenv._config = ( |
| 654 | + ("encryption_key_map", "{data/0: password1}"), |
| 655 | + ) |
| 656 | + |
| 657 | + encryption_uuid = self.mocker.replace(crypt_fs._get_encryption_uuid) |
| 658 | + encryption_uuid("/dev/absent") |
| 659 | + self.mocker.result(None) |
| 660 | + |
| 661 | + copyfile = self.mocker.replace(shutil.copyfile) |
| 662 | + src = os.path.join(crypt_fs.hookenv.charm_dir(), |
| 663 | + "files", |
| 664 | + "crypt_mount.sh") |
| 665 | + copyfile(src, "/usr/local/sbin/crypt_mount.sh") |
| 666 | + |
| 667 | + chmod = self.mocker.replace(os.chmod) |
| 668 | + chmod("/usr/local/sbin/crypt_mount.sh", 0755) |
| 669 | + |
| 670 | + self.mocker.replay() |
| 671 | + |
| 672 | + if os.path.exists(crypt_fs.CRYPTTAB_FILE): |
| 673 | + os.unlink(crypt_fs.CRYPTTAB_FILE) |
| 674 | + self.assertFalse(os.path.exists(crypt_fs.CRYPTTAB_FILE)) |
| 675 | + crypt_fs.write_encryption_details("/dev/absent") |
| 676 | + self.assertFalse(os.path.exists(crypt_fs.CRYPTTAB_FILE)) |
| 677 | + |
| 678 | + def test_write_encryption_details_no_uuid_have_crypttab(self): |
| 679 | + """ |
| 680 | + L{write_encryption_details} Adds a crypttab entry for |
| 681 | + the newly created filesystem |
| 682 | + It should leave the existing file untouched in this test |
| 683 | + """ |
| 684 | + crypt_fs.hookenv._config = ( |
| 685 | + ("encryption_key_map", "{data/0: password1}"), |
| 686 | + ) |
| 687 | + |
| 688 | + encryption_uuid = self.mocker.replace(crypt_fs._get_encryption_uuid) |
| 689 | + encryption_uuid("/dev/absent") |
| 690 | + self.mocker.result(None) |
| 691 | + |
| 692 | + copyfile = self.mocker.replace(shutil.copyfile) |
| 693 | + src = os.path.join(crypt_fs.hookenv.charm_dir(), |
| 694 | + "files", |
| 695 | + "crypt_mount.sh") |
| 696 | + copyfile(src, "/usr/local/sbin/crypt_mount.sh") |
| 697 | + |
| 698 | + chmod = self.mocker.replace(os.chmod) |
| 699 | + chmod("/usr/local/sbin/crypt_mount.sh", 0755) |
| 700 | + |
| 701 | + self.mocker.replay() |
| 702 | + |
| 703 | + crypttab_content = "sda_crypt UUID=SOME-EXISTING-UUID none luks\n" |
| 704 | + with open(crypt_fs.CRYPTTAB_FILE, "w") as outfile: |
| 705 | + outfile.write(crypttab_content) |
| 706 | + self.assertTrue(os.path.exists(crypt_fs.CRYPTTAB_FILE)) |
| 707 | + crypt_fs.write_encryption_details("/dev/absent") |
| 708 | + with open(crypt_fs.CRYPTTAB_FILE) as infile: |
| 709 | + self.assertEqual(infile.read(), crypttab_content) |
| 710 | + |
| 711 | + def test_write_encryption_details_have_uuid_no_crypttab(self): |
| 712 | + """ |
| 713 | + L{write_encryption_details} Adds a crypttab entry for |
| 714 | + the newly created filesystem |
| 715 | + It should leave the existing file untouched in this test |
| 716 | + """ |
| 717 | + crypt_fs.hookenv._config = ( |
| 718 | + ("encryption_key_map", "{data/0: password1}"), |
| 719 | + ("store_encryption_keys", "yes"), |
| 720 | + ) |
| 721 | + |
| 722 | + encryption_uuid = self.mocker.replace(crypt_fs._get_encryption_uuid) |
| 723 | + encryption_uuid("/dev/something") |
| 724 | + self.mocker.result("SOME-NEW-UUID") |
| 725 | + |
| 726 | + copyfile = self.mocker.replace(shutil.copyfile) |
| 727 | + src = os.path.join(crypt_fs.hookenv.charm_dir(), |
| 728 | + "files", |
| 729 | + "crypt_mount.sh") |
| 730 | + copyfile(src, "/usr/local/sbin/crypt_mount.sh") |
| 731 | + |
| 732 | + chmod = self.mocker.replace(os.chmod) |
| 733 | + chmod("/usr/local/sbin/crypt_mount.sh", 0755) |
| 734 | + |
| 735 | + self.mocker.replay() |
| 736 | + |
| 737 | + if os.path.exists(crypt_fs.CRYPTTAB_FILE): |
| 738 | + os.unlink(crypt_fs.CRYPTTAB_FILE) |
| 739 | + self.assertFalse(os.path.exists(crypt_fs.CRYPTTAB_FILE)) |
| 740 | + crypt_fs.write_encryption_details("/dev/something") |
| 741 | + keyfile_path = os.path.join(crypt_fs.KEYFILE_DIR, |
| 742 | + "keyfile--dev-something") |
| 743 | + crypttab_content = ( |
| 744 | + "something UUID=SOME-NEW-UUID {} luks\n".format(keyfile_path)) |
| 745 | + with open(crypt_fs.CRYPTTAB_FILE) as infile: |
| 746 | + self.assertEqual(infile.read(), crypttab_content) |
| 747 | + |
| 748 | + def test_write_encryption_details_have_uuid_have_crypttab(self): |
| 749 | + """ |
| 750 | + L{write_encryption_details} Adds a crypttab entry for |
| 751 | + the newly created filesystem |
| 752 | + It should add to the file for this test |
| 753 | + """ |
| 754 | + crypt_fs.hookenv._config = ( |
| 755 | + ("encryption_key_map", "{data/0: password1}"), |
| 756 | + ("store_encryption_keys", "yes"), |
| 757 | + ) |
| 758 | + |
| 759 | + encryption_uuid = self.mocker.replace(crypt_fs._get_encryption_uuid) |
| 760 | + encryption_uuid("/dev/something") |
| 761 | + self.mocker.result("SOME-NEW-UUID") |
| 762 | + |
| 763 | + copyfile = self.mocker.replace(shutil.copyfile) |
| 764 | + src = os.path.join(crypt_fs.hookenv.charm_dir(), |
| 765 | + "files", |
| 766 | + "crypt_mount.sh") |
| 767 | + copyfile(src, "/usr/local/sbin/crypt_mount.sh") |
| 768 | + |
| 769 | + chmod = self.mocker.replace(os.chmod) |
| 770 | + chmod("/usr/local/sbin/crypt_mount.sh", 0755) |
| 771 | + |
| 772 | + self.mocker.replay() |
| 773 | + |
| 774 | + crypttab_content = "sda_crypt UUID=SOME-EXISTING-UUID none luks\n" |
| 775 | + with open(crypt_fs.CRYPTTAB_FILE, "w") as outfile: |
| 776 | + outfile.write(crypttab_content) |
| 777 | + self.assertTrue(os.path.exists(crypt_fs.CRYPTTAB_FILE)) |
| 778 | + crypt_fs.write_encryption_details("/dev/something") |
| 779 | + keyfile_path = os.path.join(crypt_fs.KEYFILE_DIR, |
| 780 | + "keyfile--dev-something") |
| 781 | + crypttab_content = ( |
| 782 | + "sda_crypt UUID=SOME-EXISTING-UUID none luks\n" |
| 783 | + "something UUID=SOME-NEW-UUID {} luks\n".format(keyfile_path)) |
| 784 | + with open(crypt_fs.CRYPTTAB_FILE) as infile: |
| 785 | + self.assertEqual(infile.read(), crypttab_content) |
| 786 | + |
| 787 | + def test_write_encryption_details_have_uuid_have_duplicate_crypttab(self): |
| 788 | + """ |
| 789 | + L{write_encryption_details} Adds a crypttab entry for |
| 790 | + the newly created filesystem |
| 791 | + It should replace an existing entry in this test |
| 792 | + """ |
| 793 | + crypt_fs.hookenv._config = ( |
| 794 | + ("encryption_key_map", "{data/0: password1}"), |
| 795 | + ("store_encryption_keys", "yes"), |
| 796 | + ) |
| 797 | + |
| 798 | + encryption_uuid = self.mocker.replace(crypt_fs._get_encryption_uuid) |
| 799 | + encryption_uuid("/dev/something") |
| 800 | + self.mocker.result("SOME-NEW-UUID") |
| 801 | + |
| 802 | + copyfile = self.mocker.replace(shutil.copyfile) |
| 803 | + src = os.path.join(crypt_fs.hookenv.charm_dir(), |
| 804 | + "files", |
| 805 | + "crypt_mount.sh") |
| 806 | + copyfile(src, "/usr/local/sbin/crypt_mount.sh") |
| 807 | + |
| 808 | + chmod = self.mocker.replace(os.chmod) |
| 809 | + chmod("/usr/local/sbin/crypt_mount.sh", 0755) |
| 810 | + |
| 811 | + self.mocker.replay() |
| 812 | + |
| 813 | + crypttab_content = ( |
| 814 | + "sda_crypt UUID=SOME-EXISTING-UUID none luks\n" |
| 815 | + "something UUID=AN-OLD-UUID none luks\n") |
| 816 | + with open(crypt_fs.CRYPTTAB_FILE, "w") as outfile: |
| 817 | + outfile.write(crypttab_content) |
| 818 | + self.assertTrue(os.path.exists(crypt_fs.CRYPTTAB_FILE)) |
| 819 | + crypt_fs.write_encryption_details("/dev/something") |
| 820 | + keyfile_path = os.path.join(crypt_fs.KEYFILE_DIR, |
| 821 | + "keyfile--dev-something") |
| 822 | + crypttab_content = ( |
| 823 | + "sda_crypt UUID=SOME-EXISTING-UUID none luks\n" |
| 824 | + "something UUID=SOME-NEW-UUID {} luks\n".format(keyfile_path)) |
| 825 | + with open(crypt_fs.CRYPTTAB_FILE) as infile: |
| 826 | + self.assertEqual(infile.read(), crypttab_content) |
| 827 | |
| 828 | === added file 'tests/10-crypt-keep-keyfile' |
| 829 | --- tests/10-crypt-keep-keyfile 1970-01-01 00:00:00 +0000 |
| 830 | +++ tests/10-crypt-keep-keyfile 2014-11-07 12:56:29 +0000 |
| 831 | @@ -0,0 +1,79 @@ |
| 832 | +#!/usr/bin/python3 |
| 833 | +# |
| 834 | +# Tests the partition provider with disk encryption |
| 835 | +# Almost identical to 20-crypt-lose-keyfile only we want to |
| 836 | +# make sure everything works AND the keyfile still exists |
| 837 | + |
| 838 | +import amulet |
| 839 | +import time |
| 840 | +import logging |
| 841 | + |
| 842 | +d = amulet.Deployment(series="trusty") |
| 843 | + |
| 844 | +# Add units |
| 845 | +d.add('postgresql', 'postgresql') |
| 846 | +d.add('storage', 'storage') |
| 847 | + |
| 848 | +# Set the unit configs |
| 849 | +d.configure("storage", {"provider": "partition", "volume_map": "{postgresql/0: /dev/loop0}", "encryption_key_map": "{postgresql/0: password1}", "store_encryption_keys": "yes"}) |
| 850 | + |
| 851 | +try: |
| 852 | + d.setup(timeout=900) |
| 853 | + d.sentry.wait() |
| 854 | +except amulet.helpers.TimeoutError: |
| 855 | + amulet.raise_status(amulet.FAIL, msg="Environment wasn't stood up in time") |
| 856 | +except: |
| 857 | + raise |
| 858 | + |
| 859 | +unit = d.sentry['postgresql/0'] |
| 860 | + |
| 861 | +# Create a test device |
| 862 | +if unit.run("dd if=/dev/zero of=/testfile bs=1024 count=10240 2>&1")[1] != 0: |
| 863 | + amulet.raise_status(amulet.FAIL, msg="dd failed creating /testfile") |
| 864 | +if unit.run("losetup /dev/loop0 /testfile")[1] != 0: |
| 865 | + amulet.raise_status(amulet.FAIL, msg="losetup failed creating /dev/loop0") |
| 866 | +time.sleep(10) |
| 867 | + |
| 868 | +# Add relations |
| 869 | +d.relate('postgresql:data', 'storage:data') |
| 870 | +d.sentry.wait() |
| 871 | +time.sleep(30) |
| 872 | + |
| 873 | + |
| 874 | +# Check mounts |
| 875 | +mounts = unit.run("cat /proc/mounts")[0].split("\n") |
| 876 | +mountsOk = False |
| 877 | +for line in mounts: |
| 878 | + if "/dev/mapper/loop0" in line and "/srv/data" in line: |
| 879 | + mountsOk = True |
| 880 | +if not mountsOk: |
| 881 | + amulet.raise_status(amulet.FAIL, msg="/dev/mapper/loop0 mount missing") |
| 882 | + |
| 883 | +# Check fstab |
| 884 | +fstab = unit.run("cat /etc/fstab")[0].split("\n") |
| 885 | +fstabOk = False |
| 886 | +for line in fstab: |
| 887 | + if "/dev/mapper/loop0" in line and "/srv/data" in line: |
| 888 | + fstabOk = True |
| 889 | +if not fstabOk: |
| 890 | + amulet.raise_status(amulet.FAIL, msg="/dev/mapper/loop0 fstab entry missing") |
| 891 | + |
| 892 | +# Check crypt status |
| 893 | +cryptstatus = unit.run("cryptsetup status /dev/mapper/loop0")[0].split("\n") |
| 894 | +isActive = False |
| 895 | +isLuks = False |
| 896 | +for line in cryptstatus: |
| 897 | + if "/dev/mapper/loop0 is active and is in use" in line: |
| 898 | + isActive = True |
| 899 | + if "type:" in line and "LUKS1" in line: |
| 900 | + isLuks = True |
| 901 | +if not (isActive and isLuks): |
| 902 | + amulet.raise_status(amulet.FAIL, msg="Crypt status incorrect") |
| 903 | + |
| 904 | +# Make sure keyfile exists |
| 905 | +if not "keyfile--dev-loop0" in unit.directory_contents("/root")["files"]: |
| 906 | + amulet.raise_status(amulet.FAIL, msg="Keyfile not found") |
| 907 | + |
| 908 | +# And has the correct data |
| 909 | +if unit.file_contents("/root/keyfile--dev-loop0") != "password1": |
| 910 | + amulet.raise_status(amulet.FAIL, msg="Keyfile incorrect") |
| 911 | |
| 912 | === added file 'tests/20-crypt-lose-keyfile' |
| 913 | --- tests/20-crypt-lose-keyfile 1970-01-01 00:00:00 +0000 |
| 914 | +++ tests/20-crypt-lose-keyfile 2014-11-07 12:56:29 +0000 |
| 915 | @@ -0,0 +1,73 @@ |
| 916 | +#!/usr/bin/python3 |
| 917 | +# |
| 918 | +# Tests the partition provider with disk encryption |
| 919 | +# Almost identical to 10-crypt-keep-keyfile only we want to |
| 920 | +# make sure everything works AND the keyfile is deleted |
| 921 | + |
| 922 | +import amulet |
| 923 | +import time |
| 924 | + |
| 925 | +d = amulet.Deployment(series="trusty") |
| 926 | + |
| 927 | +# Add units |
| 928 | +d.add('postgresql', 'postgresql') |
| 929 | +d.add('storage', 'storage') |
| 930 | + |
| 931 | +# Set the unit configs |
| 932 | +d.configure("storage", {"provider": "partition", "volume_map": "{postgresql/0: /dev/loop0}", "encryption_key_map": "{postgresql/0: password1}", "store_encryption_keys": "no"}) |
| 933 | + |
| 934 | +try: |
| 935 | + d.setup(timeout=900) |
| 936 | + d.sentry.wait() |
| 937 | +except amulet.helpers.TimeoutError: |
| 938 | + amulet.raise_status(amulet.FAIL, msg="Environment wasn't stood up in time") |
| 939 | +except: |
| 940 | + raise |
| 941 | + |
| 942 | +unit = d.sentry.unit['postgresql/0'] |
| 943 | + |
| 944 | +# Create a test device |
| 945 | +if unit.run("dd if=/dev/zero of=/testfile bs=1024 count=10240 2>&1")[1] != 0: |
| 946 | + amulet.raise_status(amulet.FAIL, msg="dd failed creating /testfile") |
| 947 | +if unit.run("losetup /dev/loop0 /testfile")[1] != 0: |
| 948 | + amulet.raise_status(amulet.FAIL, msg="losetup failed creating /dev/loop0") |
| 949 | +time.sleep(10) |
| 950 | + |
| 951 | +# Add relations |
| 952 | +d.relate('postgresql:data', 'storage:data') |
| 953 | +d.sentry.wait() |
| 954 | +time.sleep(30) |
| 955 | + |
| 956 | +# Check mounts |
| 957 | +mounts = unit.run("cat /proc/mounts")[0].split("\n") |
| 958 | +mountsOk = False |
| 959 | +for line in mounts: |
| 960 | + if "/dev/mapper/loop0" in line and "/srv/data" in line: |
| 961 | + mountsOk = True |
| 962 | +if not mountsOk: |
| 963 | + amulet.raise_status(amulet.FAIL, msg="/dev/mapper/loop0 mount missing") |
| 964 | + |
| 965 | +# Check fstab |
| 966 | +fstab = unit.run("cat /etc/fstab")[0].split("\n") |
| 967 | +fstabOk = False |
| 968 | +for line in fstab: |
| 969 | + if "/dev/mapper/loop0" in line and "/srv/data" in line: |
| 970 | + fstabOk = True |
| 971 | +if not fstabOk: |
| 972 | + amulet.raise_status(amulet.FAIL, msg="/dev/mapper/loop0 fstab entry missing") |
| 973 | + |
| 974 | +# Check crypt status |
| 975 | +cryptstatus = unit.run("cryptsetup status /dev/mapper/loop0")[0].split("\n") |
| 976 | +isActive = False |
| 977 | +isLuks = False |
| 978 | +for line in cryptstatus: |
| 979 | + if "/dev/mapper/loop0 is active and is in use" in line: |
| 980 | + isActive = True |
| 981 | + if "type:" in line and "LUKS1" in line: |
| 982 | + isLuks = True |
| 983 | +if not (isActive and isLuks): |
| 984 | + amulet.raise_status(amulet.FAIL, msg="Crypt status incorrect") |
| 985 | + |
| 986 | +# Make sure keyfile does not exist |
| 987 | +if "keyfile--dev-loop0" in unit.directory_contents("/root")["files"]: |
| 988 | + amulet.raise_status(amulet.FAIL, msg="Keyfile found when not wanted") |
Hey Chris, nice addition to the storage charm, thanks!
My only quibble with this proposal is that there are no new tests to cover the new functionality. Before giving this a +1 I would like to see some tests to cover the new encryption stuff - maybe some unit test coverage of the crypt_fs module, and a new amulet test to test a deployment scenario.