LAVA Deployment Tool 2012.02

2012-02-21 Paul Larson <email address hidden>

 release 0.2.1

2012-02-16 Paul Larson <email address hidden>

 only stop lava-instance-* when the parent lava-instance _of that instance_ is stopped, not any lava-instance at all

2012-02-17 Michael Hudson-Doyle <email address hidden>

 only stop lava-instance-* when the parent lava-instance _of that instance_ is stopped, not any lava-instance at all

2012-02-14 Zygmunt Krynicki <email address hidden>

 Backup the instance /etc directory.

 This directory contains essential dispatcher configruation, customized
 uwsgi and lava-server settings file. If lost it could be very hard to
 recreate.

2012-02-12 Zygmunt Krynicki <email address hidden>

 Backup now includes the /etc directory

2012-02-14 Zygmunt Krynicki <email address hidden>

 Make setup step install python-pip

 This was mostly an oversight. Pip is required and is not installe by default.

2012-02-10 Fathi Boudra <email address hidden>

 Add python-pip dependency.

 The package is installed in a default Ubuntu installation.
 However, if the user disable Recommends, it isn't pulled.

2012-02-06 Michael Hudson-Doyle <email address hidden>

 add mercurial to LAVA_PKG_LIST to work around https://github.com/pypa/pip/issues/450

2012-02-01 Michael Hudson-Doyle <email address hidden>

 another attempt at making tar behave when archiving data that moves underneath it during backup

2012-01-31 Michael Hudson-Doyle <email address hidden>

 allow tar to exit with code 1 while backing up media files

2012-01-19 Zygmunt Krynicki <email address hidden>

 Reorganize requirements files and add 2012.01 data

2012-01-19 Zygmunt Krynicki <email address hidden>

 Add requirements file for 2012.01 release

 This file was obtained by building a bundle out of
 requirements/requirements-latest.txt, creating a virtualenv --no-site-packages
 and pip installing the bundle. The output of 'pip freeze' was redirected to
 this file.

2012-01-19 Zygmunt Krynicki <email address hidden>

 Add requirements-latest.txt

 This file always selects the latest set of requirements from pypi. It does not
 use any bleeding edge (from vcs) packages nor does it bind any particular
 version. It is suitable for making monthly releases.

2012-01-19 Zygmunt Krynicki <email address hidden>

 Move requirements to dedicated directory

 With this change we can create a detailed requirements file for each release
 and keep the root directory a bit more tidy.

2012-01-19 Paul Larson <email address hidden>

 change version to 0.2

2012-01-19 Zygmunt Krynicki <email address hidden>

 Add a version command

2012-01-18 Paul Larson <email address hidden>

 Improve usability of the restore command - from Zygmunt

2012-01-13 Zygmunt Krynicki <email address hidden>

 Use test -e to check for backup existence

 Previously backups had to be directories. This broke when we started symlinking
 the final snapshot as 'latest' for convenience. If one attempts to recover
 using the latest snapshot a confusing 'not found' error message would be
 displayed.

2012-01-13 Zygmunt Krynicki <email address hidden>

 More informative error message in cmd_restore

 When we cannot find the desired snapshot the error message
 left the user with no clues as to where the file should be.
 Now we display both the primary and alternate pathnames
 that were referenced.

2012-01-13 Spring Zhang <email address hidden>

 Update the doc with right reload instruction

2012-01-13 Spring Zhang <email address hidden>

 update webapp develop doc

2012-01-12 Zygmunt Krynicki <email address hidden>

 Merge lp:~zkrynicki/lava-deployment-tool/fix-915495

2012-01-12 Zygmunt Krynicki <email address hidden>

 Call _load_configuration before accessing $LAVA_SYS_USER

 This fixes cmd_restore's final stages where it tried to restore ownership
 of the media files to the instance's owner.

2012-01-12 Zygmunt Krynicki <email address hidden>

 Merge lp:~zkrynicki/lava-deployment-tool/fix-915387

2012-01-12 Zygmunt Krynicki <email address hidden>

 Compress media file tarballs

 We seem to have lost the --gzip argument that was compressing our
 file backups.

2012-01-12 Spring Zhang <email address hidden>

 Add doc how to setup web app development environment

2012-01-12 Spring Zhang <email address hidden>

 Add doc how to setup web app development environment

2012-01-12 Zygmunt Krynicki <email address hidden>

 Merged lp:~zkrynicki/lava-deployment-tool/fix-914748

2012-01-11 Zygmunt Krynicki <email address hidden>

 Download uWSGI tarball if needed

 If we need to build uwsgi module for apache we should not crash
 on missing uWSGI source tarball. This may happen when one is
 installing a bundle on a fresh system

2012-01-12 Zygmunt Krynicki <email address hidden>

 Merge lp:~zkrynicki/lava-deployment-tool/fix-914842

2012-01-11 Zygmunt Krynicki <email address hidden>

 Respect http_proxy settings when invoking pip

 It seems that pip itself does not respect http_proxy
 when downloading things. There is a dedicated parameter
 that we now pass whenever http_proxy is set in the
 environment.

2012-01-12 Zygmunt Krynicki <email address hidden>

 Merge lp:~zkrynicki/lava-deployment-tool/fix-914597

2012-01-11 Zygmunt Krynicki <email address hidden>

 Do not stop if removing rabbitmq user/vhost fails

2012-01-11 Zygmunt Krynicki <email address hidden>

 Merge lp:~zkrynicki/lava-deployment-tool/fix-907255

2012-01-11 Zygmunt Krynicki <email address hidden>

 Abort if running as root

2012-01-11 Zygmunt Krynicki <email address hidden>

 Merge lp:~mwhudson/lava-deployment-tool/backup-tweaks

2012-01-11 Michael Hudson-Doyle <email address hidden>

 add -T to the ln invocation

2012-01-11 Michael Hudson-Doyle <email address hidden>

 move set -x a touch later

2012-01-11 Michael Hudson-Doyle <email address hidden>

 make rsyncable backups; create a latest symlink to the created backup

2012-01-11 Michael Hudson-Doyle <email address hidden>

 tangent: do not set -x until a bit later in restore

2012-01-11 Zygmunt Krynicki <email address hidden>

 Merge lp:~mwhudson/lava-deployment-tool/no-scheduler-switch

2012-01-09 Michael Hudson-Doyle <email address hidden>

 delete some bogus comments

2012-01-09 Michael Hudson-Doyle <email address hidden>

 one more detail

2012-01-09 Michael Hudson-Doyle <email address hidden>

 add a comment to the scheduler upstart file

2012-01-09 Michael Hudson-Doyle <email address hidden>

 as before

2012-01-09 Michael Hudson-Doyle <email address hidden>

 bash is always harder than it should be

2012-01-09 Michael Hudson-Doyle <email address hidden>

 * add LAVA_SCHEDULER_ENABLED to the settings
 * ask about it (using whiptail) in wizard_config_app
 * add it if not already present during upgrade (in install_config_app)
 * respect it in the upstart file

2012-01-09 Michael Hudson-Doyle <email address hidden>

 have all the instance upstart files silently do nothing if the instance config is not present

2012-01-11 Michael Hudson-Doyle <email address hidden>

 fix chgrp invocation in restore

2012-01-09 Zygmunt Krynicki <email address hidden>

 Properly display RabbitMQ conf in cmd_destroy()

2012-01-09 Spring Zhang <email address hidden>

 fix typo error 'provide'

2012-01-07 Zygmunt Krynicki <email address hidden>

 Merge documentation improvements

2012-01-06 Paul Larson <email address hidden>

 Add a quickstart section to the readme

2012-01-06 Paul Larson <email address hidden>

 remove '.sh' from lava-deployment-tool commands in README

2012-01-06 Paul Larson <email address hidden>

 Modify README to mention 'bundle'

2012-01-06 Paul Larson <email address hidden>

 Add help entry for bundle command

2012-01-05 Zygmunt Krynicki <email address hidden>

 Merge bundle installation support

2012-01-04 Zygmunt Krynicki <email address hidden>

 Do not pass --upgrade when installing pybundle

 It seems that --upgrade goes out to the web to check for newer versions of each
 component (something we explicitly wanted to avoid) so let's not use it

2012-01-04 Zygmunt Krynicki <email address hidden>

 Make cmd_upgrade use bundles

 This mimicks previous changes to cmd_install

2012-01-04 Zygmunt Krynicki <email address hidden>

 Simplify bundled requirements.

 Don't ship celery yet (it needs to be released to work with stock pip) and
 don't use trunk in the mini install.

2012-01-04 Zygmunt Krynicki <email address hidden>

 Make cmd_install use bundles instead of requirements file

 This rips all the manaul installation code from each install_ step
 and changes that to one big "installation" in the install_app stage
 where we simply install the bundle we already have from somewhere
 else.

 This changes the assumptions of some of the previous commands. Most
 notably is that requirements file is only used by cmd_bundle and
 everything else just does not care.

2012-01-04 Zygmunt Krynicki <email address hidden>

 Remove instance filesystem as a last step

 This way if something fails mid-run the user can still see log files
 and check for passwords (lava instance configuration file will be intact)

2012-01-04 Zygmunt Krynicki <email address hidden>

 Fix removal of rabbitmq user

2012-01-04 Zygmunt Krynicki <email address hidden>

 Add cmd_bundle and the bundle command

 This command allows one to create standalone 'bundles' of all the code
 required to get lava instance up and running.

2012-01-04 Zygmunt Krynicki <email address hidden>

 Include a verbatim copy of the referenced requirements file

 For some reason using -r did not work for bundle (it seems that pip was changing
 directory to /tmp to do the work)

2012-01-04 Zygmunt Krynicki <email address hidden>

 Download my fork of pip in pkglist_venv

 This will allow each bundled instance to get our bug fixes automatically

2012-01-04 Zygmunt Krynicki <email address hidden>

 Drop the .sh suffix

 The script requires bash and may move to python eventually. Let's not change
 the suffix if we can (later)

2012-01-04 Zygmunt Krynicki <email address hidden>

 Add pkglist_* for each step

 Package list functions generate the expected set of pip packages
 that should be installed in the instance. This is a step towards being
 able to bundle releases as single files to avoid download issues

2012-01-04 Zygmunt Krynicki <email address hidden>

 Add support for setting up RabbitMQ user/vhost

 Previously rabbitmq was not really configured for each instance. Everyone
 used the implicit guest account on the root vhost. With this patch each
 instance gets a separate vhost and user/password. This will prevent two
 instances from seeing each other's messages (that map to celery tasks)

2012-01-03 Zygmunt Krynicki <email address hidden>

 Include additional command in basic help.

 Previously it seemed that lava-deployment-tool can only do 'install' and
 'setup' that confusingly seem alike. Now the default help output includes more
 of the basic set of commands as well as improved, more clear descriptions.

2012-01-03 Zygmunt Krynicki <email address hidden>

 Add manage command

 This command directly exposes lava-server manage for a particular instance. It
 allows administrators and developers to trouble-shoot and do "low level"
 operations easier than before.

2012-01-03 Zygmunt Krynicki <email address hidden>

 Rename default instance of cmd_backup to 'dev'

 This change is consistent with other commands that default to dev.
 Using 'lava' is confusing as instances then create users like 'lava-lava'

2012-01-03 Zygmunt Krynicki <email address hidden>

 Add developer mode.

 In developer mode the wsgi file will not use django-seatbelt and will not
 constrain how the developer sets up the virtual environment physical layout in
 the filesystem.

 In the default production mode, seatbelt mandates that all python modules
 are in their 'expected' location (that is, setup.py develop in someone's
 home folder will not work).

2012-01-03 Zygmunt Krynicki <email address hidden>

 Add defaults stage

 Defaults compute the initial automatic values of configuration variables.
 Adding this step simplifies wizard steps and paves the way to adding config
 dump/show later. It will also eventually make it possible to put each action
 (such as 'deploy a database') in a dedicated file thus making it possible to
 get such 'deployment parts' from each extension.

2011-12-15 Zygmunt Krynicki <email address hidden>

 Merge improvements from my git branch, meh, I hate merge messages

2011-12-14 Zygmunt Krynicki <email address hidden>

 Merge with bzr trunk

2011-12-14 Zygmunt Krynicki <email address hidden>

 Merge branch 'master' of github.com:zyga/lava-deployment-tool

2011-12-14 Zygmunt Krynicki <email address hidden>

 Disable top-level logging

 Logging via tee interferes with pip asking interactive questions
 about editable packages from vcs branches

2011-12-14 Zygmunt Krynicki <email address hidden>

 Use || die in more places

2011-12-14 Zygmunt Krynicki <email address hidden>

 More strategic usage of set -x/set +x

2011-12-14 Zygmunt Krynicki <email address hidden>

 Describe actions as they happen in more detail

2011-12-12 Fathi Boudra <email address hidden>

 Update software requirements section: we support Ubuntu from 10.10 (Maverick).

2011-12-10 Fathi Boudra <email address hidden>

 Fix README: the script has .sh extension.

2011-12-09 Michael Hudson-Doyle <email address hidden>

 merge changes from git:
 * Fix settings.conf typo for non-trunk deployments
 * Install uwsgi with explicit tarball URL
 * fix typo in generated settings.conf

2011-12-14 Fathi Boudra <email address hidden>

 Fix typo: deploy->deployment.

2011-12-14 Fathi Boudra <email address hidden>

 The script contains bashism. Zygmunt prefers to force bash instead of fixing the bashisms.

2011-12-12 Fathi Boudra <email address hidden>

 Update software requirements section: we support Ubuntu from 10.10 (Maverick).

2011-12-10 Fathi Boudra <email address hidden>

 Fix README: the script has .sh extension.

2011-12-09 Michael Hudson-Doyle <email address hidden>

 merge changes from git:
 * Fix settings.conf typo for non-trunk deployments
 * Install uwsgi with explicit tarball URL
 * fix typo in generated settings.conf

2011-12-07 Zygmunt Krynicki <email address hidden>

 Fix settings.conf typo for non-trunk deployments

 settings.conf is generated with a bit of conditional logic that
 does different stuff depending on using -trunk.txt requirements
 file. This patch fixes the other if branch

2011-12-07 Zygmunt Krynicki <email address hidden>

 Install uwsgi with explicit tarball URL

 uWSGI releases on pypi point to uwsgi-latest.tar.gz which is a nice way to get latest uwsgi
 if you want to get it quickly. Unfortunately if you want a precise version you need to
 install it from tarball directly. Since uwsgi is not a python project really pip does not
 know anything about it after installation. The only pip parts are integrationg sufficient
 to build and install uwsgi.

2011-12-07 Michael Hudson-Doyle <email address hidden>

 fix typo in generated settings.conf

2011-12-07 Michael Hudson-Doyle <email address hidden>

 fix typo in generated settings.conf

2011-12-05 Zygmunt Krynicki <email address hidden>

 Merge pull request #1 from mwhudson/master

 Improve backup and fix typos

2011-12-06 Michael Hudson-Doyle <email address hidden>

 spell option correctly

2011-12-06 Michael Hudson-Doyle <email address hidden>

 add a todo

2011-12-06 Michael Hudson-Doyle <email address hidden>

 don't spray password into the terminal

2011-12-06 Michael Hudson-Doyle <email address hidden>

 --ignore-failed-reads might work around the "tar: ./media/bundles: file changed
 as we read it" problem

2011-12-06 Michael Hudson-Doyle <email address hidden>

 do not get confirmation before taking a backup

2011-12-06 Michael Hudson-Doyle <email address hidden>

 another way to typo 'deployment'

2011-12-06 Zygmunt Krynicki <email address hidden>

 Require new kernel-ci-views

 The previous version had missing versiontools dependency
 and failed to install in a pristine virtualenv

2011-12-06 Zygmunt Krynicki <email address hidden>

 Add a few || die lines

2011-12-06 Zygmunt Krynicki <email address hidden>

 Update settings.conf

 Explicitly generate mount point and login URLs

2011-12-06 Zygmunt Krynicki <email address hidden>

 Add requirement file for 2011 Dec 6 release

2011-12-05 Zygmunt Krynicki <email address hidden>

 Pass instance data to scheduler

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add a note about possible upstart socket activation

2011-12-05 Zygmunt Krynicki <email address hidden>

 Bump upstart files version

2011-12-05 Zygmunt Krynicki <email address hidden>

 Fix scheduler upstart config

 Don't use sudo as it breaks server reboots (pam session)
 Don't use virtualenv, emulate the side effects without spawing
 another shell (correct pid tracking)

2011-12-05 Zygmunt Krynicki <email address hidden>

 Start instances based on instance file presence

2011-12-05 Zygmunt Krynicki <email address hidden>

 Don't start uwsgi if the instance file is not there

2011-12-05 Zygmunt Krynicki <email address hidden>

 Disable celery startup

 Due to issues with tracking the right pid and changing the user at the same time celery
 has to be disabled for the release

2011-12-05 Zygmunt Krynicki <email address hidden>

 Fix lava-instance-uwsgi upstart script

 Previous version of this script made upstart track something else. Now the combination of no
 additional shell processing, uwsgi running as root with workers running as $LAVA_SYS_USER
 (explicitly configured in uwsgi.ini) and exec .../bin/uwsgi makes upstart track the right
 process.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add a note about missing SSL configuration

2011-12-05 Zygmunt Krynicki <email address hidden>

 Update TODO file

2011-12-05 Zygmunt Krynicki <email address hidden>

 Load configuration for the hidden install_xxx commands

2011-12-05 Zygmunt Krynicki <email address hidden>

 Change how install, upgrade and remove commands are started

 This matches set -e, set -x management and default instance name
 from the cmd_xxx functions that implement those actions. In addition
 the whole log of the execution is stored in the current working directory.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Improve the remove command

 The remove command now loads instance configuration to know what to remove.
 It warns the user about the steps being performed in greater detail than before.
 It also uses set -e and -x around the sensitive part to keep the user interface part readable.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Change install and upgrade commands

 The default instance name is now 'dev' using the 'requirements-trunk.txt' requirements
 file. Install now uses the explicit configuration and installations steps. Set -x and -e are enabled
 only for the installation stage to keep configuration readable. Upgrade command uses
 _load_configuration to read the instance.conf file left by installer. Both commands display
 banner. Installation command first checks if the required instance exists before attempting
 to download a remote requirements file.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add a _banner function

2011-12-05 Zygmunt Krynicki <email address hidden>

 Keep track of all the install steps

 LAVA_INSTALL_STEPS encodes the pairs of wizard_ and install_ functions that
 need to be called to perform configuration and cleanup

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add wizard_config_app and change install_config_app

 Use the new calling interface

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add wizard_app function and change install_app

 New calling interface. It is worth pointing out that LAVA_REQUIREMENT
 is handled exactly the same as LAVA_INSTANCE (it is provided by the environment
 not by a function argument)

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add wizard_web_hosting and update install_web_hosting

 Install function now expects the new calling interface. It also cooperates with wizard
 to get the apache virtual host name. The default is the same as before, the current host name.

 The wizard needs more work, it should be configurable and should describe what is going on.
 There should be an option not to disable the 000-default site.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add wizard_datqabase and update install_database

 There are a few changes. New calling interface, explicit configuration variables for
 database name $LAVA_DB_NAME, database-level user $LAVA_DB_USER
 and password $LAVA_DB_PASSWORD. The wizard function is actually useful
 as it generates the password.

 In the future this would be the correct place to add additional options
 such as altering any of the three settings or perhaps restoring a backup.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add ignore file

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add wizard_venv and change install_venv

 Adapt to new calling interface. Describe basics of system isolation
 using virtualenv, how to enable and disable the environment for
 administration.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add wizard_fs and change install_fs

 Adapt install_fs to the new calling interface. The wizard_fs function
 is a member of family of wizard functions that don't have any
 real settings. Instead the point is to educate the user about
 how various parts work and what can be expected after the installation.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Make install_fs use LAVA_SYS_USER

 The user name is now stored in a dedicated variable
 and can be different from LAVA_INSTANCE

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add wizard_user and change install_user

 The user stage is now configurable. It also requires on a different
 calling interface (the way cmd_install is implemented) which is not
 a part of this patch. The new configuration does not need any arguments
 to the function itself but rather depends on the collection of environment
 variables.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add _configure and _install functions

 The purpose of _configure is to garther the entire configuration without actually
 committing anything. The point of _install is to install everything without
 asking any questions.

 Both use an extensible list of steps to get configuration (wizard_xxx) and to
 use it to do something install_xxx

2011-12-05 Zygmunt Krynicki <email address hidden>

 A few extra newlines

2011-12-05 Zygmunt Krynicki <email address hidden>

 Store installation logs

2011-12-05 Zygmunt Krynicki <email address hidden>

 Reorganize set -e and set-x

 This is a cleanup attempt after discovering how set -e works a few patches ago.
 There is no explicit control inside any of the worker functions. Instead all control
 is done in the main function just before we call the workers.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add load_meta function

 This function looks at the meta file created by install_meta and loads the settings back
 to environment variables. It will be used by other parts of the script, most importantly upgrade
 and remove

2011-12-05 Zygmunt Krynicki <email address hidden>

 Add install_meta function

 This function records the essential environment variables of a particular lava instance.
 It will allow us to do non-shotgun removal later.

2011-12-05 Zygmunt Krynicki <email address hidden>

 Use explicit instance template and instance name

 Previously there was a silent contract between lava-server and lava-deployment-tool. This
 patch makes it possible to deploy (but not run, that requires additional patches) an instance
 in a non-agreed location (or as in this case, where the lava-server does not yet have the right
 defaults)

2011-12-05 Zygmunt Krynicki <email address hidden>

 Merge set -e fixes from mwhudson

 set -e just does not work as we expected, for details see man bash,
 specifically the section that talks about set -e. In short it only applies to
 simple expressions and a mere usage of || makes the expression non-simple. This
 rule applies kind-of recursively so functions that are a part of || expression
 cannot observe any effects of set -e.

2011-12-02 Zygmunt Krynicki <email address hidden>

 Do not start instances automatically after installation

 This is a first step of many to get configurable startup possible

2011-12-02 Zygmunt Krynicki <email address hidden>

 Add minimal requirements file

 This only installs lava-server. It is faster for testing.
 It will probably not work very well in practice as other parts of
 the system currently assume that celery/schedler are installed as well

2011-12-02 Zygmunt Krynicki <email address hidden>

 Stop instances before restoring from backup

 Previously the restore operation would not work as hanging connections would prevent
 removal of the database

2011-12-02 Zygmunt Krynicki <email address hidden>

 Store backups in a dedicated directory outside of any instances

 This reduces the chance that a backup is wiped alongside the instance.

2011-12-02 Zygmunt Krynicki <email address hidden>

 Make PIP_DOWNLOAD_CACHE relative to LAVA_ROOT

 This way the download cache will not actually move from its original location

2011-12-02 Zygmunt Krynicki <email address hidden>

 Make LAVA_PREFIX depend on LAVA_ROOT

 The new value of LAVA_PREFIX will allow us to keep instances in a singe unambiguous subdirectory

2011-12-02 Zygmunt Krynicki <email address hidden>

 Introduce LAVA_ROOT

 This variable is where all lava stuff will originate. Unlike LAVA_PREFIX that is used to store instances
 it can be used for other pieces of data

2011-12-02 Zygmunt Krynicki <email address hidden>

 Create a directory for dispatcher configuration

2011-12-02 Zygmunt Krynicki <email address hidden>

 Mark Lucid as unsupported

2011-12-02 Michael Hudson-Doyle <email address hidden>

 Implement the lava-deployment-tool.sh 'restore' command, with bonus extras:
 * Change layout created by 'backup' command
 * Also tweak 'backup' so that it create restorable backups with pg 9.1
 (the change makes no difference to pg 8.4, which is what we use currently)
 * Also make some notes in README around which postgres version/cluster we use.

2011-12-02 Michael Hudson-Doyle <email address hidden>

 no plan survives contact etc

2011-12-02 Michael Hudson-Doyle <email address hidden>

 do not echo the db password to console

2011-12-02 Michael Hudson-Doyle <email address hidden>

 cope better with non-default db configs

2011-12-02 Michael Hudson-Doyle <email address hidden>

 some grammar nickpickery and pg details

2011-12-02 Michael Hudson-Doyle <email address hidden>

 another typo

2011-12-02 Michael Hudson-Doyle <email address hidden>

 note something

2011-12-02 Michael Hudson-Doyle <email address hidden>

 typo

2011-12-02 Michael Hudson-Doyle <email address hidden>

 another attempt at permissions

2011-12-02 Michael Hudson-Doyle <email address hidden>

 stupid mistakes

2011-12-02 Michael Hudson-Doyle <email address hidden>

 i think this is required to create restorable snapshots with 9.1 (although we use 8.4 on prod still, and it is harmless on prod)

2011-12-02 Michael Hudson-Doyle <email address hidden>

 moar (but not complete) working

2011-12-02 Michael Hudson-Doyle <email address hidden>

 fill out restore command, one XXX to go

2011-12-02 Michael Hudson-Doyle <email address hidden>

 skeleton of restore command

2011-12-02 Michael Hudson-Doyle <email address hidden>

 fix location of files backup too

2011-12-02 Michael Hudson-Doyle <email address hidden>

 change backup to work as README now says it does

2011-12-02 Michael Hudson-Doyle <email address hidden>

 ok this is how it should be

2011-11-29 Zygmunt Krynicki <email address hidden>

 Install lava.vcs by default

2011-11-29 Zygmunt Krynicki <email address hidden>

 Kill celeryd with SIGTERM

2011-11-29 Zygmunt Krynicki <email address hidden>

 Put sensible permissions on tmp

2011-11-29 Zygmunt Krynicki <email address hidden>

 Add a note about set -e bug

2011-11-29 Zygmunt Krynicki <email address hidden>

 Reduce the number of useless shells started by upstart

 We use script instead of exec to load additional configuration.
 Since the programs we start don't deamonize we can use exec to replace
 the otherwise useless shell that is used to start the programs we care for

2011-11-29 Zygmunt Krynicki <email address hidden>

 Drop trailing slash from DocumentRoot

2011-11-29 Zygmunt Krynicki <email address hidden>

 Use dispatcher trunk, don't use /next series

2011-11-28 Zygmunt Krynicki <email address hidden>

 Grr, fix paths again

2011-11-28 Zygmunt Krynicki <email address hidden>

 Fix uWSGI module build path

2011-11-28 Zygmunt Krynicki <email address hidden>

 Fix most outstanding bugs

 * Running as special user
 * Ownership and file access
 * Upstart files (due to owneship)
 * Logging
 * Apache vhosts
 * Installation of special pip version

2011-11-28 Zygmunt Krynicki <email address hidden>

 Backup is done, one less item in the TODO file

2011-11-28 Zygmunt Krynicki <email address hidden>

 Add backup command

 This command creates a dump of all the files and database contents. It does not
 stop the instance though.

2011-11-28 Zygmunt Krynicki <email address hidden>

 Add a note about lucid support and upstart

2011-11-23 Zygmunt Krynicki <email address hidden>

 Revert lucid upstart workaround

 It seems that using start-stop-daemon is not as easy as I hoped. Since it is
 not starting a new process that can be tracked by upstart then it brings
 nothing new to the table.

2011-11-23 Zygmunt Krynicki <email address hidden>

 Work around lack of 'kill signal' in old upstart

 uWSGI needs to be killed with SIGQUIT to shut down gracefully. There is no way
 to express this in Lucid's upstart so afer discussion with James Hunt (upstart
 developer) I've decided to use start-stop-daemon to run uwsgi.

2011-11-23 Zygmunt Krynicki <email address hidden>

 Install different packages based on OS

 This is currently required to install git-core on older Ubuntu versions.
 This package has been renamed to git later on.

2011-11-23 Zygmunt Krynicki <email address hidden>

 Add lucid support.

2011-11-23 Zygmunt Krynicki <email address hidden>

 Add a note about apache files

2011-11-23 Zygmunt Krynicki <email address hidden>

 Prevent installation on unsupported systems.

 This is just a temporary measure until I test deployment on additional systems
 automatically. Right now only Ubuntu Oneiric is marked as supported.

2011-11-23 Zygmunt Krynicki <email address hidden>

 Add os_check function

 This function looks at the currently running system to inspect if it is
 supported by LAVA and to alter any system-dependent installation options.

2011-11-23 Zygmunt Krynicki <email address hidden>

 Fall back to python2.6

 We will use python2.7 only on systems that support it

2011-11-23 Zygmunt Krynicki <email address hidden>

 Add dependency on rabbitmq-server

 This is the prerequisite for Celery and while celery might be deployed with a
 polling SQL backend it is not advisable for anything practical. In addition to
 that we intend to converge our development and production environments.

2011-11-23 Zygmunt Krynicki <email address hidden>

 Added TODO file

2011-11-22 Zygmunt Krynicki <email address hidden>

 Create uwsgi.reload file by default

2011-11-22 Zygmunt Krynicki <email address hidden>

 Start the scheduler automatically

2011-11-22 Zygmunt Krynicki <email address hidden>

 Keep each lava-celerybeat pid files separate

2011-11-22 Zygmunt Krynicki <email address hidden>

 Fix typo in requirements file

2011-11-22 Zygmunt Krynicki <email address hidden>

 Special case editable lava-server so that static files can be built

2011-11-22 Zygmunt Krynicki <email address hidden>

 Switch to python2.7 and install git in setup

2011-11-22 Zygmunt Krynicki <email address hidden>

 Fix variable name

2011-11-22 Zygmunt Krynicki <email address hidden>

 Add requirements for running trunk

2011-11-22 Zygmunt Krynicki <email address hidden>

 Fix instance restart + migrations

2011-11-22 Zygmunt Krynicki <email address hidden>

 Install a patched version of pip to support editable bzr branches.

 This is needed till https://github.com/pypa/pip/pull/337 gets merged and released upstream and adopted by virtualenv.

2011-11-22 Zygmunt Krynicki <email address hidden>

 Allow specifying instance and requirement file for install/upgrade/_remove

2011-11-22 Zygmunt Krynicki <email address hidden>

 Use sudo to run chgrp

2011-11-22 Zygmunt Krynicki <email address hidden>

 Update uwsgi config file

 Enable touch-to-reload (graceful worker restart) and sensible worker process names

2011-11-22 Zygmunt Krynicki <email address hidden>

 Be less verbose and more strict

2011-11-21 Zygmunt Krynicki <email address hidden>

 Update all upstart scripts

 Upstart script hierarchy is entirely rewritten. The rationale is given below:

 We want to have many copies of lava running on one machine. Each copy should be isolated from others as much as possible. All copies share upstart
 scripts.

 The topmost job, that starts with the system, is lava. This _abstract_ job represents all lava instances together. Starting or stopping lava ought to
 start all instances.

 The next job is lava-instances (note the trailing 's') which starts when lava is starting. This is a task job, that is, it runs a script and as soon as
 the script is done the whole job is also done. This job looks at $LAVA_PREFIX to see if there are any instances that ought to be started. For each
 instance a lava-instance job is started with appropriate instance name.

 The next job is lava-instance. This the first instance job (upstart wise). To start or stop it one needs to pass additional, explicit, environment
 variables that identify a particular instance. I've used LAVA_INSTANCE to tell various lava instances apart. This job is also abstract, like the lava
 job. This means that one can easily stop, start or restart any instance without affecting others. This job also exports the LAVA_INSTANCE value so that
 jobs that get started in consequence inherit the instance name.

 Next there is a collection of jobs that get started as long as a particular instance is started. Those control various specific aspects of LAVA.

 lava-instance-uwsgi is the workhorse that runs the whole web application (apache only serves static content and forwards whatever uwsgi worker says)

 lava-instance-celeryd is the celery task processor, a single celeryd can run multiple tasks (just like single uwsgi handles many requests).

 lava-instance-celerybeat is the celery task scheduler. It is basically responsible for scheduling periodic tasks in celeryd. This upstart job is
 somewhat special that in one lava cluster (a set of systems running one lava instance, sharing one database and message queue), there must be exactly
 one beat process running. In the future only celeryd should be started on "compute nodes"

 lava-instance-celerycam is the celery task snapshot service. It provides an unreliable, periodic view into what various task processors (compute nodes)
 did. It shares celerybeat characteristic in that one is enough. Unlike celerybeat though starting additional camera services merely degrades performance
 without affecting system correctness.

 lava-instance-scheduler is the lava-scheduler test job processor. It also requires lava-scheduler-monitor that is not (yet) ready.

 All of those upstart jobs are tied with their lifetime to the lifetime of the abstract lava-instance job.

2011-11-21 Zygmunt Krynicki <email address hidden>

 Make the media directory group-owned by www-data, make it group writable

2011-11-21 Zygmunt Krynicki <email address hidden>

 Work around pip not respecing --no-site-packages when invoked from outside the virtualenv

2011-11-17 Zygmunt Krynicki <email address hidden>

 Handle both upgrades and clean installs

2011-11-17 Zygmunt Krynicki <email address hidden>

 Restart uWSGI on upgrade

2011-11-17 Michael Hudson-Doyle <email address hidden>

 run the whole script with set -e

2011-11-17 Zygmunt Krynicki <email address hidden>

 Start the specific uwsgi instance

2011-11-17 Zygmunt Krynicki <email address hidden>

 Move socket to /run

2011-11-17 Zygmunt Krynicki <email address hidden>

 Make logging and socket ownership work out of the box

2011-11-17 Zygmunt Krynicki <email address hidden>

 Fix typo in upstart script

2011-11-17 Zygmunt Krynicki <email address hidden>

 Restart apache on install and start lava on install/upgrade

2011-11-17 Zygmunt Krynicki <email address hidden>

 Make setup install upstart jobs

2011-11-17 Zygmunt Krynicki <email address hidden>

 Make upgrade work

2011-11-17 Zygmunt Krynicki <email address hidden>

 Fix important typo

2011-11-17 Zygmunt Krynicki <email address hidden>

 More typos fixed

2011-11-17 Zygmunt Krynicki <email address hidden>

 Fix grammar

2011-11-17 Zygmunt Krynicki <email address hidden>

 Fix typo

2011-11-17 Zygmunt Krynicki <email address hidden>

 Fix install to create config scripts properly, add hidden _remove command, make setup create shared pip cache, be verbose in commands that are invoked in each step

2011-11-17 Zygmunt Krynicki <email address hidden>

 Die if something fails

2011-11-17 Zygmunt Krynicki <email address hidden>

 Move code around

2011-11-17 Zygmunt Krynicki <email address hidden>

 Add first attempt at saner UI, use PREFIX, chown /srv/lava/

2011-11-17 Zygmunt Krynicki <email address hidden>

 Add README, some parts are not implemented yet

2011-11-16 Michael Hudson-Doyle <email address hidden>

 don't fail if instance db does not exist when removing instance

2011-11-16 Michael Hudson-Doyle <email address hidden>

 oops

2011-11-16 Michael Hudson-Doyle <email address hidden>

 the global directory is not needed now

2011-11-16 Michael Hudson-Doyle <email address hidden>

 allow manipulating instance other than "foo"; serve at http://localhost/$LAVA_INSTANCE/

2011-11-16 Michael Hudson-Doyle <email address hidden>

 remove global_teardown stuff

2011-11-16 Michael Hudson-Doyle <email address hidden>

 disable log redirection

2011-11-16 Michael Hudson-Doyle <email address hidden>

 install django-debian before running lava-server manage

2011-11-16 Michael Hudson-Doyle <email address hidden>

 fix for starting with apache uninstalled

2011-11-16 Michael Hudson-Doyle <email address hidden>

 fixes for python != LAVA_PYTHON

2011-11-15 Zygmunt Krynicki <email address hidden>

 Here you go Michael