Support privateIpAddress and ipAddress

Evaluating for SRU to precise since this is a huge missing piece of the EC2 API

On initial investigation of the given patch, it seemed that this problem would be fairly easy to solve with a cherry picked patch.

But, this is not so. ipAddress and privateIpAddress were not present in the default version of the EC2 API that txaws supported in the release shipped in precise (EC2 API version 2008-12-01).

Passing an EC2 query object which setts the API version at 2009-11-30, when the attributes were added, does work, but then causes failures in other areas of txaws which had to be updated for the newer WSDL, such as changing 'shutdownState' to 'currentState' in terminate_instances.

This is a red flag for me, and I think this has to be either be changed to a micro release update with a ton of regression testing, or the new version must be pushed to precise-backports, and anything (like juju) that wants to take advantage of the new attributes needs to be careful to only use them if they are available.

Marking the Precise SRU as Won't Fix, and opening a task against precise-backports, as that is the more conservative approach. We may decide to go forward with a micro release update if we can come up with a good testing plan, but it doesn't seem likely.

Also attaching some test code I wrote to show where ip_address will or will not appear as a valid attribute depending on the version of txaws. If the patch is cherry picked, the ipaddress shows up, but terminateInstances raises an error because of the missing shutdownState.

Attaching successful no-change source rebuild on precise of quantal version.

This is the output of the test program. Note that the backtrace about NoneType is normal (coding error due to my ignorance of how to do things perfectly in twisted).

$ python test-txaws-ipaddress.py ami-9666e9a6
i-e5269ad6
Mon Aug 6 16:32:33 2012: polling ec2...
Mon Aug 6 16:32:33 2012: polling ec2...
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/protocols/policies.py", line 123, in connectionLost
    self.wrappedProtocol.connectionLost(reason)
  File "/usr/lib/python2.7/dist-packages/twisted/web/client.py", line 191, in connectionLost
    self.factory._disconnectedDeferred.callback(None)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 362, in callback
    self._startRunCallbacks(result)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 458, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 545, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "test-txaws-ipaddress.py", line 69, in describeInstances
    d = ec2.describe_instances(instance_id)
  File "/usr/lib/python2.7/dist-packages/txaws/ec2/client.py", line 45, in describe_instances
    d = query.submit()
  File "/usr/lib/python2.7/dist-packages/txaws/ec2/client.py", line 976, in submit
    self.sign()
  File "/usr/lib/python2.7/dist-packages/txaws/ec2/client.py", line 969, in sign
    self.params["Signature"] = self.signature.compute()
  File "/usr/lib/python2.7/dist-packages/txaws/ec2/client.py", line 1022, in compute
    bytes = self.signing_text()
  File "/usr/lib/python2.7/dist-packages/txaws/ec2/client.py", line 1041, in signing_text
    self.get_canonical_query_params())
  File "/usr/lib/python2.7/dist-packages/txaws/ec2/client.py", line 1048, in get_canonical_query_params
    result.append("%s=%s" % (self.encode(key), self.encode(value)))
  File "/usr/lib/python2.7/dist-packages/txaws/ec2/client.py", line 1059, in encode
    return quote(string, safe="~")
  File "/usr/lib/python2.7/urllib.py", line 1255, in quote
    raise TypeError('None object cannot be quoted')
exceptions.TypeError: None object cannot be quoted
Mon Aug 6 16:32:43 2012: polling ec2...
Mon Aug 6 16:32:53 2012: polling ec2...
Mon Aug 6 16:33:04 2012: polling ec2...
Mon Aug 6 16:33:15 2012: polling ec2...
dns_name = ec2-54-245-9-232.us-west-2.compute.amazonaws.com
ip_address = 54.245.9.232
private_dns_name = ip-10-248-27-120.us-west-2.compute.internal
private_ip_address = 10.248.27.120
[('i-e5269ad6', 'running', 'shutting-down')]

When run on precise w/o this backport, ip_address will = None

Testing juju (only rdep of python-txaws) with backported txaws, works perfectly:

Setting up juju (0.5+bzr531-0ubuntu1.2) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
(precise-amd64)root@clint-MacBookPro:/home/clint/pkg/txaws/backport# su - clint
(precise-amd64)clint@clint-MacBookPro:~$ dpkg -l python-txaws
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Description
+++-=======================-=======================-==============================================================
ii python-txaws 0.2.3-1ubuntu1 Twisted-based Asynchronous Libraries for Amazon Web Services
(precise-amd64)clint@clint-MacBookPro:~$ juju status -e precise-branch
Invalid environment 'precise-branch'
2012-08-06 16:37:31,081 ERROR Invalid environment 'precise-branch'
(precise-amd64)clint@clint-MacBookPro:~$ juju status -e precise-fix
2012-08-06 16:37:34,610 INFO Connecting to environment...
Enter passphrase for key '/home/clint/.ssh/id_rsa':
2012-08-06 16:37:43,170 INFO Connected to environment.
machines:
  0:
    agent-state: running
    dns-name: ec2-54-245-13-4.us-west-2.compute.amazonaws.com
    instance-id: i-236cd010
    instance-state: running
  1:
    agent-state: running
    dns-name: ec2-54-245-11-162.us-west-2.compute.amazonaws.com
    instance-id: i-eb6dd1d8
    instance-state: running
  2:
    agent-state: running
    dns-name: ec2-50-112-233-46.us-west-2.compute.amazonaws.com
    instance-id: i-9b6ed2a8
    instance-state: running
services:
  mysql:
    charm: cs:precise/mysql-2
    relations: {}
    units:
      mysql/0:
        agent-state: started
        machine: 1
        public-address: ec2-54-245-11-162.us-west-2.compute.amazonaws.com
  wordpress:
    charm: cs:precise/wordpress-3
    relations: {}
    units:
      wordpress/0:
        agent-state: started
        machine: 2
        public-address: ec2-50-112-233-46.us-west-2.compute.amazonaws.com
2012-08-06 16:37:45,727 INFO 'status' command finished successfully
(precise-amd64)clint@clint-MacBookPro:~$ juju destroy-environment -e precise-fix
WARNING: this command will destroy the 'precise-fix' environment (type: ec2).
This includes all machines, services, data, and other resources. Continue [y/N]y
2012-08-06 16:38:02,946 INFO Destroying environment 'precise-fix' (type: ec2)...
2012-08-06 16:38:05,837 INFO Waiting on 3 EC2 instances to transition to terminated state, this may take a while
2012-08-06 16:38:43,354 INFO 'destroy_environment' command finished successfully
(precise-amd64)clint@clint-MacBookPro:~$

Also tested backported aws-status, which is an indicator built on top of txaws. Works fine in precise.

build log of juju from precise building with backported txaws

This looks good, uploading based on feedback from SpamapS on IRC.

This has been accepted into precise-backports.