kuryr-kubernetes

LBaaS driver doesn't handler properly 'ensure_loadbalancer' for existing LB. Amphora fails to start with IP address already allocated error

Bug #1778516 reported by Daniel Mellado on 2018-06-25

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	kuryr-kubernetes	Fix Released	Critical	Yossi Boaron

Bug Description

All the services tests fail randomly due to Octavia failing to allocate a proper IP to the Amphora.

Jun 25 12:01:23.968818 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs [None req-8f1f35a8-2f85-4e0e-90c4-cad85a777d6c service kuryr] Error creating neutron port on network 308943da-3a00-4551-93ab-817d64af8e27.: IpAddressAlreadyAllocatedClient: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:23.970208 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-e11af8a6-e3ac-4cb3-8e4a-1813f61fcd51']
Jun 25 12:01:23.970351 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs Traceback (most recent call last):
Jun 25 12:01:23.970942 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/opt/stack/octavia/octavia/network/drivers/neutron/allowed_address_pairs.py", line 388, in allocate_vip
Jun 25 12:01:23.971726 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs new_port = self.neutron_client.create_port(port)
Jun 25 12:01:23.971871 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 794, in create_port
Jun 25 12:01:23.971988 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs return self.post(self.ports_path, body=body)
Jun 25 12:01:23.972114 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 359, in post
Jun 25 12:01:23.972231 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs headers=headers, params=params)
Jun 25 12:01:23.972732 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 294, in do_request
Jun 25 12:01:23.975158 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs self._handle_fault_response(status_code, replybody, resp)
Jun 25 12:01:23.975289 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response
Jun 25 12:01:23.975421 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs exception_handler_v20(status_code, error_body)
Jun 25 12:01:23.975582 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 93, in exception_handler_v20
Jun 25 12:01:23.975723 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs request_ids=request_ids)
Jun 25 12:01:23.975863 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs IpAddressAlreadyAllocatedClient: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:23.977340 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs Neutron server returns request_ids: ['req-e11af8a6-e3ac-4cb3-8e4a-1813f61fcd51']
Jun 25 12:01:23.977494 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs
Jun 25 12:01:23.977628 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils [None req-8f1f35a8-2f85-4e0e-90c4-cad85a777d6c service kuryr] Provider 'amphora' raised a driver error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:23.977742 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-e11af8a6-e3ac-4cb3-8e4a-1813f61fcd51']: DriverError
Jun 25 12:01:23.978295 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils Traceback (most recent call last):
Jun 25 12:01:23.978453 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils File "/opt/stack/octavia/octavia/api/drivers/utils.py", line 49, in call_provider
Jun 25 12:01:23.978675 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils return driver_method(*args, **kwargs)
Jun 25 12:01:23.978820 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils File "/opt/stack/octavia/octavia/api/drivers/amphora_driver/driver.py", line 56, in create_vip_port
Jun 25 12:01:23.978950 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils operator_fault_string=e.orig_msg)
Jun 25 12:01:23.979836 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils DriverError
Jun 25 12:01:23.979999 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils
Jun 25 12:01:23.982068 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR wsme.api [None req-8f1f35a8-2f85-4e0e-90c4-cad85a777d6c service kuryr] Server-side error: "Provider 'amphora' reports error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:23.982239 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-e11af8a6-e3ac-4cb3-8e4a-1813f61fcd51']". Detail:
Jun 25 12:01:23.983201 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Traceback (most recent call last):
Jun 25 12:01:23.985110 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/usr/local/lib/python2.7/dist-packages/wsmeext/pecan.py", line 85, in callfunction
Jun 25 12:01:23.985267 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: result = f(self, *args, **kwargs)
Jun 25 12:01:23.985792 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/opt/stack/octavia/octavia/api/v2/controllers/load_balancer.py", line 334, in post
Jun 25 12:01:23.985915 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: lock_session.rollback()
Jun 25 12:01:23.986370 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Jun 25 12:01:23.986549 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: self.force_reraise()
Jun 25 12:01:23.986743 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Jun 25 12:01:23.986910 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: six.reraise(self.type_, self.value, self.tb)
Jun 25 12:01:23.987047 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/opt/stack/octavia/octavia/api/v2/controllers/load_balancer.py", line 297, in post
Jun 25 12:01:23.987176 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: db_lb.project_id, provider_vip_dict)
Jun 25 12:01:23.987316 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/opt/stack/octavia/octavia/api/drivers/utils.py", line 54, in call_provider
Jun 25 12:01:23.987444 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: user_msg=e.user_fault_string)
Jun 25 12:01:23.988079 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ProviderDriverError: Provider 'amphora' reports error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:23.988474 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-e11af8a6-e3ac-4cb3-8e4a-1813f61fcd51']
Jun 25 12:01:23.988602 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: : ProviderDriverError: Provider 'amphora' reports error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:23.988761 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: [pid: 15078|app: 0|req: 48/96] 158.69.95.158 () {60 vars in 1203 bytes} [Mon Jun 25 12:01:22 2018] POST /load-balancer/v2.0/lbaas/loadbalancers => generated 263 bytes in 1063 msecs (HTTP/1.1 500) 4 headers in 176 bytes (1 switches on core 0)
Jun 25 12:01:53.396347 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs [None req-2c345d3b-1016-4630-952e-250000036b19 service kuryr] Error creating neutron port on network 308943da-3a00-4551-93ab-817d64af8e27.: IpAddressAlreadyAllocatedClient: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:53.399282 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-caa75dc2-d282-480d-9a8d-0d54efdcbe4e']
Jun 25 12:01:53.399517 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs Traceback (most recent call last):
Jun 25 12:01:53.400197 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/opt/stack/octavia/octavia/network/drivers/neutron/allowed_address_pairs.py", line 388, in allocate_vip
Jun 25 12:01:53.400421 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs new_port = self.neutron_client.create_port(port)
Jun 25 12:01:53.400642 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 794, in create_port
Jun 25 12:01:53.400853 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs return self.post(self.ports_path, body=body)
Jun 25 12:01:53.401083 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 359, in post
Jun 25 12:01:53.401300 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs headers=headers, params=params)
Jun 25 12:01:53.401527 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 294, in do_request
Jun 25 12:01:53.402289 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs self._handle_fault_response(status_code, replybody, resp)
Jun 25 12:01:53.402508 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response
Jun 25 12:01:53.402733 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs exception_handler_v20(status_code, error_body)
Jun 25 12:01:53.402938 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs File "/usr/local/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 93, in exception_handler_v20
Jun 25 12:01:53.403655 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs request_ids=request_ids)
Jun 25 12:01:53.403876 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs IpAddressAlreadyAllocatedClient: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:53.404701 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs Neutron server returns request_ids: ['req-caa75dc2-d282-480d-9a8d-0d54efdcbe4e']
Jun 25 12:01:53.404905 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.network.drivers.neutron.allowed_address_pairs
Jun 25 12:01:53.405132 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils [None req-2c345d3b-1016-4630-952e-250000036b19 service kuryr] Provider 'amphora' raised a driver error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:53.405950 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-caa75dc2-d282-480d-9a8d-0d54efdcbe4e']: DriverError
Jun 25 12:01:53.406171 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils Traceback (most recent call last):
Jun 25 12:01:53.406389 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils File "/opt/stack/octavia/octavia/api/drivers/utils.py", line 49, in call_provider
Jun 25 12:01:53.406615 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils return driver_method(*args, **kwargs)
Jun 25 12:01:53.406811 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils File "/opt/stack/octavia/octavia/api/drivers/amphora_driver/driver.py", line 56, in create_vip_port
Jun 25 12:01:53.407000 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils operator_fault_string=e.orig_msg)
Jun 25 12:01:53.407725 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils DriverError
Jun 25 12:01:53.407937 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR octavia.api.drivers.utils
Jun 25 12:01:53.409717 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ERROR wsme.api [None req-2c345d3b-1016-4630-952e-250000036b19 service kuryr] Server-side error: "Provider 'amphora' reports error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:53.409917 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-caa75dc2-d282-480d-9a8d-0d54efdcbe4e']". Detail:
Jun 25 12:01:53.410140 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Traceback (most recent call last):
Jun 25 12:01:53.411596 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/usr/local/lib/python2.7/dist-packages/wsmeext/pecan.py", line 85, in callfunction
Jun 25 12:01:53.411807 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: result = f(self, *args, **kwargs)
Jun 25 12:01:53.412015 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/opt/stack/octavia/octavia/api/v2/controllers/load_balancer.py", line 334, in post
Jun 25 12:01:53.412235 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: lock_session.rollback()
Jun 25 12:01:53.412446 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
Jun 25 12:01:53.412655 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: self.force_reraise()
Jun 25 12:01:53.412865 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
Jun 25 12:01:53.413120 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: six.reraise(self.type_, self.value, self.tb)
Jun 25 12:01:53.413347 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/opt/stack/octavia/octavia/api/v2/controllers/load_balancer.py", line 297, in post
Jun 25 12:01:53.413557 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: db_lb.project_id, provider_vip_dict)
Jun 25 12:01:53.413768 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: File "/opt/stack/octavia/octavia/api/drivers/utils.py", line 54, in call_provider
Jun 25 12:01:53.413995 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: user_msg=e.user_fault_string)
Jun 25 12:01:53.414208 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: ProviderDriverError: Provider 'amphora' reports error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:53.414426 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: Neutron server returns request_ids: ['req-caa75dc2-d282-480d-9a8d-0d54efdcbe4e']
Jun 25 12:01:53.414633 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: : ProviderDriverError: Provider 'amphora' reports error: IP address 10.1.0.144 already allocated in subnet 022292c2-a949-4163-b0bb-7b5484803a10
Jun 25 12:01:53.414840 ubuntu-xenial-ovh-bhs1-0000323659 <email address hidden>[15075]: [pid: 15077|app: 0|req: 49/97] 158.69.95.158 () {60 vars in 1203 bytes} [Mon Jun 25 12:01:52 2018] POST /load-balancer/v2.0/lbaas/loadbalancers => generated 263 bytes in 776 msecs (HTTP/1.1 500) 4 headers in 176 bytes (1 switches on core 0)

Daniel Mellado (daniel-mellado) on 2018-06-25

Changed in kuryr-kubernetes:
importance:	Undecided → Critical
assignee:	nobody → Antoni Segura Puimedon (celebdor)

Daniel Mellado (daniel-mellado) on 2018-06-25

Changed in kuryr-kubernetes:
assignee:	Antoni Segura Puimedon (celebdor) → Yossi Boaron (yossi-boaron-1234)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-27: Fix proposed to kuryr-kubernetes (master)

Fix proposed to branch: master
Review: https://review.openstack.org/578298

Changed in kuryr-kubernetes:
status:	New → In Progress

Revision history for this message

Yossi Boaron (yossi-boaron-1234) wrote on 2018-06-27: Re: Amphora fails to start with IP address already allocated error

Seems that the root cause for this specific case was kuryr-controller's pod being restarted periodically (cause of health check failure).
So, this due to this restarts :

1. services/LB created by Kuryr-controller weren't annotated properly.
2. When Kuryr restarted, since K8S service's annotation wasn't saved properly, LBaaS handler called LBaaS driver 'ensure_loadbalancer' function to re-create LB, and this function doesn't handle properly this case.
In most of the cases, LBaaS handler won't call LBaaS driver in that case.

So, in the bottom line :

A. The bug description should change to
LBaaS driver doesn't handler properly 'ensure_loadbalancer' for existing LB.

B. I don't think that bug's severity should be critical.

Revision history for this message

Daniel Mellado (daniel-mellado) wrote on 2018-06-27:

Hi Yossi,

Thanks for looking into it.

A. Agreed, added that to the description.
b. I marked that as critical as it's part of several issues blocking the gates ;)

summary:

- Amphora fails to start with IP address already allocated error
+ LBaaS driver doesn't handler properly 'ensure_loadbalancer' for existing
+ LB. Amphora fails to start with IP address already allocated error

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-27: Related fix proposed to kuryr-kubernetes (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/578344

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-27: Related fix merged to kuryr-kubernetes (master)

Reviewed: https://review.openstack.org/578344
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=a3e9f6a8694f566a96f3c4c6d4208b187e43aca3
Submitter: Zuul
Branch: master

commit a3e9f6a8694f566a96f3c4c6d4208b187e43aca3
Author: Michał Dulko <email address hidden>
Date: Wed Jun 27 11:06:05 2018 +0200

Fix health server binding address

    Flask update changed the default binding address from 0.0.0.0 to
    127.0.0.1. This affects Kuryr's healthchecks as we were depending on
    that behavior. This manifested with Kuryr pods being constantly
    restarted.

This commit fixes that by explicitly binding to 0.0.0.0.

    Change-Id: I298960c2e84befe74c35b32bcc6730ba27202604
    Related-Bug: 1778516
    Closes-Bug: 1778867

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-27: Fix merged to kuryr-kubernetes (master)

Reviewed: https://review.openstack.org/578298
Committed: https://git.openstack.org/cgit/openstack/kuryr-kubernetes/commit/?id=a991a8f7e07c9627e63e3d68028f45860857bd31
Submitter: Zuul
Branch: master

commit a991a8f7e07c9627e63e3d68028f45860857bd31
Author: Yossi Boaron <email address hidden>
Date: Wed Jun 27 10:16:15 2018 +0300

Services LBaaS driver: fix ensure LB for existing LB.

    The 'ensure_loadbalancer' function of LBaaS driver should support
    both new and existing LBs.
    This patch updates 'ensure_loadbalancer' to support properly
    existing LBs.

Change-Id: I083c58130b4a374bd4c80047aa158ba4eef14968
Closes-Bug: 1778516

Changed in kuryr-kubernetes:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-06-29: Fix proposed to kuryr-kubernetes (master)

Fix proposed to branch: master
Review: https://review.openstack.org/579144

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-07-06: Change abandoned on kuryr-kubernetes (master)

Change abandoned by Luis Tomas Bolivar (<email address hidden>) on branch: master
Review: https://review.openstack.org/579144
Reason: Abandon due to fix merged already on Octavia: https://review.openstack.org/#/c/579559/

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2018-08-21: Fix included in openstack/kuryr-kubernetes 0.5.0

This issue was fixed in the openstack/kuryr-kubernetes 0.5.0 release.

Report a bug

This report contains Public information

Everyone can see this information.

You are

Subscribing...

Edit bug mail

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.