Wrong IP Address for error message in keystone.log

Bug #1550127 reported by Dave McCowan
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Identity (keystone)
Won't Fix
Low
Unassigned
oslo.middleware
Fix Released
Undecided
Unassigned

Bug Description

When the keystone public endpoint sits behind a reverse proxy, messages written to keystone.log contain the IP address of the proxy, not the IP address of the client.

For example:

2016-02-25 20:48:21.409 60 WARNING keystone.common.wsgi [-] Authorization failed. Could not find user: foo (Disable debug mode to suppress these details.) (Disable debug mode to suppress these details.) from 192.168.1.100

The client's real IP address is passed with the request in the X-Forwarded-For header.

Other OpenStack services, such as nova, glance, and cinder have a configuration option

    use_forwarded_for = true

When this is set, their corresponding API log files record the client's real IP address as gleaned from X-Forwarded-For.

Revision history for this message
Steve Martinelli (stevemar) wrote :
Changed in keystone:
importance: Undecided → Medium
status: New → Triaged
Changed in keystone:
assignee: nobody → Viswanath Nuggu (nugguviswanathcse)
Changed in keystone:
importance: Medium → Low
Changed in keystone:
assignee: Viswanath Nuggu (nugguviswanathcse) → Venkat Rahul Dantuluri (rahuldantuluri)
Revision history for this message
Dolph Mathews (dolph) wrote :

Why is this even an option in other projects? If X-Forwarded-For is set, keystone should use it appropriately, right?

Revision history for this message
Dolph Mathews (dolph) wrote :

Bug 1554274 (Notification showing VIP ip but not client ip) is closely related.

Revision history for this message
Steve Martinelli (stevemar) wrote :

@dolph, yeah having it as an option doesn't seem necessary, we could just look to see if it's there.

Changed in keystone:
milestone: none → mitaka-rc1
Revision history for this message
Dave McCowan (dave-mccowan) wrote :

One reason to make it an option is that the value should only be trusted if Keystone is behind a trusted proxy that ensures it is set correctly. Otherwise, a malicious client can set the value to anything (the wrong IP address, unprintable characters, etc). By setting this option, an operator is letting Keystone know the header can be trusted based on the deployment.

Revision history for this message
Steve Martinelli (stevemar) wrote :

Thanks Dave, guess we'll follow the already established pattern

Changed in keystone:
assignee: Venkat Rahul Dantuluri (rahuldantuluri) → Steve Martinelli (stevemar)
status: Triaged → In Progress
Revision history for this message
Steve Martinelli (stevemar) wrote :

not necessary for rc1

Changed in keystone:
milestone: mitaka-rc1 → none
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on keystone (master)

Change abandoned by Steve Martinelli (<email address hidden>) on branch: master
Review: https://review.openstack.org/293575
Reason: track https://review.openstack.org/#/c/309038/1 instead

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to keystone (master)

Fix proposed to branch: master
Review: https://review.openstack.org/309098

Changed in keystone:
assignee: Steve Martinelli (stevemar) → Mikhail Nikolaenko (mnikolaenko)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on keystone (master)

Change abandoned by Mikhail Nikolaenko (<email address hidden>) on branch: master
Review: https://review.openstack.org/309098

Revision history for this message
Guang Yee (guang-yee) wrote :

I have never seen a production deployment where x-forwarded-for and x-forwarded-proto are not set by the proxy/LB. They are required if Keystone cluster is protected by SSL. I would even argue that we don't need make x-forwarded-proto header configurable. They are universal.

Changed in keystone:
assignee: Mikhail Nikolaenko (mnikolaenko) → Guang Yee (guang-yee)
Changed in keystone:
assignee: Guang Yee (guang-yee) → Steve Martinelli (stevemar)
Changed in keystone:
assignee: Steve Martinelli (stevemar) → Guang Yee (guang-yee)
Revision history for this message
Lance Bragstad (lbragstad) wrote :

Automatically unassigning due to inactivity.

Changed in keystone:
assignee: Guang Yee (guang-yee) → nobody
status: In Progress → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Boris Bobrov (<email address hidden>) on branch: master
Review: https://review.openstack.org/309038
Reason: probably won't be worked on

Revision history for this message
Steve Martinelli (stevemar) wrote :

Looks like this was resolved in oslo.middleware in this commit: https://github.com/openstack/oslo.middleware/commit/df01234bd864062a1dddd071b1d265153867f4b1

I'm marking it as WONTFIX for keystone and opening it up against oslo.middleware as fix-released

Changed in oslo.middleware:
status: New → Fix Released
Changed in keystone:
status: Triaged → Won't Fix
Revision history for this message
Steve Martinelli (stevemar) wrote :

This should be available in Ocata FYI

Revision history for this message
Steve Martinelli (stevemar) wrote :

Actually, I think we still have a TODO in keystone to add it to our pipeline by default, which we probably should do.

Changed in keystone:
status: Won't Fix → Opinion
Revision history for this message
Steve Martinelli (stevemar) wrote :

Sorry for the flip-flopping, we do include oslo.middleware's http_proxy_to_wsgi in the pipeline by default: https://github.com/openstack/keystone/commit/8b5c095d6f7e4dca93306f00416784303392a67c

This specific feature should be available in Ocata, just set the `[oslo_middleware] enable_proxy_headers_parsing` option to True in keysone.conf if you're using HA

Changed in keystone:
status: Opinion → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.