Vanilla plugin uses hardcoded password for Oozie MySQL database user

Bug #1541122 reported by Michael McCune
258
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Security Advisory
Won't Fix
Undecided
Unassigned
Sahara
Fix Released
High
Mikhail

Bug Description

When deploying clusters with the vanilla plugin, the Oozie[1] application must be configured to use a database for storing data related to the scheduling, running, and processing of Hadoop jobs. Oozie is the primary scheduler for jobs entering the Hadoop ecosystem through the vanilla plugin.

Sahara configures the credentials for Oozie to access its database, this can be seen in sahara/plugins/vanilla/hadoop2/oozie_helper.py [2]. These credentials are hardcoded, and use a weak password.

An intruder with access to the nodes of a cluster that is created by sahara with the vanilla plugin will have access to the database that backs the Oozie installation. With this access, the intruder could change the operational effects of Oozie to produce results other than expected, for example inserting new jobs or altering configurations associated with currently running jobs.

As sahara has ultimate control over the deployment and configuration of Oozie on nodes deployed in its clusters, this hardcoded password should be changed in favor of a random password that will be generated uniquely for each deployed cluster. Oozie uses the values associated with the configurations defined in [2] to create the credentials, this means that the change should be a matter of simply changing the source valueof the password for the Oozie user.

[1]: https://oozie.apache.org/
[2]: https://github.com/openstack/sahara/blob/master/sahara/plugins/vanilla/hadoop2/oozie_helper.py#L41

Tags: security
Revision history for this message
Michael McCune (mimccune) wrote :

i think the proper fix for this may be as simple as the following patch:

diff --git a/sahara/plugins/vanilla/hadoop2/oozie_helper.py b/sahara/plugins/vanilla/hadoop2/oozie_helper.py
index 734e99f..d51a2ea 100644
--- a/sahara/plugins/vanilla/hadoop2/oozie_helper.py
+++ b/sahara/plugins/vanilla/hadoop2/oozie_helper.py
@@ -13,6 +13,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.

+import uuid
+

 def get_oozie_required_xml_configs(hadoop_conf_dir):
     """Following configs differ from default configs in oozie-default.xml."""
@@ -45,5 +47,5 @@ def get_oozie_mysql_configs():
         'oozie.service.JPAService.jdbc.url':
         'jdbc:mysql://localhost:3306/oozie',
         'oozie.service.JPAService.jdbc.username': 'oozie',
- 'oozie.service.JPAService.jdbc.password': 'oozie'
+ 'oozie.service.JPAService.jdbc.password': uuid.uuid4().hex
     }

Revision history for this message
Michael McCune (mimccune) wrote :

meant to attach this

Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

Since this report concerns a possible security risk, an incomplete security advisory task has been added while the core security reviewers for the affected project or projects confirm the bug and discuss the scope of any vulnerability along with potential solutions.

description: updated
Changed in ossa:
status: New → Incomplete
Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

Is this something a regular user can easily abuse, e.g.: is the oozie backend reachable from a job submitted by a regular users, or is it in the sahara control plane ?

Apologize for the dumb question, but do sahara user have direct access to the node of the cluster?

Revision history for this message
Michael McCune (mimccune) wrote :

The oozie backend will be deployed into a tenant network that is specified when the cluster is created. In this respect the oozie node is accessible to anyone who can access that network, this is *not* on the control plane.

So, any user who is part of the project that contains a sahara cluster with an oozie node would, theoretically, be able to access the oozie mysql instance.

Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

IIUC correctly, oozie backend is not shared between tenant and it is only accessible from the tenant's network. So the risk would be an user messing with its own deployment, without impact for other tenants.

Assuming a compromised oozie service can not escalate to the sahara service (e.g., by stalling operation from the api for example), then I think this is a class D type of bug according to VMT taxonomy ( https://security.openstack.org/vmt-process.html#incident-report-taxonomy )

@sahara-coresec, what do you think ?

Revision history for this message
Michael McCune (mimccune) wrote :

your description of the networking issue is accurate Tristan.

i also concur that compromising the oozie mysql backend should not result in the capability for an attacker to escalate to the sahara service. a class D type bug sounds appropriate.

Revision history for this message
Tristan Cacqueray (tristan-cacqueray) wrote :

@sahara-coresec, any concern if we close the ossa task and make this bug public ?

Revision history for this message
Michael McCune (mimccune) wrote :

no concerns from me, i think we can move forward Tristan

Revision history for this message
Vitalii Gridnev (vgridnev) wrote :

Agreed with Michael. Let's move forward

information type: Private Security → Public Security
Changed in sahara:
assignee: nobody → Michael McCune (mimccune)
importance: Undecided → Critical
Changed in sahara:
milestone: none → newton-1
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to sahara (master)

Fix proposed to branch: master
Review: https://review.openstack.org/312497

Changed in sahara:
status: Confirmed → In Progress
description: updated
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on sahara (master)

Change abandoned by Michael McCune (<email address hidden>) on branch: master
Review: https://review.openstack.org/312497

Jeremy Stanley (fungi)
Changed in ossa:
status: Incomplete → Won't Fix
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on sahara (stable/liberty)

Change abandoned by Vitaly Gridnev (<email address hidden>) on branch: stable/liberty
Review: https://review.openstack.org/312500
Reason: It looks like this change should not be uploaded until actual fix merged to master, so I'm abandoning the change

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on sahara (stable/mitaka)

Change abandoned by Vitaly Gridnev (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/312499
Reason: It looks like this change should not be uploaded until actual fix merged to master, so I'm abandoning the change

Changed in sahara:
milestone: newton-1 → newton-2
Changed in sahara:
importance: Critical → High
no longer affects: sahara/liberty
no longer affects: sahara/mitaka
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on sahara (master)

Change abandoned by Michael McCune (<email address hidden>) on branch: master
Review: https://review.openstack.org/312497
Reason: abandoning this as there is a change coming from another author

Mikhail (mlelyakin)
Changed in sahara:
assignee: Michael McCune (mimccune) → Mikhail (mlelyakin)
Changed in sahara:
milestone: newton-2 → newton-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to sahara (master)

Reviewed: https://review.openstack.org/325897
Committed: https://git.openstack.org/cgit/openstack/sahara/commit/?id=d29b4cfd04e201500e861086e86274a443243eba
Submitter: Jenkins
Branch: master

commit d29b4cfd04e201500e861086e86274a443243eba
Author: Mikhail Lelyakin <email address hidden>
Date: Mon Jun 6 15:49:40 2016 +0300

    Remove hardcoded password for Oozie service

    Change will correct the oozie configuration to use a random
    password instead of the hardcoded one currently in use.
    Closes-bug: 1541122
    Change-Id: I31c54aefc3c3ac65d928d9892be485ac754b6b10

Changed in sahara:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/sahara 5.0.0.0b3

This issue was fixed in the openstack/sahara 5.0.0.0b3 development milestone.

To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.