Add 'labels', a list of opaque strings, to the neutron 'port' object

I'll try to tweak the title wording/examples a bit, but I gather that all you need is to ability to qualify a port based on a list of attributes or metadata that do not necessarily need to be understood or processed by the Neutron platform. A bit like proposal [1], where all we wanted to do is to tag resources in a way that's meaningful to an external party, human or otherwise. Am I on the right track?

[1] https://review.openstack.org/#/c/134069

Similar, though I am looking for a list of strings, and only think I need
them on ports.
(I would not object to something similar on other objects of that was
helpful to folks).

Happy to have you tweet wording, feel free :)

Ed

On Friday, May 29, 2015, Armando Migliaccio <email address hidden>
wrote:

> I'll try to tweak the title wording/examples a bit, but I gather that
> all you need is to ability to qualify a port based on a list of
> attributes or metadata that do not necessarily need to be understood or
> processed by the Neutron platform. A bit like proposal [1], where all we
> wanted to do is to tag resources in a way that's meaningful to an
> external party, human or otherwise. Am I on the right track?
>
> [1] https://review.openstack.org/#/c/134069
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1460222
>
> Title:
> Add 'labels', a list of opaque strings, to the neutron 'port' object
>
> Status in OpenStack Neutron (virtual network service):
> Confirmed
>
> Bug description:
> *What is being requested*
>
> This request is to add an attribute 'labels' to the neutron 'port'
> object. The type of 'labels' is a list of strings.
>
> *Why is it being requested*
>
> There are many use cases in which you wish to provide hint-by-
> reference to downstream providers of neutron services (like
> controllers) for which you would like to allow the user to provide
> hint-by-reference about the nature of the port and the service it
> should receive.
>
> In the parlance of the Neutron API Documentation:
>
> Object: ports
> Parameter: labels
> Style: plain
> Type: xsd:list
> Description: A list of opaque strings to be interpreted as
> hints-by-reference by the neutron provider
>
>
> *Examples*
>
> 1) Indicate the Network Policy that should be applied to the port.
> Most network policy systems resolve policy based upon the
> 'Endpoint Group(s)' (abbreviated EPGs) and Endpoint (think port)
> (abbreviated EP) is placed in. 'labels' could be used to indicate EPG
> membership for a port. For example:
>
> {
> "port": {
> "labels": [ "epg:blue", "epg:green" ],
> ...
> }
> }
>
> 2) Indicate the type of Virtual Network Function (VNF) of the VM
> being deployed on the port. This would be important for a controller
> rendering Service Function Chains (SFCs) to know so that it knew it
> had additional VNFs of that type to use in the SFCs it was
> constructing. For example:
>
> {
> "port": {
> "labels": [ "vnf-type:firewall-3" ],
> ...
> }
> }
>
> *Who needs it*
>
> This is needed to assist the OpenDaylight Controller in being able to
> better provide the Policy and SFC services. However, this mechanism
> is agnostic as to the underlying controller, and also can be used for
> a variety of other useful purposes. Anywhere that it would be useful
> to pass on hints-by-reference to downstream providers could take
> advantage of it.
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/neutron/+bug/1460222/+subscri...

For completeness, one other mechanism that can be leveraged today (without any changes to Neutron) for these types of use cases is the binding:profile [1]. This is part of Neutron port binding extension [2], and it would allow you to address your use cases too. If you intend to use the ODL driver, then it should be a trivial change to have the binding profile be passed all the way down to the Neutron northbound endpoint.

Therefore at this point it's a matter of gauging interest as to whether we'd want to go with a fully fledged new tagging mechanism (potentially extended to other core resources), or settle for something that may fit the bill and have the lowest implementation cost.

[1] https://blueprints.launchpad.net/neutron/+spec/ml2-binding-profile
[2] https://blueprints.launchpad.net/neutron/+spec/ml2-portbinding

If I understand correctly, binding:profile is only settable by admins, and port labels would nee to be set by non-admins... so while one could POC things potentially with binding:profile... I'm not sure that it actually meets the need due to that constraint (presuming I've understood the constraint correctly).

Moving back to confirmed based on discussion in neutron-drivers meeting today.

I'm -1 on this as proposed. I think the idea of labels (or as Nova calls it, tags), that is *purely* used by API users only is fine to me. It seems useful as a mechanism to do filtering. I am *not* in favor of this being used in any way by a Neutron backend. I think any more additions of backend specific pass-through to the Neutron API makes Neutron less valuable as an abstraction. It's very harmful for interop because the API is effectively different depending on the backend used. I realize there is some of this in the API already, but I don't think any more should be added to make that worse.

I think we're missing the point here. Once a mechanism is introduced (like binding:profile), no-one can stop anyone misusing the tool. It's not our job to police how the tool is used, I can agree that certain tools can easily be misused, and it's our job to make sure that the tool is not easily misused.

Shooting down an idea that can have a number of positive applications just because there's one that can be a bad one sounds like a terrible way of stifling innovation. I wonder what Nobel would have done if he hadn't published TNT.

I thought that triaged means:

The bug goes into the "Triaged" state once a discussion around the RFE has taken place.

https://github.com/openstack/neutron/blob/master/doc/source/policies/blueprints.rst#neutron-request-for-feature-enhancements

Once we have a final verdict we could move it to the bin or let it progress. Either way, we're splitting hairs.

@Ed: under what circumstances you would need this to be set by a low-privileged 'tenant'?

So, this kind of arbitrary metadata on objects has been rejected in the past, and the solution that is currently in the works is neutron flavors. They are defined here:
https://github.com/openstack/neutron-specs/blob/master/specs/liberty/neutron-flavor-framework.rst

... with a further enhancement here:
https://github.com/openstack/neutron-specs/blob/master/specs/liberty/neutron-flavor-framework-templates.rst

The short-form is that it allows attaching meta-data to any neutron object, if the operator configures it, and then the backend plugins/drivers can access it as described above. So you'd end up having an operator defining an "ODL firewall-3" port flavor, or an "ODL policy:blue" port flavor, and if that flavor is used for the port, it'd include that metadata.

In this scheme, operators must pre-define the flavors they want. I expect vendors will provide pre-canned flavors or templates, but this was the compromise that was acceptable in the last release in terms of letting operators be the ones to choose if they want to break object interoperability.

@Ed: btw the binding:profile access can be relaxed via policy.json [1]. So you can make it tenant-accessible.

[1] https://github.com/openstack/neutron/blob/master/etc/policy.json#L62

Russell's points are valid.

Agree with both of Doug's points.

In response to #9:

Simple example 1:

NFV Use Case.
The user has a pCPE (physical CPE, think a box in a bank branch). The tenant has decided that traffic from that pCPE is to be moved through a variety of Network Service Functions, some physical, some virtual. Because the controller has full scope on the pCPE and the physical network functions, as well as being able to handle the virtual network functions for purposes of Service Function Chaining (SFC) it is handling all of that.

One of those service functions is a vCPE. The tenant would like to deploy a new VM of type vCPE via OpenStack. The downstream controller needs to (in addition to handling normal network plumbing for that VM) *know* that its has a new instance of vCPE it can use. So the tenant uses a label 'vnf-type:vCPE' to indicate it on the 'port', and the controller's SFC subsystem can pick up on that and put it into rotation to handle traffic.

Most of the use cases I've heard for labels are both dynamic and scoped to the tenant deploying the VM .

In response to #10:

I have read through the links, and couldn't find answers to two questions there:

1) What is the relationship between a flavor and a port?
2) Can a port have multiple flavors?

In response to #11:

Thank you for that pointer. I would be curious to know though, as binding:profile was intended for admin use only, what the risk profile of exposing it to tenants is?

Suggestions as to how to look into that?

Being an internal mechanism to communicate capabilities to ML2 driver, an admin-only seems the most sensible thing to do. Bob Kukura, being the original author of the extension, would be the right contact to learn more about it.

So I've looked at the suggestions of using binding:profile or flavors for this, and this is what I *think* I'm seeing:

1) binding:profiles - is intended for admin use only, and while it *can* be tweaked to allow non-admin use, there is risk here in allowing tenants to set things that are dangerous for tenants to set.
2) flavors - It appears that you get only *one* flavor on an object. Since labels are intended to be used as a list, and in the use cases identified for them so far, that list could be an arbitrary member of the power set of labels, using flavors would require 2^n flavors (where is the number of labels) and thus wouldn't scale in any reasonable way.

Thinking a great deal about #1 (binding:profiles) it seems that we already *have* something like labels for admins in the form of binding:profiles. Because it expose things (traditionally) that are only safe for admin use, my request is effectively simply that we
have 'labels' so as to discourage dangerous use of binding:profiles for that purpose by allowing a 'safer' mechanism in the form of labels.

But labels is introduced as a mechanism which the backend is aware of, and as such is as 'dangerous' as the binding:profile if we decide to relax its access to regular tenants. Since labels would have to be used in the exact same way by the backends, how do you think they are any safer? I would argue that they are, only in case the backend is totally oblivious to them.

@Armando: I tend to break down the problem this way:

Tenant Safety:

binding:profile - by tradition used only by admin, exposes settings that may be dangerous if used by non-admins.

labels - explicitly tenant settable, and thus things not appropriate to tenants should be exposed via binding:profiles, not labels.

Having something which is explicitly tenant safe is less dangerous than abusing something that is traditionally not.

Tenant Safety is an orthogonal issue to the question of "Attaching arbitrary metadata to ports". As you point out, binding:profiles already does that. That ship has sailed. The real question is about Tenant Safety, are we going to encourage more dangerous or less dangerous behavior. If binding:profile is the only way to meet the use cases, it will be used, and it *is* more dangerous than having explicitly scoped to tenant use attribute 'labels' on ports, because it would mix things which should only be admin settable with things that should be tenant settable... thus providing to much privilege to tenants.

The more I think about it, the more I actually like using *both* binding:profiles (for admin only metadata) and labels (for tenant settable metadata).

Thoughts?

Ed, thanks for clarifying this further.

What I am not sure I understand is how you plan to guarantee the safety of labels if they are too used to influence the driver's behavior. The only way to ensure that, by design, is to make the platform completely oblivious to them (aka internal components can't manipulate them or make decision based on them). The contract of labels (as defined above) can become as weak as binding:profiles as they can be misused just like binding:profile. Just warning that labels are appropriate to tenants doesn't sell it; how do you draw the line, if ultimately the implementation is going to look like just like binding:profile? As a developer I would be utterly confused as to which one to use.

It sounds to me that we'd be introducing labels just to say: 'yeah, let's not touch binding:profile, as we don't know what happens if we do' :)

If we recognize that both can be dangerous to the tenant, then one mechanism is equivalent to the other and I would be in favor of labels, if we treat them as a binding:profile but extended to all the other core resources (not just ports alone).

Don't get me wrong, I am not opposed to the idea of labels, all I am saying is that perhaps they can just as 'dangerous' as binding:profile and we may want to restrict access only to those (human or otherwise) who can take responsability ;)

@Armando

So its perhaps useful to try to classify the kinds of potential danger, to see why labels are significantly safer (barring incorrect usage by drivers) than using binding:profile would be.

Types of Danger:

1) Tenant can make requests that are potentially harmful to the overall system stability
2) Tenants can make requests that are potentially harmful to other tenants
3) Tenants can make requests that are potentially harmful to the tenant themselves, beyond the scope of the port they added metadata to.
4) Tenants can make requests that are potentially harmful to the port they added metadata to.

I suspect we can all agree that #1 is worse than #2 is worse than #3 is worse than #4.

Exposing binging:profiles to tenants has the potential (as I understand some of its uses) to introduce at least type #2 (and possibly type #1) dangers.

Given the scoping of labels to *a* port, and the semantics of exposing things appropriate for tenants to set, barring nasty driver bugs, it should at worst be a type #4 danger, thus a significant improvement in tenant safety. And an intelligently written driver can avoid most type #4 dangers.

As to extending them to other neutron resources (not just ports), I'm completely fine with that if its useful for folks. I don't currently have any use cases for labels on things other than ports, but that doesn't mean there aren't any, and I'm generally pro-folks being able to get their needs met :)

Is your counter suggestion then that we have in binding:profiles as key 'labels' that is of type list-of-string, and applies to all neutron core resources? If so I'm open to this, but curious your thoughts on managing the classes of tenant safety noted above :)

I realize we've all glossed over Russell's comments from #6, but lets be clear: I agree with his sentiment. If the labels are being interpreted by the backend they are going to make things non-interopable because they will affect behavior differently based on what backend you have. If we all agree one of the core tenants of the Neutron API is portability between implementations, then using labels as defined here seems wrong.

Doug's comment in #10 around flavors is worth looking at. That's work we as a community have already committed to for Liberty.

@Kyle:

As noted, neutron already has a mechanism semantically that passes information to be interpreted by the backend (binding:profile).

Additionally, wrt to Doug's comment #10, as noted, the combinatronics kill you, as a flavor is a single thing, and thus given n possible labels, you wind up with 2^n possible flavors... it just doesn't scale.

@Ed, I agree with your point! We don't want to add more methods which skirt things and cause the commonality to increasingly be different and tie operators to a specific backend.

Ed, I believe that as proposed, labels cannot bar the incorrect usage by drivers, just as much as binding:profile. Therefore I would be wary of exposing them to tenants, and if we do, then we'd just have to accept the consequences as much as we'd have to if we relaxed access to binding:profile.

How would you plan to ensure that labels are not harmful from the platform standpoint if they are meant to be opaque? There's no safeguard that we can put in place other than writing big warning signs in the Neutron devref, asking developers to be careful. But even if that was acceptable what sort of pitfalls should we warn them of, if the use of labels is pretty much open ended? Have you consider how to solve this problem?

In the end, both profile and labels add metadata to ports for drivers to parse in order to do clever things. Once we have allowed tenant access, everything becomes a gamble and they can both potentially cause 3, and 4, when exposed to low-priv tenants, and 1 and 2 when exposed to the admin. Obviously no-one wants to cause this on purpose, as no-one writes buggy software willingly :O

I don't think just neutron-flavor-framework could fix this requirement, we need more based on that at least, correct me if I am wrong.

If we use the flavor concept here, we could set some flavors for port first, and it will redirect to the port create driver to create that kind of port, but where is the 'metadata' or 'tag' of the ports stored? how could we list all the ports with the same 'tag'? the tags only store in flavor db table. Flavor bp only works as a driver call entrance filter here.

@yalei, the requirement for another method to pass opaque data from users to backends isn't inline with where we want to take Neutron. Adding another method which allows for backend specific passthrough dilutes the value of Neutron as an abstraction.

There are a number of cases where this feature can be useful, not just the ones above: bare metal provisioning is one of them.

I have finally come to the conclusion, after these many cycles involved in Neutron, that it is fine to provide the ability to tag resources, with the caveat that in lack of a strong protection mechanism against abuse by neutron drivers, we restrict access to admin only as the default choice, just to err on the side of caution.

I understand that this can there's a potential danger of lock-in, but technically speaking no two Neutron deployments are alike, and interoperability (or lack thereof) if not properly defined feels like a weak argument, especially in face of so much API diversity, and extensibility of the existing platform. Surely we don't want to make things worse, but in light of what the platform can do, it doesn't seem like we're making things that much more terrible.

My 2c

I found another potential use case for labels [1]. Kindly offered by Russell's comments on [2].

[1] http://networking-ovn.readthedocs.org/en/latest/containers.html#creating-a-container-port
[2] https://review.openstack.org/#/c/94612/13/specs/liberty/vlan-aware-vms.rst

I don't think I'd want to use labels for that use case. That's stuff that deserves to be properly reflected as port attributes.

I still think labels make sense as end user defined metadata. Otherwise, for things that backends are expected to actually interpret, I think they should be well defined attributes. That provides an opportunity to review what other use cases it might satisfy and find a definition that can apply to more than one use case and backend.

@Russell, thanks for clarifying...that's why I say 'potential', as I was not sure :)

For User defined metadata that are oblivious to backends, I think we are all in agreement that they are a non brainer. In fact [1] was along those lines and it didn't get it only for lack of time.

As for your other comment, it sounds to me that the very nature of the problem we're trying to address makes incredibly difficult to come with a common ontology that can fit to the majority of the backends involved and use cases.

[1] https://review.openstack.org/#/c/134069

Kubernetes use the tag (maybe here tag) lots to dynamically select backend objects. I can think of two cases:
1. from API to list some data tagged by some tags
2. form dynamically member ship of Load balances

@Ed,

Another potential avenue that might be worth looking into is leveraging the ML2 extension framework [1]. This will allow to experiment more cleanly and without the reliance of the binding extension. It'll obviously be driver specific, but many extensions have been canonized to 'core' this way, so it might not a terrible after all.

Food for thought.

Cheers,
Armando

[1] https://github.com/openstack/neutron-specs/blob/master/specs/juno/neutron-ml2-mechanismdriver-extensions.rst

Tags (or labels, as you call them) are useful for many different use cases. Since multiple personas may leverage such an interface, it is useful to use {scope, tag} tuples instead of bare strings. This allows multiple consumers to use their own label ontology without coordination.

@Peter: I would tend to agree... you probably see a first (semi-naive) poke at that in my examples (epg:<foo>) etc.

Could you say more about a mechanism you think might be useful for this?

@Ed: Sure, here's a contrived example:

A service provider may have multiple tiers ("gold, silver, and bronze") for purchasing network bandwidth as well as latency. If a customer is able to purchase these levels separately, they may wind up with a port that is tagged for gold latency, but bronze bandwidth. While you can always mangle strings (e.g. ["latency:gold", "bandwidth:bronze"]), it is generally better to keep the scope separate from the value: [["latency", "gold"], ["bandwidth", "bronze"]] or [{"scope": "latency", "tag": "gold"}, {"scope": "bandwidth", "tag":"bronze"}]. This facilitates queries such as "which of customers purchased gold service plans?" and "what ports don't specify any latency tier".

Non-contrived examples (within and beyond networking) abound, but are left as an exercise for the reader :)

Watching the bug page like a hawk now....

I did miss his comment on the call. Do you remember?

Thx

Uri (“Oo-Ree”)
C: 949-378-7568

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Ed Warnicke
Sent: Friday, June 19, 2015 12:39 PM
To: Elzur, Uri
Subject: [Bug 1460222] Re: Add 'labels', a list of opaque strings, to the neutron 'port' object

@Peter: I would tend to agree... you probably see a first (semi-naive) poke at that in my examples (epg:<foo>) etc.

Could you say more about a mechanism you think might be useful for this?

--
You received this bug notification because you are subscribed to the bug report.
https://bugs.launchpad.net/bugs/1460222

Title:
  Add 'labels', a list of opaque strings, to the neutron 'port' object

Status in OpenStack Neutron (virtual network service):
  Triaged

Bug description:
  *What is being requested*

  This request is to add an attribute 'labels' to the neutron 'port'
  object. The type of 'labels' is a list of strings.

  *Why is it being requested*

  There are many use cases in which you wish to provide hint-by-
  reference to downstream providers of neutron services (like
  controllers) for which you would like to allow the user to provide
  hint-by-reference about the nature of the port and the service it
  should receive.

  In the parlance of the Neutron API Documentation:

  Object: ports
  Parameter: labels
  Style: plain
  Type: xsd:list
  Description: A list of opaque strings to be interpreted as
               hints-by-reference by the neutron provider

  *Examples*

  1) Indicate the Network Policy that should be applied to the port. Most network policy systems resolve policy based upon the
  'Endpoint Group(s)' (abbreviated EPGs) and Endpoint (think port) (abbreviated EP) is placed in. 'labels' could be used to indicate EPG membership for a port. For example:

  {
    "port": {
        "labels": [ "epg:blue", "epg:green" ],
         ...
     }
  }

  2) Indicate the type of Virtual Network Function (VNF) of the VM
  being deployed on the port. This would be important for a controller
  rendering Service Function Chains (SFCs) to know so that it knew it
  had additional VNFs of that type to use in the SFCs it was
  constructing. For example:

  {
    "port": {
        "labels": [ "vnf-type:firewall-3" ],
         ...
     }
  }

  *Who needs it*

  This is needed to assist the OpenDaylight Controller in being able to
  better provide the Policy and SFC services. However, this mechanism
  is agnostic as to the underlying controller, and also can be used for
  a variety of other useful purposes. Anywhere that it would be useful
  to pass on hints-by-reference to downstream providers could take
  advantage of it.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1460222/+subscriptions

I would rather the tags apply to all resources external to users.

A detail here that I am not clear on; I'm thinking of Ed's use case about a vCPE.

Why do opaque labels need to be passed through Neutron, for this use case? Presumably when the entity that would know about those labels creates a port, it gets back a port ID in the REST response. Why can't it just pass that port ID and the labels out of band to the controller (or mechanism driver, or agent, as appropriate) with the implication "when you see <port ID> come up, apply to it the behaviour implied by <labels>"?

Per the RFE decoder ring [1], marking this as "Won't Fix" since we've rejected it.

[1] https://review.openstack.org/#/c/202797/