So it was a usual day in the wonderful world of networking and projects. Then I was suddenly lost in the Private VLANs. You know, the feature that has existed a long time in Cisco switches but still there are some known and unknown limitations in using it. Let’s talk about PVLANs.
First the basics (use configuration guides to support you if you don’t understand what I’m talking about):
- Primary VLAN: That’s the VLAN that is the “umbrella” around the secondary VLANs.
- Secondary VLAN: One primary VLAN can have one or more secondary VLANs. Secondary VLAN can be either isolated VLAN (only at most one for each primary VLAN) or community VLAN (zero or more can exist for each primary VLAN).
- Hosts connected to PVLAN host ports that are configured with isolated VLANs (or PVLAN isolated trunk ports) can only talk to the promiscuous ports. Promiscuous ports are the ports that connect to the common network elements in that VLAN like routers.
- Hosts connected to PVLAN host ports that are configured with community VLANs can talk to the promiscuous ports and to the other community ports in the same community VLAN. They cannot talk to the other community VLANs.
- Hosts connected to the promiscuous ports can talk to every port in the area of that particular primary VLAN.
These are kind of hard to explain in very exact way but I hope you got the points:
- Isolated means that hosts cannot talk to each other, promiscuous port is always an exception
- Community means that there is normal communication within that community but no access to other communities, and promiscuous port is again the exception that can talk to everyone (and back).
If you want to understand why these primary and secondary VLANs are needed you can think the frame flow like this:
- The primary VLAN is used to direct traffic from the promiscuous ports to the secondary ports
- The secondary VLAN is used to direct traffic from the isolated ports to the promiscuous ports, and from the community ports to the other identical community ports and to the promiscuous ports.
At this point we are ready for some configuration.
Let’s say I have VLAN 100 as the primary VLAN and VLAN 101 as the isolated (secondary) VLAN:
n5k-1(config)# feature private-vlan n5k-1(config)# vlan 101 n5k-1(config-vlan)# private-vlan isolated n5k-1(config-vlan)# exit n5k-1(config)# vlan 100 n5k-1(config-vlan)# private-vlan primary n5k-1(config-vlan)# private-vlan association 101 n5k-1(config-vlan)# exit n5k-1(config)# sh vlan private-vlan Primary Secondary Type Ports ------- --------- --------------- ------------------------------------------- 100 101 isolated
Now, I have a router that I want to be routing this VLAN to the outside world, and I want to connect it with a PVLAN promiscuous trunk. Note that PVLAN promiscuous port is not supported on FEX so a physical port on Nexus 5000 is needed:
n5k-1(config)# int e1/2 n5k-1(config-if)# switchport mode private-vlan trunk promiscuous n5k-1(config-if)# switchport private-vlan mapping trunk 100 101
Now the switch knows that the traffic going to the router needs to be mangled specially. It will adjust the outgoing VLAN tags so that the router only sees packets in VLAN 100. (You remember, traffic coming from the secondary VLAN ports will be using VLAN 101, that’s how the switches know how to keep the traffic from going in wrong places.)
I want to connect a server on the FEX with a PVLAN isolated trunk (because I want it as a trunk port on the server for whatever reason, to save interfaces or so):
n5k-1(config-if)# int e100/1/9 n5k-1(config-if)# switchport mode private-vlan trunk promiscuous ERROR: requested config change not allowed
That was just a distraction to show you that promiscuous port is really not supported on the FEX! Let’s continue with our real configuration, PVLAN isolated trunk:
n5k-1(config-if)# switchport mode private-vlan trunk secondary n5k-1(config-if)# switchport private-vlan association trunk 100 101
“Secondary” keyword kind of differentiates it from the promiscuous trunk that was used for the router port. The command is accepted also without it, it is then added automatically in the configuration.
The association statement will advise the switch to change the VLAN tags accordingly: the server will only see the secondary VLAN tag entering the NIC. VLAN 101 is then configured on the server side on this trunk.
Let’s see what the switch says about the port status:
n5k-1(config-if)# sh int e100/1/9 switchport Name: Ethernet100/1/9 Switchport: Enabled Switchport Monitor: Not enabled Operational Mode: Private-vlan trunk secondary Access Mode VLAN: 1 (default) Trunking Native Mode VLAN: 1 (default) Trunking VLANs Allowed: 1-3967,4048-4093 Voice VLAN: none Extended Trust State : not trusted [COS = 0] Administrative private-vlan primary host-association: none Administrative private-vlan secondary host-association: none Administrative private-vlan primary mapping: none Administrative private-vlan secondary mapping: none Administrative private-vlan trunk native VLAN: 1 Administrative private-vlan trunk encapsulation: dot1q Administrative private-vlan trunk normal VLANs: none Administrative private-vlan trunk private VLANs: (100,101) Operational private-vlan: (100,101) Unknown unicast blocked: disabled Unknown multicast blocked: disabled
By the way, don’t bother checking the PVLAN trunk ports with “sh int trunk” command, it does not really help you at all regarding the PVLAN situation.
I want also trunk a normal VLAN (30) to the server:
n5k-1(config-if)# switchport private-vlan trunk allowed vlan 30 n5k-1(config-if)# sh run int e100/1/9 ... version 5.1(3)N2(1a) interface Ethernet100/1/9 switchport mode private-vlan trunk secondary switchport private-vlan trunk allowed vlan 30 switchport private-vlan association trunk 100 101 n5k-1(config-if)# sh int e100/1/9 switchport Name: Ethernet100/1/9 Switchport: Enabled Switchport Monitor: Not enabled Operational Mode: Private-vlan trunk secondary Access Mode VLAN: 1 (default) Trunking Native Mode VLAN: 1 (default) Trunking VLANs Allowed: 1-3967,4048-4093 Voice VLAN: none Extended Trust State : not trusted [COS = 0] Administrative private-vlan primary host-association: none Administrative private-vlan secondary host-association: none Administrative private-vlan primary mapping: none Administrative private-vlan secondary mapping: none Administrative private-vlan trunk native VLAN: 1 Administrative private-vlan trunk encapsulation: dot1q Administrative private-vlan trunk normal VLANs: 30 Administrative private-vlan trunk private VLANs: (100,101) Operational private-vlan: (100,101) Unknown unicast blocked: disabled Unknown multicast blocked: disabled
Note that the primary or secondary VLAN was not needed to be added in the “switchport private-vlan trunk allowed vlan” statement at all. This is kind of logical because we explicitly configured the PVLAN isolated trunk on the interface so why would we then want to prune the isolated VLAN from the trunk? It goes there automatically. If you want to prune it then just don’t configure the PVLAN association at all!
If I would need more PVLANs trunked to the same server (because it is a member of several PVLANs), I can do that as well:
n5k-1(config-if)# vlan 111 n5k-1(config-vlan)# private-vlan isolated n5k-1(config-vlan)# vlan 110 n5k-1(config-vlan)# private-vlan primary n5k-1(config-vlan)# private-vlan association 111 n5k-1(config-vlan)# int e100/1/9 n5k-1(config-if)# switchport private-vlan association trunk 110 111 n5k-1(config-if)# sh run int e100/1/9 ... interface Ethernet100/1/9 switchport mode private-vlan trunk secondary switchport private-vlan trunk allowed vlan 30 switchport private-vlan association trunk 100 101 switchport private-vlan association trunk 110 111 n5k-1(config-if)# sh int e100/1/9 switchport Name: Ethernet100/1/9 Switchport: Enabled Switchport Monitor: Not enabled Operational Mode: Private-vlan trunk secondary Access Mode VLAN: 1 (default) Trunking Native Mode VLAN: 1 (default) Trunking VLANs Allowed: 1-3967,4048-4093 Voice VLAN: none Extended Trust State : not trusted [COS = 0] Administrative private-vlan primary host-association: none Administrative private-vlan secondary host-association: none Administrative private-vlan primary mapping: none Administrative private-vlan secondary mapping: none Administrative private-vlan trunk native VLAN: 1 Administrative private-vlan trunk encapsulation: dot1q Administrative private-vlan trunk normal VLANs: 30 Administrative private-vlan trunk private VLANs: (100,101) (110,111) Operational private-vlan: (100,101) (110,111) Unknown unicast blocked: disabled Unknown multicast blocked: disabled
Let’s return to our PVLAN promiscuous trunk again (port e1/2):
n5k-1(config-if)# sh run int e1/2
...
interface Ethernet1/2
switchport mode private-vlan trunk promiscuous
switchport private-vlan mapping trunk 100 101
I just added the VLANs 110 and 111 above, so I’ll configure the router to route those as well, and configure the switch port accordingly:
n5k-1(config-if)# int e1/2 n5k-1(config-if)# switchport private-vlan mapping trunk ? <1-3967,4048-4093> Primary private VLAN n5k-1(config-if)# switchport private-vlan mapping trunk 110 ? <1-3967,4048-4093> Secondary VLAN IDs add Add a VLAN to private VLAN list remove Remove a VLAN from private VLAN list n5k-1(config-if)# switchport private-vlan mapping trunk 110 111 n5k-1(config-if)# sh run int e1/2 ... interface Ethernet1/2 switchport mode private-vlan trunk promiscuous switchport private-vlan mapping trunk 100 101 switchport private-vlan mapping trunk 110 111
Note that there is “add/remove” syntax as well. In this version of NX-OS (5.1(3)N2(1a)) there is however no difference in using the “add” keyword or not. Let me show that:
n5k-1(config-if)# vlan 102 n5k-1(config-vlan)# private-vlan community n5k-1(config-vlan)# vlan 100 n5k-1(config-vlan)# private-vlan association 102 n5k-1(config-vlan)# sh vlan id 100 private-vlan Primary Secondary Type Ports ------- --------- --------------- ------------------------------------------- 100 101 isolated Eth1/2, Eth100/1/9 100 102 community n5k-1(config-vlan)# int e1/2 n5k-1(config-if)# switchport private-vlan mapping trunk 100 102 n5k-1(config-if)# sh run int e1/2 ... interface Ethernet1/2 switchport mode private-vlan trunk promiscuous switchport private-vlan mapping trunk 100 101-102 switchport private-vlan mapping trunk 110 111
So I didn’t use “add” but it was added, it didn’t replace VLAN 101 there. I’ll remove 102 and try with “add”:
n5k-1(config-if)# switchport private-vlan mapping trunk 100 remove 102 n5k-1(config-if)# sh run int e1/2 ... interface Ethernet1/2 switchport mode private-vlan trunk promiscuous switchport private-vlan mapping trunk 100 101 switchport private-vlan mapping trunk 110 111 n5k-1(config-if)# switchport private-vlan mapping trunk 100 add 102 n5k-1(config-if)# sh run int e1/2 ... interface Ethernet1/2 switchport mode private-vlan trunk promiscuous switchport private-vlan mapping trunk 100 101-102 switchport private-vlan mapping trunk 110 111
The effect was exactly the same. Note that this behaviour might change, and the configuration guide for 5.1(3)N2(1) don’t mention “add/remove” syntax for promiscuous trunks at all.
Now I have covered the cases of
- PVLAN promiscuous trunk
- PVLAN isolated trunk.
Let’s see how goes the other non-trunk PVLAN cases: PVLAN promiscuous port and PVLAN host port.
PVLAN promiscuous port is used for connecting common-access devices that are not using trunk. An example is a monitoring server that needs access to all hosts in the segment. As mentioned above, PVLAN promiscuous ports cannot be FEX ports so a Nexus 5000 port must be used.
n5k-1(config-if)# int e1/3 n5k-1(config-if)# switchport mode private-vlan promiscuous n5k-1(config-if)# switchport private-vlan mapping 100 101 n5k-1(config-if)# sh run int e1/3 ... interface Ethernet1/3 switchport mode private-vlan promiscuous switchport private-vlan mapping 100 101 n5k-1(config-if)# sh int e1/3 switchport Name: Ethernet1/3 Switchport: Enabled Switchport Monitor: Not enabled Operational Mode: Private-vlan promiscuous Access Mode VLAN: 100 (VLAN0100) Trunking Native Mode VLAN: 1 (default) Trunking VLANs Allowed: 1-3967,4048-4093 Voice VLAN: none Extended Trust State : not trusted [COS = 0] Administrative private-vlan primary host-association: none Administrative private-vlan secondary host-association: none Administrative private-vlan primary mapping: 100 Administrative private-vlan secondary mapping: 101 Administrative private-vlan trunk native VLAN: 1 Administrative private-vlan trunk encapsulation: dot1q Administrative private-vlan trunk normal VLANs: none Administrative private-vlan trunk private VLANs: none Operational private-vlan: (100,101) Unknown unicast blocked: disabled Unknown multicast blocked: disabled
Since the port was promiscuous the mapping statement is needed to instruct the switch with VLAN mangling. For the host (the monitoring server) this does not look anything special since it is using normal untagged traffic, just like an access port (but this is really a PVLAN promiscuous port). Access VLAN is interestingly shown here as well, that kind of confirms our principles of the forwarding directions (primary VLAN (100) is used to carry traffic from the promiscuous ports).
The “add/remove” syntax is similar to the PVLAN promiscuous trunk example above.
The last case is PVLAN host port. Since it is not a promiscuous port it can reside in the FEX as well:
n5k-1(config-if)# int e100/1/10 n5k-1(config-if)# switchport mode private-vlan host n5k-1(config-if)# switchport private-vlan host-association 100 101 n5k-1(config-if)# sh run int e100/1/10 ... interface Ethernet100/1/10 switchport mode private-vlan host switchport private-vlan host-association 100 101 n5k-1(config-if)# sh int e100/1/10 switchport Name: Ethernet100/1/10 Switchport: Enabled Switchport Monitor: Not enabled Operational Mode: Private-vlan host Access Mode VLAN: 101 (VLAN0101) Trunking Native Mode VLAN: 1 (default) Trunking VLANs Allowed: 1-3967,4048-4093 Voice VLAN: none Extended Trust State : not trusted [COS = 0] Administrative private-vlan primary host-association: 100 Administrative private-vlan secondary host-association: 101 Administrative private-vlan primary mapping: none Administrative private-vlan secondary mapping: none Administrative private-vlan trunk native VLAN: 1 Administrative private-vlan trunk encapsulation: dot1q Administrative private-vlan trunk normal VLANs: none Administrative private-vlan trunk private VLANs: none Operational private-vlan: (100,101) Unknown unicast blocked: disabled Unknown multicast blocked: disabled
It looks like a normal access port for the host but the host-association statement again tells the switch what kind of traffic to expect from this port and how to forward it. Now the access VLAN is shown as 101 because that is the VLAN that the switch internally uses for forwarding the frames from this access port.
There you go. I’ll probably write later about other PVLAN trunks, let’s see.
Nexus 5000 Series Configuration Guides: http://www.cisco.com/en/US/products/ps9670/products_installation_and_configuration_guides_list.html
Thanks, this helped me understand how trunking works with PVLANs.