How do I set up a VLAN between an ESXi VM and Physical Hardware?
First of all, I am a total noob. The extent of my VLAN experience is setting up a guest network on a wireless router.
I have recently gotten a request to set up a public network requiring filtering and throttling (via Untangle). Meanwhile, there is a workstation in the same building which should have unrestricted access.
My question is, what do I do on the ESXi switches and/or the Netgear GS108E switch to logically separate the networks? As of now, everything is flowing through the "red" path of no/all/default LAN. I need the workstation [w] to flow through the "blue" path, while keeping everything else on the "red" path. I have read dozens of VLAN articles, but nothing is making the default/tagging/untagged/native VLAN concepts click.
802.1q is the technical standard for VLAN tagging (http://en.wikipedia.org/wiki/IEEE_802.1Q). This standard includes the placement of a "VLAN Tag" inside the header of Ethernet frames. This tag allows a link to carry multiple VLANs, as long as both devices recognize 802.1q tagging, because the tag contains the VLAN ID that the traffic belongs to.
Such a link is commonly referred to as a "trunk link" or "802.1q trunk", etc. In such an environment, there is typically a single VLAN that's assigned the role of "native VLAN". The term may also be "untagged VLAN", because that's what a native VLAN is - a particular VLAN that crosses a trunk link with no VLAN tag. Since there's no way to identify which VLAN a packet belongs to without this tag, only one VLAN can be designated the "native VLAN", and it's a good idea to make sure this value matches on both sides of a trunk.
In ESXi, you can define port groups with a VLAN ID. If you leave this field blank (or specify 0) then that port group's traffic egresses the host with no VLAN tag. This is okay if you have only one port group on the host and that vSwitch only plugs into one link, because you can simply make that port an access port, no trunk necessary. However, you need multiple VLANs to be passed on the same link (or link bundle), so I would make sure trunking is configured on the switchports that your host connects to, then all you'll need to do is enter the appropriate VLAN ID per vSwitch port group. ESXi will tag frames entering that port group as they exit the host.
It's important to configure your switchports in "VLAN Trunk" or "VLAN Tagging" mode because non trunk ports do not accept tagged frames (they get dropped). Trunk ports accept tagged frames, and you will be sending tagged frames from your ESXi host with the above configuration.
From what I can tell, the switch you show in the diagram should support all of this but it might not be the most intuitive or use the same terms. I'd recommend sticking with the documentation and see if you can get it working. http://www.netgear.com/business/products/switches/prosafe-plus-switches/GS108E.aspx
Port 1 should have VLAN 14 tagged. Port 4 should belong to the VLAN 14 group but have VLAN 14 untagged as the workstation does not have a concept of VLANs.
In the Netgear switch configuration select 802.11Q, not port based. It is a little more complex, but you need it.
You can have as many VLANs as you need on each port, but only one (called the PVID) can be untagged. Set port 1's PVID as 1, but have it joined to both VLANs 1 and 14. Set port 4's PVID as 14, and only join it to VLAN 14. All other ports are VLAN 1 PVID 1
The only way I could get hosts to properly communicate through a windows ESXI based machine (I could see traffic tagged frm the ESXI but return traffic was showing up untagged -- without vlan)
Needed to enable Monitor Mode on Windows to not strip the VLAN
here is how it's done. One entry into the registry and it's all working. http://www.intel.com/support/network/sb/CS-005897.htm
Hint when experimenting with VLANs: Have all your switch ports be either trunk (carrying ONLY tagged packets and rejecting anything else) or access (only non-tagged packets, and carrying untagged packets to and from exactly one VLAN that is on the trunk). Disable any port "modes" you do not need.
Mixed tagged-untagged on one port might make sense for eg shoehorning a VOIP network into a LAN cable plant - but even there it is a somewhat unclean solution with questionable security. For a multi-DMZ "fabric" shared by vSphere hosts, it is simply bad practice in most cases. It could make sense on hosts that sorely lack physical network ports (but then, you could tag EVERYTHING and let the switch take care of it). Consider the "Default VLAN" a drainage sump, you don't want anything getting soaked in it.