Tracking down an invalid source mac address
I have inherited support of a remote site which contains a Cisco 4500 and is connected to ~2 dozen cisco access switches - primarily 2960s with a couple of 3750s and 3560s. Not all access switches are directly connected to the 4500 - there is some daisy chaining of switches which was apparently done as a result of inadequate cabling. Recently i've noticed serror messages on the 4500 which indicate frames have been received with an invalid source mac address:
*Sep 10 09:29:48.609: %C4K_L2MAN-6-INVALIDSOURCEADDRESSPACKET: (Suppressed 102563 times)Packet received with invalid source MAC address (00:00:00:00:00:00) on port Te5/1 in vlan 1460
The device connected to Te5/1 is an access switch (Cisco 3750). It in turn is connected to 6 other access switches. After a bit of googling it appears the 4500 is the only cisco platform which logs invalid source mac addresses. From my reading, other platforms (2960, 3750, etc) seem to forward the frames along but don't log them as invalid, nor do they add an entry to the mac address-table. I suspect the root cause of the invalid source mac addresses could be a faulty nic, a software bug or perhaps a misconfigured vmware server. What tools are available on the access switches to track down the offending port?
Deleted my post, didn't realize that they weren't visible at all. If the switch won't put them in CAM then I guess your best bet is to run SPAN session on switches but even then it would be tricky to find the source port. Another option would be to disable unknown unicast and see if anything breaks. I'm surprised that communication works though. If a host with that MAC sends something outside it subnets the GW would have to ARP to see the MAC of the host and encapsulate the frame, does the GW have any weird ARP mappings?
According to https://supportforums.cisco.com/docs/DOC-36000 these frames should be dropped in HW so at least it shouldn't affect performance of the switches.
Yes, according to that link: "Please note that packets with invalid MAC address will be dropped anyway, all other Cisco Catalyst switches are silently dropping these packets in HW, 4k platform is explicitly generating logging message when such event is observed." ...but I know this can't really be the case since the 4500 is complaining about frames that are arriving on Te5/1 which is the port connected to the 3750. This would indicate the 3750 is forwarding the frames w/ the invalid source mac despite what DOC-36000 says.
You could try if the frames can be blocked using a MAC ACL on interfaces and/or on vlans on the access switches. By applying the blocks selectively and checking if the error messages on the 4500 disappear or not, you can home in on the source of the traffic.
Moving cables around to see if the port mentioned in the error message on the 4500 follows could also help, but might prove tricky in a production environment.