HI Patrick,
We went ahead and used 'ethregs' to pull the NIC registers in hopes of finding an issue. One of our enginners then looked at the code and the registers and came up with a write-up. In his mind, something is amiss and he sees the multicast registers amiss. See below for his write up. I dont see a way to upload the dumps to this forum but if you need them, we have them ready.
Many thanks!
Shawn
====================================================================
We dumped the registers for the ixgbe and ixgbevf
When an IPv6 address is configured in the VM, 2500::339:0:0:42:110 in our case, that triggers assigning an IPv6 multicast interface address, ff02::1:ff42:110 to the interface, to react to Neighbor Discovery requests. That IP multicast address is also translated to a multicast MAC address, as are all multicasts, 33:33:ff:42:01:10 in this case. When we try to pingv6 that address, the trace looks like this:
42:4c:43:50:01:64 (oui Unknown) > 33:33:ff:42:01:10 (oui Unknown), ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32)
2500::339:0:0:42:10 > ff02::1:ff42:110: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has 2500::339:0:0:42:110
source link-address option (1), length 8 (1): 42:4c:43:50:01:64
0x0000: 424c 4350 0164
When the IPv6 address is assigned on the VM, I expect that the VM will send a request to the host, asking for the multicast address to be configured. Since only the multicast overflow is used, that should cause the relevant PFVML2FLT to have ROPE set, and that the MTA will have data for the multicast MAC, and that MCSTCTRL will have bit 2 set to enable the multicast table array filter.
What we see, is that the MCSTCTRL has bit 2 set to 0, which should only happen when the MTA array is all zero. The only thing which indirectly causes the MCSTCTRL bit 2 to be reset after the MTA array is populated is
Additionally, PFVML2FLT[00] has receive multicast overflow set, but [01] does not:
PFVML2FLT[00]=0x0b000000: No Multicast Promiscuous, Broadcast Accept, not receive MAC overflow, receive overflow multicast, accept untagged packets without VLAN tag.
PFVML2FLT[01]=0x19000000: Multicast Promiscuous, Broadcast Accept, not receive MAC overflow, not receive overflow multicast, accept untagged packets without VLAN tag
The only way MCSTCTRL bit 2 (IXGBE_MCSTCTRL_MFE) should be reset according to the software is if there are no multicast MACs in the MTA table, but there are such MACS.
MCSTCTRL: 0: 1:0 selects multicast MAC offset -> 47:36. BUT!!! bit 2=0 -> multicast table array filter is disabled!!!!!
MTA: Multicast Table Array, selects 4096 multicast addresses which can be forwarded, bits selected based on MCSTCTRL register:
MTA[000]=0x00010000
MTA[046]=0x10000000
MTA[050]=0x00010000
MTA[113]=0x20000000
For second port:
MTA[000]: 0x00010000
MTA[008]: 0x00000003
MTA[050]: 0x00000001
MTA[070]: 0x00002000