Summary:
Align Kernel IPsec Full offload implementation in the DPU to the upstream Full
offload in all components: OFED, Strongswan, etc.
This is in order for DPU Kernel IPsec to include policy offload and be fully
aligned to what CX Kernel customers will use.
How to test:
Host 1:
/opt/mellanox/iproute2/sbin/devlink dev eswitch set pci/0000:03:00.0 mode legacy
echo 'dmfs' > /sys/bus/pci/devices/0000:03:00.0/net/p0/compat/devlink/steering_mode
echo 'full' > /sys/class/net/p0/compat/devlink/ipsec_mode
/opt/mellanox/iproute2/sbin/devlink dev eswitch set pci/0000:03:00.0 mode switchdev
BF on host 1:
/opt/mellanox/iproute2/sbin/ip xfrm policy add src 196.234.181.165 dst 196.234.182.166 dir out tmpl src 196.234.181.165/16 dst 196.234.182.166/16 proto esp reqid 0xefa83812 mode transport priority 10
/opt/mellanox/iproute2/sbin/ip xfrm policy add src 196.234.182.166 dst 196.234.181.165 dir in tmpl src 196.234.182.166/16 dst 196.234.181.165/16 proto esp reqid 0x63a7db74 mode transport priority 10
/opt/mellanox/iproute2/sbin/ip xfrm policy add src 196.234.182.166 dst 196.234.181.165 dir fwd tmpl src 196.234.182.166/16 dst 196.234.181.165/16 proto esp reqid 0x63a7db74 mode transport priority 10
/opt/mellanox/iproute2/sbin/ip xfrm state add src 196.234.181.165/16 dst 196.234.182.166/16 proto esp spi 0xefa83812 reqid 0xefa83812 mode transport aead 'rfc4106(gcm(aes))' 0xe2fe3857301d8f72b5d71d295a462ef21868e407 128 offload packet dev p0 dir out sel src 196.234.181.165/16 dst 196.234.182.166/16 flag esn replay-window 32
/opt/mellanox/iproute2/sbin/ip xfrm state add src 196.234.182.166/16 dst 196.234.181.165/16 proto esp spi 0x63a7db74 reqid 0x63a7db74 mode transport aead 'rfc4106(gcm(aes))' 0xe916c4d0db1886e8c877b023e8cebef53b4d2d0f 128 offload packet dev p0 dir in sel src 196.234.182.166/16 dst 196.234.181.165/16 flag esn replay-window 32
Start OVS and set following configure on BF:
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:max-idle=300000
Host2:
/opt/mellanox/iproute2/sbin/devlink dev eswitch set pci/0000:03:00.1 mode legacy
echo 'dmfs' > /sys/bus/pci/devices/0000:03:00.1/net/p1/compat/devlink/steering_mode
echo 'full' > /sys/class/net/p1/compat/devlink/ipsec_mode
/opt/mellanox/iproute2/sbin/devlink dev eswitch set pci/0000:03:00.1 mode switchdev
BF on host 2:
/opt/mellanox/iproute2/sbin/ip xfrm policy add src 196.234.182.166 dst 196.234.181.165 dir out tmpl src 196.234.182.166/16 dst 196.234.181.165/16 proto esp reqid 0xefa83812 mode transport priority 10
/opt/mellanox/iproute2/sbin/ip xfrm policy add src 196.234.181.165 dst 196.234.182.166 dir in tmpl src 196.234.181.165/16 dst 196.234.182.166/16 proto esp reqid 0x63a7db74 mode transport priority 10
/opt/mellanox/iproute2/sbin/ip xfrm policy add src 196.234.181.165 dst 196.234.182.166 dir fwd tmpl src 196.234.181.165/16 dst 196.234.182.166/16 proto esp reqid 0x63a7db74 mode transport priority 10
/opt/mellanox/iproute2/sbin/ip xfrm state add src 196.234.181.165 dst 196.234.182.166 proto esp spi 0xefa83812 reqid 0xefa83812 mode transport aead 'rfc4106(gcm(aes))' 0xe2fe3857301d8f72b5d71d295a462ef21868e407 128 offload packet dev p0 dir out sel src 196.234.181.165/16 dst 196.234.182.166/16 flag esn replay-window 32
/opt/mellanox/iproute2/sbin/ip xfrm state add src 196.234.181.165 dst 196.234.182.166 proto esp spi 0x63a7db74 reqid 0x63a7db74 mode transport aead 'rfc4106(gcm(aes))' 0xe916c4d0db1886e8c877b023e8cebef53b4d2d0f 128 offload packet dev p0 dir in sel src 196.234.181.165/16 dst 196.234.182.166/16 flag esn replay-window 32
Start OVS and set following configure on BF:
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:hw-offload=true
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:max-idle=300000
Send the traffic between host 1 and host 2 and check IPsec counters in "ethtool -S" statistics on both BF.
How to fix:
Need to backport a series of xfrm patches into BlueField 5.15 kernel, from 6.0 upstream kernel.
Patches needed for 5.15 kernel:
Summary:
Align Kernel IPsec Full offload implementation in the DPU to the upstream Full
offload in all components: OFED, Strongswan, etc.
This is in order for DPU Kernel IPsec to include policy offload and be fully
aligned to what CX Kernel customers will use.
How to test: iproute2/ sbin/devlink dev eswitch set pci/0000:03:00.0 mode legacy pci/devices/ 0000:03: 00.0/net/ p0/compat/ devlink/ steering_ mode net/p0/ compat/ devlink/ ipsec_mode iproute2/ sbin/devlink dev eswitch set pci/0000:03:00.0 mode switchdev
Host 1:
/opt/mellanox/
echo 'dmfs' > /sys/bus/
echo 'full' > /sys/class/
/opt/mellanox/
BF on host 1: iproute2/ sbin/ip xfrm policy add src 196.234.181.165 dst 196.234.182.166 dir out tmpl src 196.234.181.165/16 dst 196.234.182.166/16 proto esp reqid 0xefa83812 mode transport priority 10 iproute2/ sbin/ip xfrm policy add src 196.234.182.166 dst 196.234.181.165 dir in tmpl src 196.234.182.166/16 dst 196.234.181.165/16 proto esp reqid 0x63a7db74 mode transport priority 10 iproute2/ sbin/ip xfrm policy add src 196.234.182.166 dst 196.234.181.165 dir fwd tmpl src 196.234.182.166/16 dst 196.234.181.165/16 proto esp reqid 0x63a7db74 mode transport priority 10 iproute2/ sbin/ip xfrm state add src 196.234.181.165/16 dst 196.234.182.166/16 proto esp spi 0xefa83812 reqid 0xefa83812 mode transport aead 'rfc4106(gcm(aes))' 0xe2fe3857301d8 f72b5d71d295a46 2ef21868e407 128 offload packet dev p0 dir out sel src 196.234.181.165/16 dst 196.234.182.166/16 flag esn replay-window 32 iproute2/ sbin/ip xfrm state add src 196.234.182.166/16 dst 196.234.181.165/16 proto esp spi 0x63a7db74 reqid 0x63a7db74 mode transport aead 'rfc4106(gcm(aes))' 0xe916c4d0db188 6e8c877b023e8ce bef53b4d2d0f 128 offload packet dev p0 dir in sel src 196.234.182.166/16 dst 196.234.181.165/16 flag esn replay-window 32
/opt/mellanox/
/opt/mellanox/
/opt/mellanox/
/opt/mellanox/
/opt/mellanox/
Start OVS and set following configure on BF: hw-offload= true max-idle= 300000
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:
Host2: iproute2/ sbin/devlink dev eswitch set pci/0000:03:00.1 mode legacy pci/devices/ 0000:03: 00.1/net/ p1/compat/ devlink/ steering_ mode net/p1/ compat/ devlink/ ipsec_mode iproute2/ sbin/devlink dev eswitch set pci/0000:03:00.1 mode switchdev
/opt/mellanox/
echo 'dmfs' > /sys/bus/
echo 'full' > /sys/class/
/opt/mellanox/
BF on host 2: iproute2/ sbin/ip xfrm policy add src 196.234.182.166 dst 196.234.181.165 dir out tmpl src 196.234.182.166/16 dst 196.234.181.165/16 proto esp reqid 0xefa83812 mode transport priority 10 iproute2/ sbin/ip xfrm policy add src 196.234.181.165 dst 196.234.182.166 dir in tmpl src 196.234.181.165/16 dst 196.234.182.166/16 proto esp reqid 0x63a7db74 mode transport priority 10 iproute2/ sbin/ip xfrm policy add src 196.234.181.165 dst 196.234.182.166 dir fwd tmpl src 196.234.181.165/16 dst 196.234.182.166/16 proto esp reqid 0x63a7db74 mode transport priority 10 iproute2/ sbin/ip xfrm state add src 196.234.181.165 dst 196.234.182.166 proto esp spi 0xefa83812 reqid 0xefa83812 mode transport aead 'rfc4106(gcm(aes))' 0xe2fe3857301d8 f72b5d71d295a46 2ef21868e407 128 offload packet dev p0 dir out sel src 196.234.181.165/16 dst 196.234.182.166/16 flag esn replay-window 32 iproute2/ sbin/ip xfrm state add src 196.234.181.165 dst 196.234.182.166 proto esp spi 0x63a7db74 reqid 0x63a7db74 mode transport aead 'rfc4106(gcm(aes))' 0xe916c4d0db188 6e8c877b023e8ce bef53b4d2d0f 128 offload packet dev p0 dir in sel src 196.234.181.165/16 dst 196.234.182.166/16 flag esn replay-window 32
/opt/mellanox/
/opt/mellanox/
/opt/mellanox/
/opt/mellanox/
/opt/mellanox/
Start OVS and set following configure on BF: hw-offload= true max-idle= 300000
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:
/usr/bin/ovs-vsctl set Open_vSwitch . other_config:
Send the traffic between host 1 and host 2 and check IPsec counters in "ethtool -S" statistics on both BF.
How to fix:
Need to backport a series of xfrm patches into BlueField 5.15 kernel, from 6.0 upstream kernel.
Patches needed for 5.15 kernel: