Traffic from OnPrem nodes behind GW to Cloud fails

The iptables rules that was applied on the GW to get this working was :

iptables -t filter -A FORWARD -i bond0 -o tap0 -j ACCEPT
iptables -t filter -A FORWARD -i tap0 -o bond0 -j ACCEPT

The pvt customer facing subnet was on the bond0 and the cloud facing subnet on tap0.

By design we are not touching FW on onPrem host. This decision was made 2 months ago. Currently the playbook for ansible looks like this:

 - name: Deploy iptables
   hosts: gateways
   become: yes
   roles:
     - role: iptables
       when: provider != "onprem"

This change was applied to not break existing FW configuration.

As a fix for this bug, we will stop the execution of the play with the detailed message and give a possible solution to fix the issue and expect that the operator/user would fix before re-trying.

Moving the importance of this issue to Medium as this is seen only on 1 testbed, where an IPTables policy to deny all except the ones in the rules in the forward tables.

For OnPrem, multicloud provisioning does not set any rule, since this is a complete customer equipment and the customer would be responsible for this. As a preventative fix, we will detect the deny policy and stop the play and recommend that the IPTables be fixed.

We do not want to change IPTables on onprem CPE as I personally have seen customers not like that approach at all. Hence we will follow the standard of what ansible does, detect an issue, abort, suggest the corrective action

As commit is not updated here, moving back to New

Adam created PR https://github.com/Juniper/contrail-multi-cloud/pull/192