4 Replies Latest reply on Sep 24, 2013 2:53 PM by tincg_cw

    Multi-VRF with NAT on NV3448 with Eth0/1 as WAN - Need same subnet on multiple internal VLANs

    tincg_cw New Member

      I have a NV3448 deployed in a small data center hosting scenario where I need to provide the same private subnet to multiple hosted customers.  The NV3448 provides firewalling and NAT svcs for the hosted virtual clients while keeping all clients seperated via VLAN interfaces throughout (ie. router, switch and virtual switch).  My attempts to implement Multi-VRF have been unsuccessful thus far.  My NAT rule doesn't seem to be working.

       

      The basic configuration is NV3448 (ADTRAN, Inc. OS version R10.5.3.E) with VLAN 106 (Client A) and VLAN 206 (Client B) for two internal client interfaces.  Both clients are required to use the same private subnet of 10.1.0.0 /16.  Utilizing different Public IPs for NAT'd traffic.

       

      1. Created vrf RED

       

      vrf RED route-distinguisher as-2byte 2:2

       

      2. Turned on ip firewall for vrf RED

       

      ip firewall

      ip firewall vrf RED

       

      3. Created multiple VLAN interfaces

       

      vlan 106

           name "ClientA"

      !

      vlan 206

           name "ClientB_VRF RED"

      !

      interface vlan 106

        description ClientA_NET

        ip address  10.1.1.45  255.255.255.0

        ip mtu 1500

        ip access-policy ClientAVMNet_VLAN106

        no shutdown

      !

      interface vlan 206

        description VRF_RED_ClientB_NET

        vrf forwarding RED

        ip address  10.1.1.45  255.255.0.0

        ip mtu 1500

        ip access-policy RED_ClientB_206

        no shutdown

       

      4. Created ACLs

       

      ip access-list extended ClientA_NAT

        remark Many:1 NAT ClientA_VM_Net

        permit ip 10.1.1.0 0.0.0.255  any

      !

      ip access-list extended RED_ClientB_NAT

        remark Many:1 NAT ClientB_VM_Net

        permit ip 10.1.0.0 0.0.255.255  any

       

      5. Created ACPs

       

      ip policy-class ClientA_106

        nat source list ClientA_NAT address x.x.x.1 overload

      !

      ip policy-class RED_ClientB_206

        nat source list RED_ClientB_NAT address x.x.x.2 overload

       

      6. WAN Interface Config

       

      interface eth 0/1

        ip address  x.y.y.202  255.255.255.252

      ip address range  x.x.x.1  255.255.255.128  secondary

        ip access-policy WAN

        no shutdown

      !

      interface loop 2

        description RED_VRF_WAN

        vrf forwarding RED

        ip address  x.x.x.2  255.255.255.255

        ip mtu 1500

        {ip access-policy WAN_RED_VRF} - not currently added, but should add correct?

        no shutdown

       

      7.  Default Routes

       

      ip route 0.0.0.0 0.0.0.0 x.y.y.201

      ip route vrf RED 0.0.0.0 0.0.0.0 loop 2

       

      *****************************************************

      I have tried various options to this configuration.  I've read through the Adtran "Configuring Multi-VRF in AOS" dated May 2013.  On the NAT, I tried it to the interface (Loop 2) and also to the IP Address...same results.  When done to the IP address, you have to add in the additional vrf RED classifier, but it still didn't make any difference.  I do not have the "ip firewall check reflexive-traffic" enabled, hence why I added the loop back interace in.

       

      Testing results:

      -From switch sourced on VLAN206, I'm able to ping NV3448 VLAN206 IP.

      -I'm able to ping the Loop 2 interface when sourced with 10.1.1.45 by adding vrf RED addition.

      -I'm NOT able to ping Loop 2 interface from the switch when sourced on VLAN 206, nor do I see any activity in the ip policy-session vrf RED

      -I'm obviously not able to ping external public IPs when sourced from 10.1.1.45 on the vrf RED.

       

      Command Syntax Used for testing:

      ping 8.8.8.8 vrf RED source 10.1.1.45

      show ip policy-sessions vrf RED

       

      I ommited several configuration sections that I didn't think were pertinent the Multi-VRF configuration.  I'm seeking clarification on the following:

      1. Why isn't my NAT statement working for the vrf RED network?

      2. In addition to creating the interface loop 2 as shown above, is there anything else I need to do with it?

      3. When do you put the loop 2 interface in the vrf and when would you not need to?

      4. If I'm not seeing the ping traffic on the vrf RED policy sessions then where is it going when I ping the Loop 2 address from the switch?

       

      Thank you in advance for your comments and suggestions.  I've also attached a visio diagram for clarity.

        • Re: Multi-VRF with NAT on NV3448 with Eth0/1 as WAN - Need same subnet on multiple internal VLANs
          Employee

          tincg_cw -

          Thank you for asking this question in the support community.  I noticed you also opened a ticket with ADTRAN Technical Support.  When you get a resolution, will you post the results on the forum so other users can benefit?

           

          Thanks,

          Noor

          • Re: Multi-VRF with NAT on NV3448 with Eth0/1 as WAN - Need same subnet on multiple internal VLANs
            petersjncv Visitor

            1. I believe you should be using the vrf classifier on your NAT statement, but I also believe you need to allow interface traffic in the policy first, then apply the NAT statement. We commonly use NAT inside VRF's on NV3448 and each one is setup in this manner.  Remember with firewalls the rule of thumb is to keep everything out, which includes keeping traffic on the LAN from reaching other networks.  You basically want to allow LAN traffic to reach anything.

             

            ip access-list extended self

              remark Traffic to Total Access

              permit ip any  any     log

             

            ip policy-class RED_ClientB_206

              allow list self self

              nat source list RED_ClientB_NAT address x.x.x.2 vrf RED overload

             

            2. You should not need to put a policy on the loopback interface.  Policies should apply to the interfaces that will actually pass the traffic.

             

            3.  You are correct to put loop 2 in VRF RED if that is the IP you are attempting to use for outbound NAT.  Your outbound default route lives in the default VRF though and herein lies the problem.  Ultimately what you want to do is get traffic to follow the default route in the default VRF, but the traffic in VRF RED is basically stuck there.   I don't think the 3448 can be configured with a "global" route table that can redistribute routes between VRF's, so this becomes problematic and things can get a little complicated.

            What I have done to get around this issue is use the eth 0/2 port as another interface in the network for VRF RED but leave it in the default VRF.  Then cable a switch port in VLAN206 to eth 0/2 directly (or through a local switch) and point your VRF RED default route to the IP on eth 0/2.  You could also make this a dot1q port and trunk multiple VLANs, thus allowing you to use this for multiple VRF's.  The major caveat here is that you need to use different subnets, because the 3448 will balk at having IPs from the same subnet range assigned to VLAN106 (default VRF) and eth0/2 (default VRF).  Also, your outbound NAT statement will actually stay the same as you have it now, so you can ignore the first post (except for adding the "self" list - that is necessary regardless) because now the NAT will take place in the default VRF and the loopback doesn't need to be in the VRF either.

             

            vlan 206

              name "vrf_red"

            !

            interface loop 2

              description RED_VRF_WAN

              ip address  x.x.x.2  255.255.255.255

            !

            interface eth 0/2

            encapsulation 802.1q

            !

            interface eth 0/2.206

              description Use for Default route - connects from sw0/1

              ip address  10.1.1.1 255.255.0.0

              vlan-id 206

              ip access-policy RED_ClientB_206

            !

            interface switchport 0/1

              desc Connects to Eth0/2

              switchport mode trunk

             

              switchport trunk allowed vlan 206

            !

            ip policy-class RED_ClientB_206

            allow list self self

            nat source list RED_ClientB_NAT address x.x.x.2 overload

            !

            ip route 0.0.0.0 0.0.0.0 x.y.y.201

            ip route vrf RED 0.0.0.0 0.0.0.0 10.1.1.1

             

            Note you do not need a policy on the VLAN interface this time because it will only be forwarding its traffic to an IP within its subnet which shouldn't be a problem for the firewall.  If you do have an issue, though, just use an access policy with only the allow list self applied.


            4.  Applying the "self" list to the policy should fix that issue.


            Are your 3448 switchports setup as access ports or trunk ports to the separate customer networks?  If you are using the switchports as access ports, you may also have issues using the same subnets because the switchports do not recognize the VRF's and you could have issues with broadcast traffic.

             

            I also recommend setting up each customer in their own VRF and leaving the default VRF customer network free.  This will give you a little more control over the specific customer networks without having to worry about affecting the other customers.


            1 of 1 people found this helpful
              • Re: Multi-VRF with NAT on NV3448 with Eth0/1 as WAN - Need same subnet on multiple internal VLANs
                tincg_cw New Member

                Sorry that it took a while to respond back.  Yes, the response helped in addition to clarification from Adtran Support.  My final solution was as follows:

                 

                vrf RED route-distinguisher as-2byte 2:2

                !

                ip firewall vrf RED

                ip firewall check reflexive-traffic

                !

                vlan 206

                  name "VRF_RED"

                !

                interface eth 0/1

                  description WANConn

                  ip address  w1.x1.y1.z1  255.255.255.252

                  ip address range  w2.x2.y2.z129  w2.x2.y2.z254  255.255.255.128  secondary

                  ip access-policy WAN

                !

                interface vlan 206

                  description VRF_RED

                  vrf forwarding RED

                  ip address  10.1.1.45  255.255.0.0

                  ip access-policy RED_VLAN_206

                !

                ip policy-class RED_VLAN_206

                  nat source list RED_VLAN206Net_NAT address w2.x2.y2.z254 overload

                !

                ip policy-class WAN

                  nat destination list RED_VLAN206_PrtFwd address 10.1.1.200 vrf RED

                !

                ip route vrf RED 0.0.0.0 0.0.0.0 10.1.255.254

                !

                 

                **********************************************************************************

                Things to note:

                 

                1. the IP in the VRF Route (10.1.255.254) doesn't live anywhere on the router.  For reasons I can't clearly explain, it kind of tricks the router in routing the traffic from the VRF firewall down to the default VRF firewall so the NAT can work properly.

                 

                2.  I also found that is you use the GUI then the VRF rules don't show up in the security zones however they are there.  Since the only practical way to reprioritize rules is in the GUI, I found that if you move the default VRF rules in security zones then they non-default rules flow in suite.  I realize this is probably clear as mud, basically move the rule above or below where the non-default VRF rule would be in the GUI and it will reprioritize the non-default vrf rules in sequence as if they were showing.

              • Re: Multi-VRF with NAT on NV3448 with Eth0/1 as WAN - Need same subnet on multiple internal VLANs
                levi Employee

                tincg_cw:

                 

                I marked this post as "assumed answered," but reply with the outcome of this post when you get a chance.

                Levi