3 Replies Latest reply on Nov 21, 2019 7:56 AM by brian.carnett

    NV3140 High cpu usage when used for a sip trunk

    brian.carnett New Member

      NV3140 being used for a SIP Trunk

      Issues are intermittent fast busy when calling out, and possibly some incoming calls failing.  Secondary issue is the customer is trying to use a fax service across this, which we have recommended against, and faxes are all partial failures.

       

      I added  "ip ffe max-entries 500000"  to the interfaces and it has greatly helped with cpu usage lowering from hitting 100% consistently to only rarely hitting as high as 92%.

      The sip trunk is limited to 8 call paths and doing a sip debug it looks normal.  I am not seeing SIP messages from any outside IP's or reasons for failed calls.  This is leading me to believe the high cpu usage was causing issues and while ffe has helped it does not solve the problem with the config.

       

       

      #show running-config

      Building configuration...

       

      ! ADTRAN, Inc. OS version R13.5.1

      ! Boot ROM version R11.5.0

      ! Platform: NetVanta 3140, part number 1700341F1

      ! Serial number xxxxxxxxx

       

      hostname xxxxxxx

      enable password md5 encrypted xxxxxxx

       

      clock timezone -5-Eastern-Time

       

      ip subnet-zero

      ip classless

      ip routing

      ipv6 unicast-routing

       

      name-server xxxxxxx xxxxxxx

       

      no auto-config

      auto-config authname xxxxxx encrypted password xxxxxx

       

      event-history on

      no logging forwarding

      no logging email

       

      service password-encryption

      username "admin" password encrypted "xxxxxx"

      ;

       

      ip firewall

      ip firewall stealth

      no ip firewall alg msn

      no ip firewall alg mszone

      no ip firewall alg h323

       

      no dot11ap access-point-control

       

      qos map VoIP 0

        match ip list SIP_port

        set dscp 24

      qos map VoIP 1

        match ip list RTP_port

        match dscp 46 ef

        set dscp 46

       

      interface gigabit-eth 0/1

        description UPLINK

        ip address  xxx.xxx.218.135  255.255.255.192

        ip ffe max-entries 500000

        ip access-policy Private

        no shutdown

        media-gateway ip primary

       

      interface gigabit-eth 0/2

        no ip address

        shutdown

       

      interface gigabit-eth 0/3

        description PBX

        ip address  192.168.150.1  255.255.255.252

        ip ffe max-entries 500000

        ip access-group SIP_CPE_ACL in

        qos-policy out VoIP

        no shutdown

        media-gateway ip primary

       

      ip access-list standard SIP_NET_ACL

        remark SIP Trunk to Voice Access Net

        permit host xxx.xxx.172.244

        permit host xxx.xxx.172.245

        permit host 192.168.150.2

       

      ip access-list standard VTY_SVC

        remark ACL for VTY Service on Router

        permit host xxx.xxx.247.133 log

        permit host xxx.xxx.247.18 log

        permit host xxx.xxx.247.4 log

       

      ip access-list extended RTP_port

        permit udp any range 1634 65535 any   

       

      ip access-list extended SIP_CPE_ACL

        remark SIP Trunk to CPE

        permit udp host 192.168.150.2 range 5060 5061 host 192.168.150.1   

        permit udp host 192.168.150.2 range 1634 32767 host 192.168.150.1   

       

      ip access-list extended SIP_port

        permit udp any  any eq 5060  

       

      ip policy-class Private

        allow list self self

        nat source list wizard-ics interface gigabit-ethernet 0/2 overload

        nat source list wizard-ics interface gigabit-ethernet 0/3 overload

       

      ip route 0.0.0.0 0.0.0.0 xxx.xxx.218.129

       

      no tftp server

      no tftp server overwrite

      no http server

      http secure-server

      no snmp agent

      no ip ftp server

      no ip scp server

      no ip sntp server

       

      auto-link

      auto-link server primary xxxxxx

      auto-link server secondary xxxxxx

       

      sip

      sip udp 5060

      no sip tcp

       

      voice feature-mode network

      voice forward-mode network

       

      voice spre 1 *[12345689]X&

      voice spre 2 *78&

      voice spre 3 *77

      voice spre 4 *75XX&

      voice spre 5 *73

      voice spre 6 *72&

      voice spre 7 *70

      voice spre 8 *67&

      voice spre 9 *55

       

      voice dial-plan 1 international 011-N$

      voice dial-plan 2 local M11

      voice dial-plan 3 local NXX-NXX-XXXX

      voice dial-plan 4 local [01]911

      voice dial-plan 5 local [01][235678]11

      voice dial-plan 6 long-distance 1-NXX-NXX-XXXX

      voice dial-plan 7 operator-assisted 00

      voice dial-plan 8 operator-assisted 0[23456789]1[023456789]XXXXXXX

      voice dial-plan 9 operator-assisted 0[23456789][023456789]XXXXXXXX

      voice dial-plan 10 operator-assisted [01]411

      voice dial-plan 11 specify-carrier 101XXXX$

       

      voice codec-list Default_List

        codec g711ulaw

        codec g729

       

      voice trunk-list CPE_TRK

        trunk T30

       

      voice trunk-list NET_TRK

        trunk T10

       

      voice trunk T10 type sip

        description "Trunk to Voice Access Network SIP_245"

        sip-server primary 10.195.1.4

        outbound-proxy primary xxx.xxx.172.245

        dial-string source to

        hmr SIP_META out

        register xxxxxx auth-name "xxxxxx" password encrypted "xxxxxx"

        codec-list Default_List both

        authentication username "xxxxxx" password encrypted "xxxxxx"

       

      voice trunk T30 type sip

        description "Trunk to CPE"

        sip-server primary 192.168.150.2

        grammar from host local

        transfer-mode network

        sip-server monitor

          no shutdown

       

      voice grouped-trunk NET

        description "Route inside to outside"

        trunk T10

        accept $ cost 0

        permit list CPE_TRK

        !deny all other trunks

        !deny all other ani

       

      voice grouped-trunk CPE

        description "Route outside to inside"

        trunk T30

        accept $ cost 0

        permit list NET_TRK

        !deny all other trunks

        !deny all other ani

       

      sip access-class ip "SIP_NET_ACL" in

      sip qos dscp 24

       

      ip rtp media-anchoring

       

      hmr policy SIP_META

        rule-set META_AUTH 10

       

      hmr rule-set META_AUTH

        message-rule CONTACT_HEADER message-type any 10

          modify header contact position first match-value /sip:.*@/ new-value /sip:xxxxxx@/  10

        message-rule INVITE message-type request 20

          set private-variable INVITE_PAI  header sip-req-uri position first match-value "/^INVITE /" new-value true  10

        message-rule ADD_PAI message-type request 30

          match private-variable INVITE_PAI match-value true

          add header p-asserted-identity position first new-value /<sip:xxxxxx@10.195.1.4>/  10

       

      line con 0

        no login

       

      line telnet 0 4

        login local-userlist

        shutdown

        ip access-class VTY_SVC in

      line ssh 0 4

        login local-userlist

        no shutdown

        ip access-class VTY_SVC in

       

      ntp server xxxxxx

      ntp server xxxxxx

       

       

       

       

       

       

      0Idle0R2,593,785,152086786.78
      1init0W1,600,000000
      2kthreadd0W0000
      3ksoftirqd/00W190,000000
      4kworker/0:00W0000
      5kworker/u:00W10,000000
      6rcu_kthread99W16,930,000000
      7khelper0W0000
      8kworker/u:10W0000
      85sync_supers0W200,000000
      87bdi-default0W0000
      89kblockd0W0000
      184rpciod0W0000
      185kworker/0:10W170,000000
      209khungtaskd0W10,000000
      214kswapd00W0000
      278fsnotify_mark0W0000
      284nfsiod0W0000
      292crypto0W0000
      375mtdblock00W0000
      380mtdblock10W0000
      385mtdblock20W0000
      390mtdblock30W0000
      395mtdblock40W0000
      400mtdblock50W0000
      405mtdblock60W0000
      410mtdblock70W0000
      432edac-poller0W0000
      439rcS0W10,000000
      470ubi_bgt0d0W0000
      479ubi_bgt1d0W20,000000
      486ubifs_bgt0_00W0000
      488ubifs_bgt1_00W0000
      605runstarter0W0000
      611syslogd0W0000
      617amom0W2,020,000000
      618AIPC Session Th0W0000
      619AMOM network th0W3,450,000000
      632khubd0W0000
      751starter0W0000
      753Init1W420,000000
      755InterruptThread52W0000
      756KCall49W2,874,060,0000414.37
      757PC Config8R694,680,0000151.14
      758PacketRouting45W35,940,000000
      759Timers47W211,220,000000
      760I2C47W0000
      761CallControlQue~38W5,120,000000
      762Thread Pool5W50,000000
      763FrontPanel44W44,530,000050.51
      764RSTP44W0000
      765sec47W1,125,030,0000151.14
      766con033W0000
      767gigTSEC47W1,037,060,000000
      768ICP Session9W110,000000
      769RSTP44W16,430,000000
      770RSTP-BG43W0000
      771MLD Thread7W0000
      772RouteTableTick7W2,900,000000
      773RouteTableTick7W2,880,000000
      774IGMPTick7W2,400,000000
      775IGMP-Receiver7W0000
      776IP Events28W3,710,000000
      777tcptimer26W530,000000
      778tcpinp26W6,030,000000
      779tcpout26W5,310,000000
      780DnsClient20W2,020,000000
      781DnsProxy20W300,000000
      782DnsTable20W140,000000
      783PhoneManagerQu~42W6,090,000000
      784SnmpThread7R47,460,000000
      785WWW23W58,980,000000
      786MediaConnectio~40W10,870,000000
      787FTPServer List~6W0000
      788SMTP Client20W0000
      789SNTP Client23W0000
      790CPU Usage9R2,514,200,0000353.33
      791CLIInjectQ7W0000
      793OSPF7W0000
      795RipOut7W1,070,000000
      796RipIn7W0000
      797AUTOLINKQ5W280,000000
      798HttpClientQ7W100,000000
      799SIP_Stack39W8,960,000000
      800SIP Registrati~37W17,630,000000
      801ntpd23W10,420,000000
      802ActiveQueueDea~4W60,000000
      803DHCPv635W0000
      804RvSipProc040W23,810,000000
      805UDP Relay23W0000
      806PacketCapture5W3,730,000000
      807DHCP Server35W0000
      808Flow Meter Log~21W2,100,000000
      809OSPFv37W0000
      810TWAMP-Control7W0000
      811TWAMP-Test20W0000
      812UDP In43W2,850,000000
        • Re: NV3140 High cpu usage when used for a sip trunk
          jayh Hall_of_Fame

          The CPU is over 86% idle in your example, this doesn't seem like excessive CPU usage. Note that "idle" in the listing is not usage, it's actually the percentage of non-usage.

           

          Fax can be made quite reliable on most networks with the proper tweaks, typically T.38 will help but it needs to be supported by the other endpoint as well.

          • Re: NV3140 High cpu usage when used for a sip trunk
            brian.carnett New Member

            I was able to reproduce 100% cpu just by sending 2-3 ssh session attempts at once.  Even with mgmt heavily restricted it looked like general port scanning and attempts were causing the high cpu.

             

            We replaced the 3140 with another 3140 using the exact same config.  Even trying  with dozens of login attempts the highest I can get it is 86%.  While that still seems high it should not be an issue for the customer.  Right now I am hoping it was just a bad CPU/board.

             

            Another 3140 that I am using for SIP to PRI with almost the same config barely reacts to ssh attempts, cpu never going over 50%.  The difference in the two are the NAT rules since the SIP to PRI does not allow for any nat or data traffic from the lan.  We may have room for improvement in NAT and firewall rules.