11 Replies Latest reply on Mar 30, 2018 3:11 PM by jayh

    what are some common causes for cpu utilization to reach 100% on NV1544

    service1icn New Member

      Pings sent to NV1544 steady at 22ms then spike to 600-800ms during low traffic. CPU utilization also spikes to 100% at the same time.

        • Re: what are some common causes for cpu utilization to reach 100% on NV1544
          jayh Hall_of_Fame

          The command "show process cpu" should give some detail as to what is causing it.

           

          Is the switch reachable from the Internet either directly or by port translation behind a NAT? One cause of high CPU is bots scanning or trying to brute-force SSH passwords. Filtering management to trusted networks if the device is accessible from the Internet will fix this as well as prevent the bad guys from succeeding in guessing your credentials.

           

          Also, traffic to a switch or router is handled at a much lower priority than traffic through it, so unless it's an extreme case high CPU typically doesn't impact performance.

          • Re: what are some common causes for cpu utilization to reach 100% on NV1544
            jroad Employee

            High CPU can be from many things.  The enable command "sh proc cpu" will show you what processes are running and what is causing the CPU to run high.

            Using the web interface and any other management access will show up as "PC Config" and it is normal for this to spike to 100% when you are pulling up a new or updating a web page.

             

            You asked for common reasons -

            - SNMP requests that are not in the standard or ADTRAN MIBs.

            - Managment interface(s) being used

            - Multicast or broadcast network traffic

            - spanning tree is reconverging

             

            All Management access ( including ping of an interface ) is the lowest priority to the CPU, so dropped, missing, or even unresponsive pings is not unusual with high broadcast, multicast and even other management traffic.

            With network design, you should always be able to access the switch.

             

            Your network is running well if the average load on the CPU is lower than 30% most of the time., as shown by the enable command  "sh proc cpu his" .

            Hope this information helps -  Jay's comments are all true, here also :-)

              • Re: what are some common causes for cpu utilization to reach 100% on NV1544
                service1icn New Member

                show proc cpu his

                #: Average load % per interval

                @: Maximum load % per interval

                Most current interval starts on the left.

                 

                Previous 1 minute system load:

                100     # ##                                                   

                90     # ##                                                   

                80     # ##                                                   

                70     # ##                                                   

                60    #####                                                   

                50    #####                                                   

                40  #######                                                   

                30  #######                                                   

                20  #######                    ##                             

                10  #######################################  ##################

                  0 +++++++++1+++++++++2+++++++++3+++++++++4+++++++++5+++++++++6

                             0         0         0         0         0         0

                                              seconds

                 

                Previous 1 hour system load:

                100 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                90 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                80 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                70 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                60 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                50 #@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                40 #@@@@@@@#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                30 #@@@@@@@#@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

                20 ######################@######################@##############

                10 ############################################################

                  0 +++++++++1+++++++++2+++++++++3+++++++++4+++++++++5+++++++++6

                             0         0         0         0         0         0

                                              minutes

                 

                Previous 72 hours system load:

                100 @@@@@@@@@@@@@@@@@

                90 @@@@@@@@@@@@@@@@@

                80 @@@@@@@@@@@@@@@@@

                70 @@@@@@@@@@@@@@@@@

                60 @@@@@@@@@@@@@@@@@

                50 @@@@@@@@@@@@@@@@@

                40 @@@@@@@@@@@@@@@@@

                30 @@@@@@@@@@@@@@@@@

                20 ###############@#

                10 #################

                  0 +++++++++1+++++++++2+++++++++3+++++++++4+++++++++5+++++++++6+++++++++7++

                             0         0         0         0         0         0         0 

                                                     hours

                Can you help interpret this output?

                  • Re: what are some common causes for cpu utilization to reach 100% on NV1544
                    jroad Employee

                    This shows the CPU is running at 10 to 20 % average and something spiked it in the last minute.  "sh proc cpu" will show processes.

                      • Re: what are some common causes for cpu utilization to reach 100% on NV1544
                        service1icn New Member

                        show proc cpu

                        System load: 1sec:6.89%  1min:14.83%  5min:13.94%  Min: 0.00%  Max: 100.00%

                        Context switch load: 0.05%

                                                              Invoked  Exec Time    Runtime    Load %%

                        Task Id    Task Name        PRI STA   (count)     (usec)     (usec)     (1sec)

                        1          Idle               0 W    34709142       1745     934672      93.47

                        3          PC Config          7 S     5091181       4952      13596       1.36

                        4          PacketRouting     44 W     2654130        300       5101       0.51

                        5          Timer             46 W     6242199         15       1232       0.12

                        6          Thread Pool        4 W         246        239          0       0.00

                        7          FrontPanel        43 W     1253441         63       1058       0.11

                        8          con0              46 W      254434         10         39       0.00

                        9          Flash Maintena~    4 W        3809       2050          0       0.00

                        10         ICP Session        8 W        6561         10          0       0.00

                        11         RSTP              43 W     1405139        135       2640       0.26

                        12         RSTP-BG           42 W         466       9418          0       0.00

                        13         MLD Thread         6 W           0      41245          0       0.00

                        14         RouteTableTick     6 R      102462         77         77       0.01

                        15         RouteTableTick     6 R      100792         73         73       0.01

                        16         IP Events         27 W      119236         64         68       0.01

                        17         tcptimer          25 W      102071         12         68       0.01

                        18         tcpinp            25 W       98184        211       1554       0.16

                        19         tcpout            25 W      166667         28       1775       0.18

                        20         eth0Switch        46 W    27884263        126       3887       0.39

                        21         Stacking           9 W       68896         55         55       0.01

                        23         bcmDPC            40 W           0        380          0       0.00

                        24         BcmSDK Diag Sh~    5 R      591742          5         44       0.00

                        26         bcmL2X.0          40 W      438890       5942      18079       1.81

                        27         bcmCNTR.0         40 W     2276373        592       8440       0.84

                        28         bcmTX             40 W      378057         36        314       0.03

                        29         bcmXGS3AsyncTX    40 W           0        361          0       0.00

                        30         bcmLINK.0         40 W    17422336         16       3680       0.37

                        31         bcmRX             11 W     1411449         55       2460       0.25

                        32         SwitchQ           18 W    10214539         30       3796       0.38

                        33         Igmp Reswitch      6 W          93         39          0       0.00

                        34         DhcpSnooping      35 W           0        162          0       0.00

                        35         SnmpThread         6 W     2947633         14        531       0.05

                        36         DnsClient         19 W       32789         80         80       0.01

                        37         DnsProxy          19 W       12827         19          0       0.00

                        38         DnsTable          19 W       12783          6          0       0.00

                        39         WWW               22 W     1926226        112          0       0.00

                        40         FTPServer List~    5 W          13          7          0       0.00

                        41         SMTP Client       19 W           0        258          0       0.00

                        42         SNTP Client       22 W           0        385          0       0.00

                        43         CLIInjectQ         6 W           0       1926          0       0.00

                        45         OSPF               6 W           1         14          0       0.00

                        46         RipOut             6 W       61630         16         15       0.00

                        47         RipIn              6 W           0      42246          0       0.00

                        48         Fan Control        8 W      416297         24        187       0.02

                        50         AUTOLINKQ          4 W         785         61          0       0.00

                        51         HttpClientQ        6 W           0        381          0       0.00

                        52         UDP In            42 W       12164         95          0       0.00

                        53         UDP Relay         22 W           1        179          0       0.00

                        54         DHCP Server       34 W           0        342          0       0.00

                         

                        Not much going on here.

                  • Re: what are some common causes for cpu utilization to reach 100% on NV1544
                    james-in-ca New Member

                    I had this happen in my 3120. Help here alerted me to IP FLow/Monitoring causing this.

                    3120 - High CPU Utilization and slowness