This document describes the steps to take after an ADTRAN Bluesocket deployment to troubleshoot and resolve RF issues and to maintain a wireless network functioning at its maximum efficiency. Wireless interference can be detrimental to proper wireless functionality. Regular monitoring, troubleshooting, and maintenance is required to ensure optimum user experience.
To ensure this document is properly understood, consult Avoiding RF Interference with a Successful BlueSocket Wireless Deployment prior to continuing for a technology overview and audit of your wireless design. If your wireless design does not follow the rules mentioned in that document, steps should be taken to adjust the network accordingly. Alternatively a site survey can be utilized to suggest changes to optimize the current network.
Sections Included in this Document
The vWLAN methods used in this document to troubleshoot and avoid interference may not be present in prior versions of firmware. It is recommended you are on the most recent version of vWLAN and BSAP firmware. All screenshots and examples in this document are provided from vWLAN version 2.4.0-12.
vWLAN possesses many tools that help in troubleshooting and avoiding interference. The "Adjacent AP's" table can help a user understand power settings and co-channel interference as well as look at non-BSAP neighboring APs and their settings. The "Rogue AP", Interference, and WIDS alerts can help a user react properly when a new possible interference source is detected. All of these a vital to properly creating and maintaining a stable wireless network.
- Adjacent AP's Table
In vWLAN, the Adjacent AP's table can be found under Status->Adjacent APs. This table catalogs all the RF data sent from the APs to vWLAN and allows the user to look back through current and historical data to troubleshoot RF issues and get a closer feel for the layout of the network. The table is organized by each particular SSID seen by an AP while also recording the source MAC broadcasting it, channel information, Signal level in dBm, the AP that saw the SSID broadcast (Sensor Name), and the time this particular SSID was last detected. A picture of a populated Adjacent AP table is shown in figure 1 below:
The information in this section can assist in troubleshooting RF issues. When a user experiences a wireless problem, the user may describe the issue as "slow connection", "connection issues", or "low signal". In a lot of cases these issues can all be attributed to an RF problem. After finding the particular AP the user may be connected to, the Adjacent APs page can be searched using the particular AP name. The result shows all of the AP signals the particular AP senses in its surroundings along with the specific information from each adjacent AP. There are two important pieces of data to correlate together from this result: The last seen time and the signal level. In general, because networks are filled with many mobile devices that themselves can act as APs, many adjacent APs may show in the table, but only a single time or at a very low signal. Using the Last Seen column, you can click to sort and see the signals that were most recently observed. These can be monitored to see if they are recurring periodically meaning it is a constant broadcast rather than a passer-by.The signal reading helps a user see what signals are strong enough to actually cause interference. ADTRAN recommends to look specifically for signals that are greater than -60dBm as these are the signals that will noticeably interfere with a given AP's signal. At this point, decisions must be made whether an investigation into the source is needed. For example, the channel or power settings in this area may need to be adjusted.
The Adjacent AP's data also includes detected BSAP's on the same vWLAN domain meaning that AP's near to each other in your wireless deployment will sense each other (These will show up with a mac address that will need to be cross-referenced with the Status-APs page. Use the first 14 characters, excluding the last two, of the mac address to match these to an AP name). This will help verify settings on the AP's or make decisions about power and channel changes. For example, when searching for a specific AP an administrator sees that it is sensing another BSAP on the same channel with a signal of -55 dBm. At this point, co-channel interference could cause problems in the wireless experience. Looking at this data, the user can make decisions to either change the channels of the two APs or to lower the power settings on either of the APs to reduce the overlap in coverage. Anything detected in the Status-WIDS Alerts section as a rogue AP will show up in this section as well. When a WIDS alert is received about a rogue AP, the Adjacent AP table can be used to see the power setting and possible changes required to work around the issue.
- Rogue AP's and WIDS Alerts
Wireless IDS (Intrusion Detection System) Alerts can be found inside vWLAN by going to Status->WIDS Alerts. This is a list of RF alarms that have been triggered while the APs are scanning the RF environment. The most commonly reported problems are rogue APs. Rogue APs are devices that are not controlled by the vWLAN wireless system but are sensed to be nearby and interfering with signals. WIDS Alerts and data are very important to any wireless network because so many different devices can become mobile AP's and cause a disruption in the wireless network. Similarly, an employee could bring a wireless AP from home and plug it into their network jack in their office for personal wireless which could interfere with the business wireless network. This data can help troubleshoot wireless issues just like the Adjacent APs data, but with a focus on Rogue AP's specifically.
As with the Adjacent AP's table, when a user is having issues in a particular area or with a particular AP, the AP name(s) can be searched on this page using the search bar on the top right to filter down to the networks that particular AP has seen and when they were last seen. Correlating this data with the signal data in the Adjacent APs page can help determine if this threat is enough to attribute to the current wireless problems, as well as warrant investigation into finding the rogue AP and disabling it.
The Wireless IDS Alerts section can also record many other RF alarms and attacks that could also contribute to wireless issues in a network besides a rogue AP. For more information on configuring the Wireless IDS alerts, please see page 197 of the Administrator's Guide vWLAN Version 2.2.1 and Later.Be aware, this data should not only be used to troubleshoot a wireless issue voiced by a user. The best way to prevent RF wireless issues is to proactively monitor and react to possible changes in the RF environment. vWLAN provides several alerts that can be setup to notify a vWLAN administrator that a new network has been detected as well as interference from this network. By going to Configuration->Notifications->Info Messages in vWLAN, specifically rf_alarm_detected and rf_radio_interfence should be configured to notify an administrator that a rogue AP or RF alarm has been detected so that it can be investigated. Failing to act on these alarms quickly can cause future wireless issues. If disruptive changes in the RF environment are left un-investigated, the wireless network can become degraded to the point that users are denied service. Being proactive is the only way to ensure the required up time of a wireless network
Being proactive is the key to solving interference issues in a wireless environment. RF is not visible to the human eye, and not always resolved with a simple configuration change. To properly maintain a wireless network, an administrator will need an on-site presence with the ability to detect, monitor, and react to wireless issues and RF changes. Not all RF issues can be resolved - for instance the installation of a neighboring business' wireless system on improper channels - but adjustments can always be made to improve the situation.
- Adjusting Power Settings and AP Distances
Generally it will be simpler to change an APs power settings than the physically move the AP making a site survey and verification that much more important. However, there are many cases in normal operation where power settings can be very helpful. In the case of co-channel interference between two APs in a deployment, power settings can mitigate these problems. Even in a proper channel plan, APs on the same channel may even be able to "hear" each other signal. In this case, you would see these APs in the Adjacency table alerting you to the problem. Lessening the power settings on a particular AP, or both the interfering APs, can keep the signals from reaching the opposite AP meaning the APs will not consider the medium busy when the other is transmitting. This will provide for better bandwidth and client experience in the area. In the same manner, APs that do not seem to be adequately covering the area between them on different channels can be changed to transmit at a higher power strengthening the signal in those areas. This is common when users notice gaps in coverage when walking through a certain area with mobile devices.
Unlike physical AP distance, power settings should be constantly monitored and used to troubleshoot and resolve wireless issues. It may seem as though depending on a power setting, a signal may always transmit the same distance at the same signal. However, seemingly harmless changes to the environment can trigger a change in how RF signal propagates. Even the addition of something like a large filing cabinet can change the way RF signal propagates and reflects in an environment. Power settings will very rarely be the same over a deployment.The best way to verify power settings is by using a Spectrum Analyser. Remember that power settings will directly change the signal client devices are receiving so changes will affect production if done during business hours.
While physical AP distance and placement is generally not changed after deployment, it is important to be ready and have a process for doing this if needed. If the environment around an AP changes in any significant way (consider the changing of the ceiling tiles in a deployment of APs mounted above the tiles) the APs may need to be physically moved to accommodate the change for the health of the wireless network. The wireless network should be considered before any major physical building changes are implemented to limit effect to a stable wireless network beforehand.
- Limiting Rogue APs and Responding to WIDS Alerts
In most cases, completely preventing and eliminating rogue APs (APs that are broadcasting in the same wireless space as your business that you do not control) is impossible. Most every phone, tablet, or computer on the market can act as a WiFi host spot, Ad-Hoc (networks created by two or more devices wireless connecting directly to each other) networks can be set up, and nearby businesses with mis-configured AP's can become rogue AP's that directly affect wireless coverage and vitality in a network. A user could even plug a wireless AP into an exposed Ethernet port and set up their own wireless network.
To combat this, there are four major areas that need to be addressed:
- Wireless Network Policy
- Inside your business, Ad-Hoc networks, WiFi hot spots generated by any device, and personal APs should be disallowed.
- Employee Education
- All employees should be aware of wireless network policy and consequences of violating the policy
- Employees should be taught a basic understanding of wireless connectivity and what actions can cause network problems to arise and why.
- Device and Network Configuration
- All end-user devices, PCs, etc. should be configured with restrictive policies that disallow the use of Ad-Hoc and HostSpot software or addition of hardware that allows for this (i.e. USB WiFi dongle)
- Exposed Switch Ports should be configured with Port Security (For ADTRAN Devices please see Configuring Port Access Control in AOS) so that multiple MAC addresses can not be learned on the port preventing a user from created a wireless network others can join.
- All APs should use secure settings including 802.1X with Radius or LDAP for employee login. Guest networks should be secured with a Captive Portal and Terms of Service.
- Wireless Network Analysis and Documented Steps to Address Security Issues
- Using the information in the Adjacent APs table and the Rogue APs and WIDS alert section, a wireless network administration should monitor the health of the network
- All possible threats should have a documented plan in place of how to deal with the issue. An example is shown below:
- A nearby business turns up an AP on channel 3 of the 2.4GHz radio (interfering with Channel 1 and 6).
- Step 1: Evaluate which APs this interferes with using the Adjacent APs section
- Step 2: Investigate and implement temporary channel changes to avoid the channel
- Example: Set interfering APs to channel 11 and/or lessen power settings so that they minimally overlap each other or the nearby AP
- Step 3: Contact the nearby business' IT department to make them aware of the issue.
- Step 4: Once the business resolves the issue, return APs to original settings.
- step 5: Verify normal functionality has returned.
- A nearby business turns up an AP on channel 3 of the 2.4GHz radio (interfering with Channel 1 and 6).
- Find and eliminate Rogue APs in your infrastructure using a Spectrum Analyzer
Failing to properly plan for and handle Rogue AP's and WIDS alerts can lead to poor user experience, wireless network failure, and in even much more dire circumstances, critical business data could be compromised. Wireless is not inherently a secure protocol (though proper security settings can make it very secure) because when broadcasting, any listener on the same frequency channel can see the information with a Packet capture/Sniffer. Vigilant analysis and planning along with proper policy and security settings are the only ways to ensure security while providing the best user experience.
Wireless environments change continually, and these changes should be accounted for in the documentation. New devices get introduced to the network continually, causing different sources of interference. New additions to a building or obstructions in an environment can cause coverage areas to change. Neighboring companies may install new wireless systems or non-802.11 systems that operate in the same frequency ranges and cannot be controlled.
Imagine if a neighboring business deployed a new wireless solution, and, because their office building is connected, two of your nearby APs are now contending with the neighbor wireless network. In this case, a channel change might address this interference. If this change is not properly documented, in the future it could be viewed as a mistake in the design because it doesn’t match the original channel plan.
Besides deployment documentation, a wireless change log is a very vital part of wireless troubleshooting as well as keeping track of historical RF problems.
A Wireless Change Log is an historical, detailed report of changes made to the system. A wireless environment can change daily. While attempting to control a wireless environment is the best way to counter this, changes will inevitably have to be made to keep the network stable for users. When this happens, detailed information about the issue, why the change was made, and the date/time it was made should be logged in the change log. This serves several purposes. Specially, a change log will help when troubleshooting network problems later on by helping help you correlate changes or tendencies to specific network problems. Also, if any others are curious as to why these changes were made (for example, changing a channel in a channel plan because of interference) these will be justified by the documentation. A wireless change log can be set up in many ways, but an example of a simple Excel spreadsheet change log is provided below:
- For information regarding RF planning and design around avoiding RF interference, please see Avoiding RF Interference with a Successful BlueSocket Wireless Deployment
- For general information regarding configuring all of vWLAN, please see Administrator's Guide vWLAN Version 2.2.1 and Later.
- For general information around deployment configuration of vWLAN, please see the vWLAN General Deployment Guide and the vWLAN Quick Deployment and Configuration Guide.
- For information regarding site surveys and spectrum analyzers, please see Site Survey's and their place in Wireless Network Design and Maintenance.