M18 Policy Tuning


Upon successful completion of this module you will be able to: • Summarize tuning and its purpose • Discuss what is “noise” in the network • Give example of how to identify and prevent false positives • Explain why you should look at high-volume attacks • Configure NSP to prevent false positives


The policies applied to the Sensors determine what traffic analysis they will perform. Policy tuning is renowned to be an involved task. However, because networks and attacks constantly evolve, the policy tuning process is never truly complete. The ultimate goal of policy tuning is to eliminate false positives and noise, and avoid overwhelming quantities of legitimate, but anticipated alerts. Some things to remember when tuning policies: • When initially deployed, McAfee Network Security frequently exposes unexpected conditions in

the existing network and application configuration. What may at first seem like a false positive might actually be the manifestation of a misconfigured router or Web application.

• Take steps to reduce false positives and noise from the start. If you allow a large number of “noisy” alerts to continue to sound on a very busy network, parsing and pruning the database can quickly become cumbersome tasks. It is preferable to all parties involved to put energy into preventing false positives than into working around them. One method may be is to disable all alerts that are obviously not applicable to the hosts that are being protected. For example, if only using Apache Web servers, it’s preferable to disable IIS-related attacks.


There are great advantages for effective policy tuning. An appropriately tuned NSP deployment will not only reduce false positives, but will also: Improve Security • Speed (Want Maximum IPS performance) • Fuel efficiency (Want to reduce Analyst expenses) • No accidents (Want to harden the policy for maximum protection) • Maintenance (Monitor Constantly)

Improve Scaling • McAfee Network Security Manager (NSM) can handle up to 150 alerts per second. Each alert is

~650 bytes. If sustaining alerts at that rate, then it equates to a bandwidth rate of 780 Kbps resulting in 67.392 Gigabytes (GB) of alerts space per day!

• A tuned NSP deployment can now manage more sensors across an organization, and it also saves precious network bandwidth and storage.

Help Engine Performance • The bulk of IPS engine resources are consumed in protocol parsing. This entails doing a deep

packet inspection, protocol anomaly detection, parsing protocol content against respective vulnerability signatures etc. Therefore the greater the number of unwanted attack signatures that are included, the more bulky the IPS engine becomes. It also increases the possibility of false positives.

• A customized and relevant IPS policy is crucial to providing best protection.


The initial tuning of Network Security Platform is very important in establishing the solution’s credibility and usability. Though each network environment will have its own characteristics, there are some best practice steps that will make tuning more efficient and effective. Setting expectations about tuning is very important. A proper McAfee Network Security implementation includes multiple tuning phases. False positives and excess noise are generally routine for the first 3 to 4 weeks. Once properly tuned, however, they can be reduced to a rare occurrence. Attack definitions are written to accommodate many different types of customers. Therefore, they will not be perfect in every environment. Before beginning, be aware of the network topology and the hosts in the network, so you can enable the policy to detect the correct set of attacks for the environment. There is a difference between “evaluation tuning” and tuning for the production environment. An initial reaction is to install the Network Security Platform into the busiest part of the network and just see what it is detecting. However, you can minimize the need for tuning by focusing the IPS policy on a hot spot, for example, Web Server attacks. Then determine sources of devices in the network that would be known false positive offenders–SNMP Managers, Vulnerability Scanners, and so forth. You can further tune by having Wireshark (formerly Ethereal) installed and ready on a client PC.


There are 2 phases to IPS tuning. Generally, one is owned by the security architecture team, and the other is owned by security operations team. The first phase is relevant from security design, where you make sure that the IPS sensors are placed in the right locations, and are protecting with the correct configured policies. If there are Linux servers behind a VIPS and you have not enabled the Linux OS attacks in the policy, then IPS is not protecting the Linux servers. Similarly, if you have placed the IPS with just one VIPS and all default policy whereas the network behind is far more diverse, then the IPS will not be able to do its job efficiently or effectively. It is important that customers review their IPS security architecture before going to the second phase. The second phase involves the day-to-day operations of IPS alerts. If receiving 100,000 alerts per day, of course it is going to take a bit of time of the security analysts to review those. And if receiving those repetitively, then there is something needing to be reviewed. If those 100,000 are all attacks, then of course you need to block the source of attacks and find a way to reduce the attack reporting. But if a bulk of those 100,000 are not attacks, then you need to ensure that they do not show up every day. From experience, 70% or more of those repeating alerts can be eliminated, without impacting security.


The mere mention of false positives always causes concern in the mind of any security analyst. However, false positives may mean quite different things to different people. Incorrect identification These alerts typically result from overly aggressive signature design, special characteristics of the user environment, or system bugs. For example, typical users will never use nested file folders with a path more than 256 characters long; however, a particular user may push the Windows' free-style naming to the extreme and create files with path names more than 1024 characters. Issues in this category are rare. They can be fixed by signature modifications or software bug fixes. Correct identification; significance subject to usage policy Events of this type include those alerting on activities associated with Instant Messaging (IM), Internet Relay chat (IRC), and Peer to Peer programs (P2P). Some security policies forbid such traffic; for example, within a corporate common operation environment (COE); others may allow them to various degrees, such as Universities. McAfee Network Security provides two means by which to tune out such events: • customized policy in which these events are disabled. In doing so, the sensor will not even look for these

events. • create attack filters to suppress alerts when a few hosts are involved.

Correct identification; significance subject to user sensitivity (also known as noise) McAfee Network Security will detect a UDP-based host sweep when a given host sends UDP packets to a certain number of distinct destinations within a given time interval. Some users will consider alerts from this type of event as noise; however, others will take notice because it indicates possible reconnaissance activity.


False positives tend to result in a high attack count. This is because the traffic creating the false positive is usually commonplace on the particular network. The approach to identifying and removing false positives therefore starts with identifying the attacks with the highest attack counts. Look for a pattern within each attack. For example, if you see that the attack is originating from the same group of sources and is taking place at a regular interval, there is a pretty good chance the alert is a false alarm. If those source hosts happen to be internal, you can often figure out what service they have in common by having a closer look at a subset of the group. You can then temporarily shut down services on those hosts until the culprit service is isolated. Be sure to leverage the knowledge of someone who is very familiar with the network. He or she may recognize a source or destination address as belonging to a host with a special function. Also consult attack descriptions for clues; the attack in question might just have a high Benign Trigger Probability. If there is no obvious pattern or convincing attack information, take advantage of McAfee Network Security's options to capture additional packets. Specifically, temporarily increase the quantity of traffic captured for the alert in question. The customer can use the capture to analyze the attack. If still not able to isolate the pattern, the customer should save the capture file as part of an evidence report.


Enterprise HTTP caching servers, DNS servers, and even outbound mail relays, are susceptible to false positives. These types of hosts are constantly receiving requests from many different clients and, in turn, making requests to many different hosts to service those clients. Vulnerability assessment software will also generate false positives as it scans the network. Check the source and destination addresses within the attack details for these known internal hosts. If you find such a false positive, create an exception or a firewall rules for that host. Once it is determined that an alert is indeed a false positive, take measures to “reduce the noise.” The most secure way is to create an exception for the hosts and attacks in question. Of course, this assumes the hosts are consistent and few in number. If there are too many hosts or the range of IP addresses involved is too unpredictable, the options are to live with the false positives or disable the alert (possibly until McAfee has produced an updated signature set). Most folks disable the alert. One of the options in Manager is to create Firewall Rules. Customers can configure an firewall rules to process specific traffic without sending it through the IDS engine. So if there is a need to not process traffic to or from a given host or range of hosts, Firewall rules make it easy to do so.


The Rule Set Editor action enables the use of a powerful tool for defining the exact environment resources needing protection. A Rule Set consists of select attacks specific to a network environment, such as the operating system(s) employed, the installed applications (email, chat), and the transport and application protocols (HTTP, FTP) used for data delivery. Within a Rule Set are Include/Exclude rules that in turn contain one or multiple rules. The rules are based upon selected attacks, and they are filtered through seven different views: Categories, Protocols, OS, Applications, Severity, Benign Trigger probability and Attack. McAfee Network Security Manager provides pre-defined rule sets to enable customers jump start with the right policy. Each rule set may have include rules, exclude rules or both. The Include/Exclude rules contain one or multiple rules and the rules can be based upon attacks NSP will analyze for in the data traversing the network.


The single most effective way to prevent false positives is to apply a scanning policy that is specific to the network at hand. It is therefore imperative to understand the details of the protocols and applications used on the network needing protection. For example, if a sensor port is connected to a network containing only Windows workstations, there is no reason to use a policy that looks for attacks against UNIX hosts, Web servers, and so on. Once the network configuration has been identified, use an applicable default policy or create a custom one. Also consider virtualization of the IPS to refine scanning by VLAN tag or CIDR address. If a DMZ contains isolated networks of Web and mail servers, for example, you can prevent false positives by creating sub-interfaces and applying policies specific to Web and mail servers, respectively. The noise-to-incorrect-identification ratio can be fairly high, particularly in the following conditions: • The configured policy includes a lot of Informational alerts, or scan alerts which are based on

request activities (such as the All Inclusive policy) • Deployment links where there is a lot of hostile traffic, such as in front of a firewall • Overly coarse traffic virtual interface definitions that contains very disparate applications, e.g. a

highly aggregated link in dedicated interface mode Users can effectively manage the noise level by defining appropriate virtual interfaces and customize the policy accordingly.


It’s best to begin the tuning process by identifying high-volume attacks. To do this, use the Top10 table in the Consolidated View within Threat Analyzer to view the attacks that are generating the most alerts within the network. You will want to investigate the legitimacy of these alerts. The investigation will involve use of packet captures to decode these attacks. The Wireshark decoding function is used at this stage to discover the packet level characteristics of a data flow that triggered a particular alert. At this point it usually becomes evident whether the alert is generated by an attack or by a false positive. Many of the top alerts that are seen on the initial deployment of sensors will be common false positives seen in many environments. As a result, this initial tuning of high volume alerts will move along quickly. Typically, at the beginning of the tuning process, it will be evident which network or security policy will affect the overall level of alerts. If, for instance, IM is allowed traffic on the network then there might not be a need to alert on IM set-up flows.


As in the case of a 1 to Many, if we are seeing one alert, then an exception should be made. If we are seeing a multitude of alerts then it’s advised to create a firewall rule. NOTE: The create exception wizard can create both a firewall rule or an exception.


Once it’s determine that an alert is indeed a false positive, it’s important to take measures to "reduce the noise.“ When a particular alert is declared a false positive, the next decision is whether to disable this alert altogether or apply a particular exception to that attack that will disable alerting for a particular IP address or range of IP addresses. In almost all cases it is best practice to implement the latter. For instance, an SMS server may be generating the alert ‘Netbios: Copy Executable file attempt’ during the legitimate transfer of login scripts. Rather than disable the alert altogether, and cancel the possibility of finding a real attack of this nature, it is recommended that an exception be created for the SMS server and applied to this attack. Every exception created is globally stored so that the exception can be applied to any Exploit or Reconnaissance attack chosen. It is also best practice to document all tuning activities. Configuration reports for exception objects can be used to assist the documentation process. These reports can be used to list Exceptions that have been applied and attacks that have been otherwise customized.


By choosing to create an exception for an alert, the user is presented with Create Exception wizard. Here you can create an exception for a specific attack that will be applied to all devices in the admin domain, a particular Sensor, or interface.


However, the wizard can also disable the attack by setting the exception for any combination of source and destination. By doing so, the selected attack definition will therefore be disabled.


The wizard can also create a statefull firewall rule. Selecting Attack Name as Any, will create an ignore rule that will be added to the top of the firewall policy currently assigned to the device identified.


The scope of the exception can be restricted to an device, an interface or to the admin domain as a whole.


You can also, optionally, select to acknowledge or delete the alerts that match the settings specified.


Another method of preventing false positives is to disable attacks. For example, you can edit the policy to disable all attacks that have a severity level of low or informational (lower than 4 ). To do this, you can sort the attacks by severity and then bulk disable all Low and Informational attacks. For these alerts, you can choose to uncheck the option “Enable Attack” You can do this for both Inbound and Outbound attacks. It’s best practice to disable all alerts that are obviously not applicable to the hosts being protected. For example, if only using Apache Web servers, you can safely disable IIS-related attacks. Adding low severity attacks Once you have eliminated the false positives with only medium and high severity attacks enabled, you can come back and re-enable low and informational attack alerts. Low and informational severity attacks tend to generate more alerts than higher severity attacks, so expect another round of false positives after enabling low and informational severity attacks.


Causes of Excessive Alerts The large set of alerts can be categorized into 3 categories.

• Valid Alerts: In this case, the alerts are pertinent to your network and taken as attacks. The impact of the attack may be critical, or moderate, or none. In this scenario, you’ve a challenge if you see 1000’s of alerts being generated for the same attack, yet analyzing one alert is same as analyzing all those 1000’s alerts. Then it becomes important how to design the configuration so that it reduces the number of alerts, without impacting security. Alert Throttling feature can help in this situation.

• Irrelevant Alerts: By default NSP will raise alerts when it detects P2P traffics. Now that may be relevant for some administrators, and may not be relevant for some. It depends on the company policy of specific customer. Another scenario is where you see 100’s of UDP sweeps or probes. NSP, by default, has certain thresholds defined for detecting such reconnaissance attacks. Those thresholds may need to be changed for your network scenario. Alternately, a massive set of alerts are being generated from a IT approved scanner, then you need to white-list the scanner or use granular set of attack filters.

• False Positives: These refer to incorrectly identified events by the IPS. An example is a Vendor signature alerts an attack, where the attack was not there at all. This signifies an error in vendor signature. This is actually false positive. False Positives are error on part of a Vendor IPS inspection. In contrast, False negative is when a vendor protection delivered for an attack is completely missing to detect the attack.


A high level bottoms up approach to tuning can be summed up as follows:

1. Access Threat Analyzer 2. Identify the noisiest event 3. Investigate the event and correlate with network resources 4. Understand the characteristics of the event 5. Determine the criticality of the event and of the components involved 6. Decide on an action to remediate/tune the event 7. Depending on action to be taken, proceed with the proper response procedures

8. Repeat


Analyzing events can be done in two ways, either by starting on the Threat Explorer and using the filtering capabilities, or by jumping straight into the Threat Analyzer and sorting events from there. As shown in the image, the Threat Analyzer IPS tab gives a quick glimpse of attacks over time, the top hosts under attack, and it allows you to drill down using various alert properties.


Using the Threat Analyzer Alert page, customers can display alerts sorted by Attack name. As you can see, in this image, there are a large number of HTTP: Gomplayer ActiveX Control OpenURL Overflow alerts. This is a good starting point to investigating the relevancy and applicability of the highest volume alerts.


In this next section we will look at some examples and use cases of typical high volume alerts. This example is dealing with a high volume of “ICMP unsolicited Echo Reply” alerts.


This example is dealing with a high volume of ”DNS: Microsoft DNS Services Resolver Overflow” alerts.


This example is dealing with a high volume of “ICMP: Nachi Like Ping” alerts.


This example is dealing with a high volume of “Backdoor: Back Orifice Trojan” alerts.


This example is dealing with a high volume of “HTTP: HTTP Login Bruteforce Detected” alerts.


This example is dealing with a high volume of “ARP: ARP Spoofing Detected” alerts.


This example is dealing with a high volume of “ARP: MAC Address Flip-Flop” alerts.


This example is dealing with a high volume of “P2P: BitTorrent Meta-Info Retrieving” alerts.


This example is dealing with a high volume of “IM: Yahoo Messenger Server Lookup” alerts.


This example is dealing with a high volume of “UDP: Host Sweep” alerts.

M18 Policy Tuning

Documents

Transcript of M18 Policy Tuning