Preprocessors were introduced in version 1.5 of Snort. They allow the functionality of Snort to be extended by allowing users and programmers to drop modular plugins into Snort fairly easily. Preprocessor code is run before the detection engine is called, but after the packet has been decoded. The packet can be modified or analyzed in an out-of-band manner using this mechanism.
Preprocessors are loaded and configured using the preprocessor keyword. The format of the preprocessor directive in the Snort rules file is:
preprocessor <name>: <options>
The Snort Portscan preprocessor is developed by Patrick Mullen.
The arguments to this module are:
Option | Description |
monitored network | The network/CIDR block to monitor for portscans. |
number of ports | The number of ports accessed during the detection period. |
detection period | The number of seconds to count that the port access threshold is considered for. |
logdir/filename | The directory/filename to place alerts in. Alerts are also written to the standard alert file. |
portscan: <monitored network> <number of ports> <detection period> <logdir/filename>
Portscan Ignorehosts is another module from Patrick Mullen that modifies the portscan detection system's operation. If you have servers that tend to trip the portscan detector (such as NTP, NFS, and DNS servers), you can tell portscan to ignore TCP Syn and UDP portscans from certain hosts. The arguments to this module are a list of IPs/CIDR blocks to be ignored.
portscan-ignorehosts: <host list>
The sfPortscan module, developed by Sourcefire, is designed to detect the first phase in a network attack: Reconnaissance. In the Reconnaissance phase, an attacker determines what types of network protocols or services a host supports. This is the traditional place where a portscan takes place. This phase assumes the attacking host has no prior knowledge of what protocols or services are supported by the target, otherwise this phase would not be necessary.
As the attacker has no beforehand knowledge of its intended target, most queries sent by the attacker will be negative (meaning that the service ports are closed). In the nature of legitimate network communications, negative responses from hosts are rare, and rarer still are multiple negative responses within a given amount of time. Our primary objective in detecting portscans is to detect and track these negative responses.
One of the most common portscanning tools in use today is Nmap. Nmap encompasses many, if not all, of the current portscanning techniques. sfPortscan was designed to be able to detect the different types of scans Nmap can produce.
sfPortscan will currently alert for the following types of Nmap scans:
These alerts are for oneone portscans, which are the traditional
types of scans; one host scans multiple ports on another host. Most of
the port queries will be negative, since most hosts have relatively
few services available.
sfPortscan also alerts for the following types of decoy portscans:
Decoy portscans are much like the Nmap portscans described above, only the attacker has spoofed source address inter-mixed with the real scanning address. This tactic helps hide the true identity of the attacker.
sfPortscan alerts for the following types of distributed portscans:
These are manyone portscans. Distributed portscans occur when
multiple hosts query one host for open services. This is used to evade
an IDS and obfuscate command and control hosts.
NOTE
sfPortscan alerts for the following types of portsweeps:
These alerts are for onemany portsweeps. One host scans a single port
on multiple hosts. This usually occurs when a new exploit comes out and the
attacker is looking for a specific service.
NOTE
sfPortscan alerts on the following filtered portscans and portsweeps:
``Filtered'' alerts indicate that there were no network errors (ICMP unreachables or TCP RSTs) or responses on closed ports have been suppressed. It's also a good indicator of whether the alert is just a very active legitimate host. Active hosts, such as NATs, can trigger these alerts because they can send out many connection attempts within a very small amount of time. A filtered alert may go off before responses from the remote hosts are received.
sfPortscan only generates one alert for each host pair in question during the time window (more on windows below). On TCP scan alerts, sfPortscan will also display any open ports that were scanned. On TCP sweep alerts however, sfPortscan will only track open ports after the alert has been triggered. Open port events are not individual alerts, but tags based off the orginal scan alert.
You may want to use the following line in your snort.conf to disable evasion alerts within stream4 because some scan packets can cause these alerts to be generated:
preprocessor stream4: disable_evasion_alerts
Use of the Flow preprocessor is required for sfPortscan. Flow gives portscan direction in the case of connectionless protocols like ICMP and UDP. You should enable the Flow preprocessor in your snort.conf by using the following:
preprocessor flow: stats_interval 0 hash 2
The parameters you can use to configure the portscan module are:
Available options:
Available options:
Available options:
Defines what IPs or networks to watch. IPs or networks not falling into this range are ignored.
Ignores the source of scan alerts.
Ignores the destination of scan alerts
This option will output portscan events to the file specified. If <file> does not contain a leading slash, this file will be placed in the Snort config dir.
preprocessor sfportscan: proto <protocols> \ scan_type <portscan|portsweep|decoy_portscan|distributed_portscan|all>\ sense_level <low|medium|high> watch_ip <IP or IP/CIDR> ignore_scanners <IP list>\ ignore_scanned <IP list> logfile <path and filename>
Unified Output
In order to get all the portscan information logged with the alert, snort generates a pseudo-packet and uses the payload portion to store the additional portscan information of priority count, connection count, IP count, port count, IP range, and port range. The characteristics of the packet are:
Src/Dst MAC Addr == MACDAD IP Protocol == 255 IP TTL == 0
Other than that, the packet looks like the IP portion of the packet that caused the portscan alert to be generated. This includes any IP options, etc. The payload and payload size of the packet is equal to the length of the additional portscan information that is logged. The size tends to be around 100 - 200 bytes.
Open port alerts differ from the other portscan alerts, because open port alerts utilize the tagged packet output system. This means that if an output system that doesn't print tagged packets is used, then the user won't see open port alerts. The open port information is stored in the IP payload and contains the port that is open.
The sfPortscan alert output was designed to work with unified packet logging, so it is possible to extend favorite snort GUIs to display portscan alerts and the additional information in the IP payload using the above packet characteristics.
Log File Output
Logfile output is displayed in the following format, and explained further below:
Time: 09/08-15:07:31.603880 event_id: 2 192.168.169.3 -> 192.168.169.5 (portscan) TCP Filtered Portscan Priority Count: 0 Connection Count: 200 IP Count: 2 Scanner IP Range: 192.168.169.3:192.168.169.4 Port/Proto Count: 200 Port/Proto Range: 20:47557
If there are open ports on the target, an additional tagged packet(s) will be appended:
Time: 09/08-15:07:31.603881 event_ref: 2 192.168.169.3 -> 192.168.169.5 (portscan) Open Port Open Port: 38458
These fields are used to link an alert with the corresponding Open Port tagged packet
Priority Count keeps track of bad responses (resets, unreachables). The higher the Priority Count, the more bad responses have been received.
Connection Count lists how many connections are active on the hosts (src or dst). This is accurate for connection-based protocols, and is more of an estimate for others. Whether or not a portscan was filtered is determined here. High connection count and low priority count would indicate filtered (no response received from target).
IP Count keeps track of the last IP to contact a host, and increments the count if the next IP is different. For one-to-one scans, this is a low number. For active hosts this number will be high regardless, and one-to-one scans may appear as a distributed scan.
This field changes depending on the type of alert. Portsweeps (one-to-many) scans display the scanned IP range; Portscans (one-to-one) display the scanner IP.
Port Count keeps track of the last port contacted and increments this number when that changes. We use this count (along with IP Count) to determine the difference between one-to-one portscans and one-to-one decoys.
The most important aspect in detecting portscans is tuning the detection engine for your network(s). Here are some tuning tips:
It's important to correctly set these options. The watch_ip option is easy to understand. The analyst should set this option to the list of Cidr blocks and IPs that they want to watch. If no watch_ip is defined, sfPortscan will watch all network traffic.
The ignore_scanners and ignore_scanned options come into play in weeding out legitimate hosts that are very active on your network. Some of the most common examples are NAT IPs, DNS cache servers, syslog servers, and nfs servers. sfPortscan may not generate false positives for these types of hosts, but be aware when first tuning sfPortscan for these IPs. Depending on the type of alert that the host generates, the analyst will know which to ignore it as. If the host is generating portsweep events, then add it to the ignore_scanners option. If the host is generating portscan alerts (and is the host that is being scanned), add it to the ignore_scanned option.
When determining false positives, the alert type is very important. Most of the false positives that sfPortscan may generate are of the filtered scan alert type. So be much more suspicious of filtered portscans. Many times this just indicates that a host was very active during the time period in question. If the host continually generates these types of alerts, add it to the ignore_scanners list or use a lower sensitivity level.
The portscan alert details are vital in determining the scope of a portscan and also the confidence of the portscan. In the future, we hope to automate much of this analysis in assigning a scope level and confidence level, but for now the user must manually do this. The easiest way to determine false positives is through simple ratio estimations. The following is a list of ratios to estimate and the associated values that indicate a legimite scan and not a false positive.
Connection Count / IP Count: This ratio indicates an estimated average of connections per IP. For portscans, this ratio should be high, the higher the better. For portsweeps, this ratio should be low.
Port Count / IP Count: This ratio indicates an estimated average of ports connected to per IP. For portscans, this ratio should be high and indicates that the scanned host's ports were connected to by fewer IPs. For portsweeps, this ratio should be low, indicating that the scanning host connected to few ports but on many hosts.
Connection Count / Port Count: This ratio indicates an estimated average of connections per port. For portscans, this ratio should be low. This indicates that each connection was to a different port. For portsweeps, this ratio should be high. This indicates that there were many connections to the same port.
The reason that Priority Count is not included, is because the priority count is included in the connection count and the above comparisons take that into consideration. The Priority Count play an important role in tuning because the higher the priority count the more likely it is a real portscan or portsweep (unless the host is firewalled).
If none of these other tuning techniques work or the analyst doesn't have the time for tuning, lower the sensitivity level. You get the best protection the higher the sensitivity level, but it's also important that the portscan detection engine generates alerts that the analyst will find informative. The low sensitivity level only generates alerts based on error responses. These responses indicate a portscan and the alerts generated by the low sensitivity level are highly accurate and require the least tuning. The low sensitivity level does not catch filtered scans, since these are more prone to false positives.
Frag2 is a new IP defragmentation preprocessor introduced in Snort 1.8 and is designed to replace the Defrag preprocessor. This defragmenter is designed to be memory efficient and use the same memory management routines that are in use in other parts of Snort.
Frag2 has configurable memory usage and fragment timeout options. Given no arguments, Frag2 uses the default memory limit of 4194304 bytes (4MB) and a timeout period of 60 seconds. The timeout period is used to determine a length of time after which an unassembled fragment should be discarded.
In Snort 1.8.7, several options were added to help catch the use of evasion techniques, such as fragroute.
preprocessor frag2: [memcap <xxx>], [timeout <xx>], [min_ttl <xx>], \ [detect_state_problems], [ttl_limit <xx>]
Option | Description |
---|---|
timeout <seconds> | Amount of time to keep an inactive stream in the state table; sessions that are flushed will automatically be picked up again if more activity is seen. The default value is 30 seconds. |
memcap <bytes> | Number of bytes to set the memory cap at; if this limit is exceeded, Frag2 will aggressively prune inactive reassemblers. The default value is 4MB. |
detect_state_problems | Turns on alerts for events such as overlapping fragments. |
min_ttl | Sets the minimum ttl that Frag2 will accept. |
ttl_limit | Sets the delta value that will set off an evasion alert (initial fragment ttl +/- ttl limit). |
The Stream4 module provides TCP stream reassembly and stateful analysis capabilities to Snort. Robust stream reassembly capabilities allow Snort to ignore ``stateless'' attacks (which include the types of attacks that Stick and Snot produce). Stream4 also gives large scale users the ability to track more than 256 simultaneous TCP streams. Stream4 should be able to scale to handle 32,768 simultaneous TCP connections in its default configuration.
Stream4 contains two configurable modules, the Stream4 preprocessor and the associated Stream4 reassemble plugin. The stream4_reassemble options are listed below.
NOTE
preprocessor stream4: [noinspect], [keepstats], [timeout <seconds>], \ [memcap <bytes>], [detect_scans], [detect_state_problems], \ [enforce_state], [disable_evasion_alerts], [ttl_limit <count>]
Option | Description |
---|---|
noinspect | Disables stateful inspection. |
keepstats | Records session summary information in ![]() ![]() |
timeout <seconds> | Amount of time to keep an inactive stream in the state table; sessions that are flushed will automatically be picked up again if more activity is seen. The default value is 30 seconds. |
memcap <bytes> | Number of bytes to set the memory cap at; if this limit is exceeded stream4 will aggressively prune inactive sessions. The default value is 8MB. |
detect_scans | Turns on alerts for portscan events. |
detect_state_problems | Turns on alerts for stream events of note, such as evasive RST packets, data on the SYN packet, and out of window sequence numbers. |
enforce_state | Enforces statefulness so that sessions aren't picked up mid-stream. |
disable_evasion_alerts | Turns off alerts for events such as TCP overlap. |
ttl_limit | Sets the delta value that will set off an evasion alert. |
preprocessor stream4_reassemble: [clientonly], [serveronly],\ [noalerts], [ports <portlist>]
Option | Description |
---|---|
clientonly | Provides reassembly for the client side of a connection only. |
serveronly | Provides reassembly for the server side of a connection only. |
noalerts | Won't alert on events that may be insertion or evasion attacks. |
ports <portlist> | Provides reassembly for a whitespace-separated list of ports. By default, reassembly is performed for ports 21, 23, 25, 53, 80, 110, 111, 143, and 513. To perform reassembly for all ports, use all as the port list. |
Just setting the Stream4 and Stream4_reassemble directives without
arguments in the snort.conf file will set them up in their default
configurations shown in Table and Table
.
Stream4 introduces a new command line switch: -z. On TCP traffic, if the -z switch is specified, Snort will only alert on streams that have been established via a three way handshake or streams where cooperative bidirectional activity has been observed (i.e., where some traffic went one way and something other than a RST or FIN was seen going back to the originator). With -z turned on, Snort completely ignores TCP-based Stick/Snot attacks.
Option | Default |
---|---|
session timeout (timeout) | 30 seconds |
session memory cap (memcap) | 8388608 bytes |
stateful inspection (noinspect) | active (noinspect disabled) |
stream stats (keepstats) | inactive |
state problem alerts (detect_state_problems) | inactive (detect_state_problems disabled) |
portscan alerts (disable_evasion_alerts) | inactive (disable_evasion_alerts enabled) |
The Flow tracking module is meant to start unifying the state keeping mechanisms of Snort into a single place. As of Snort 2.1.0, only a portscan detector is implemented, but in the long term, many of the stateful subsystems of Snort will be migrated over to becoming flow plugins. With the introduction of flow, this effectively makes the conversation preprocessor obsolete.
An IPv4 flow is unique when the IP protocol (ip_proto), source IP (sip), source port (sport), destination IP (dip), and destination port (dport) are the same. The dport and sport are 0 unless the protocol is TCP or UDP.
preprocessor flow: [memcap <bytes>], [rows <count>], \ [stats_interval <seconds>], [hash <1|2>]
|
preprocessor flow: stats_interval 0 hash 2
NOTE
This module is designed to detect portscans based off flow creation in the
flow preprocessors. The goal is to catch onemany hosts and one
many ports
scans.
The Flow preprocessor to portscan recognizer is taken from experience with spp_conversation/portscan2 by Jason Larsen & Jed Haile and IPAudit by Jon Rifkin.
This subsystem became a bit more complicated than originally intended, but it does a good job of mitigating false positives from devices such as Squid proxies. The new design is also a lot more memory-consistent than Portscan1 or 2. It also ignores single port SYN floods as they are a DOS, not a portscan.
Memory requirements should be way down from Portscan2 architecture though, but there's slightly less information saved off. The new architecture operates similarly to a ring buffer--when a scanner has not been active in a long time, it's only reclaimed when there is no more memory to use.
All of the prior methods for portscan detection in Snort are deprecated and will be removed in the near future. If you have custom code against Conversation or one of the Portscan preprocessors, consider making it a module in Flow or Portscan.
The Flow preprocessor must first be enabled for Flow-Portscan to function properly.
The basic components of Flow-Portscan are:
Scoreboards contain information about timescales for a single IP address. There are two scoreboards, one for talkers (nodes that are active on your network) and one for scanners (nodes that have talked to a previously unknown port in your server-watch-net).
The uniqueness tracker determines whether this connection should count as something "new" for a particular IP. It verifies that a connection is a new type of connection for a source IP by disregarding the source port. Any change in sip, dip, ip_proto, and dport indicates a new unique connection and will be processed further for the server statistics table and scoring. This keeps things like a web page with 15 images from rapidly increasing point scores with lots of accesses to the same web server.
The server statistics tracker tracks flows destined to the "server-watchnet" and keeps "hitcounts" on the number of times a particular service has been requested with unique requests since Snort has started. This hitcount is tracked by dip, dport, and protocol.
If a service is very popular, connections can be ignored for scoring by comparing the hitcount to the ``server-ignore-limit.'' If there are more requests to this service than the server-ignore-limit, then flow-portscan will completely ignore this service. Similarly, the ``server-scanner-limit'' controls if a request to a service counts as scanner points or as talker points.
If a request to a service is not in the server-watchnet, it will count as talker points. If no server-watchnet is defined, all alerts will be talker alerts.
There are 4 time scales; 2 each for the IP scanner and IP talker.
The fixed timescales detect n events in m seconds. This is the typical type of portscan alert.
The sliding timescales adjust the "score reset point" on each event after the first. This adjusts the side of the window we're detecting portscan events in by taking
end = end + ((end - start) * sliding-scale-factor)
Each time scale has its own point tally that is incremented per new flow. Each set of points only touches either the talker-fixed-score and talker-sliding-score or the scanner-fixed-score and scanner-sliding-score
if(fixed_limit <= fixed_score) generate_alert()
preprocessor flow-portscan: [scoreboard-memcap-talker <bytes>] \ [scoreboard-rows-talker <count>] \ [scoreboard-rows-scanner <count>] \ [scoreboard-memcap-scanner <bytes>] \ [scanner-fixed-threshold <integer>] \ [scanner-sliding-threshold <integer>] \ [scanner-fixed-window <integer>] \ [scanner-sliding-window <integer>] \ [scanner-sliding-scale-factor <float>] \ [talker-fixed-threshold <integer>] \ [talker-sliding-threshold <integer>] \ [talker-fixed-window <integer>] \ [talker-sliding-window <integer>] \ [talker-sliding-scale-factor <float>] \ [unique-memcap <bytes>] \ [unique-rows <integer>] \ [server-memcap <bytes>] \ [server-rows <integer>] \ [server-watchnet <ip list in Snort notation>] \ [src-ignore-net <ip list in Snort notation>] \ [dst-ignore-net <ip list in Snort notation>] \ [tcp-penalties <on|off>] \ [server-learning-time <seconds>] \ [server-ignore-limit <hit count>] \ [server-scanner-limit <hit count>] \ [alert-mode <once|all>] \ [output-mode <msg|pktkludge>] \ [base-score <integer>] \ [dumpall <1>]
Number of rows to use for the talker table.
Number of rows to use for the scanner table.
Number of rows to allocate for the uniqueness tracker.
Number of rows to allocate for server learning.
A general note about rows: higher row counts will take more memory away from the memory caps for a specific subsystem. In Snort output, this is referred to as ``overhead bytes'' and the percentage of overhead encountered will be shown. Higher row counts provide a larger hash table to minimize collisions, with a faster overall processing time at the expense of memory. The hash tables themselves use a pseudorandom hardening salt that is picked at initialization time.
Number of bytes to use for the talker table.
Number of bytes to use for the scanner table.
Number of bytes to allocate to the uniqueness tracker. The more memory given, the less connections to a busy server will appear as a scan target on a popular service.
Number of bytes to allocate for server learning.
Number of points that a scanner must accumulate in the scanner-fixed-window time range. Set to 0 to disable this type of alert.
Number of points that a scanner must accumulate in talker-fixed-window time range. Set to 0 to disable this type of alert.
Number of points that a scanner must accumulate in scanner-sliding-window time range. set to 0 to disable this type of alert.
Number of points that a scanner must accumulate in talker-sliding-window time range. Set to 0 to disable this type of alert.
Number of seconds Flow-Portscan runs before resetting the fixed scanner score.
Number of seconds Flow-Portscan runs before resetting the fixed talker score.
Number of seconds Flow-Portscan runs before resetting the sliding scanner score.
Number of seconds Flow-Portscan runs before resetting the sliding talker score.
How much to increase the sliding window by each time we get a new sliding
scanner entry. Its current size + (scale factor
* current_size).
How much to increase the sliding window by each time we get a new sliding
talker entry. Its current size + (scale factor
* current_size).
The IP list of which source IPs to ignore.
The IP list of which destination IPs to ignore.
If this is enabled, when a new tcp flow enters the portscan detection set, Flow-Portscan checks the TCP flags for non-standard session initiators and assigns penalty points for odd combinations such as SYN+FIN.
The IP list of which machines to learn services on. Busy servers should be placed here to help the portscan detector learn what services are requested on the network.
Number of seconds Flow-Portscan should keep increment hitcounts of services on IPs in the server-watchnet.
This does not validate that the service is connected correctly. It is possible that, while learning, someone floods the table with unique connections, causing something to become a service that you do not wish to be a service. It's generally assumed that the learning time will occur at a time where traffic is ``typical.'' Future versions of Snort should allow this state to be saved and modifiable. If this caveat is a concern in your environment, do not set a server watchnet and rely only on talker scores.
Number of requests a port on an IP in the server-watchnet must see before it is ignored for the purposes of portscans.
Number of requests a port on an IP in the server-watchnet must see before it is is treated as a talker rather than a scanner. This is the minimum number of requests that must be seen during the server-learning-time for the flow to be treated as a talker connection rather than as a scanner connection.
preprocessor flow-portscan: server-watchnet [10.0.0.0/8] \ unique-memcap 5000000 \ unique-rows 50000 \ tcp-penalties on \ server-scanner-limit 50 \ alert-mode all \ output-mode msg \ server-learning-time 3600
The telnet_decode preprocessor allows Snort to normalize Telnet control
protocol characters from the session data. In Snort 1.9.0 and above,
it accepts a list of ports to run on as arguments. Also in 1.9.0,
it normalizes into a separate data buffer from the packet itself so
that the raw data may be logged or examined with the rawbytes content
modifier.
By default, telnet_decode runs against traffic on ports 21, 23, 25, and 119.
preprocessor telnet_decode: <ports>
The rpc_decode preprocessor normalizes RPC multiple fragmented records into a single un-fragmented record. It does this by normalizing the packet into the packet buffer. If stream4 is enabled, it will only process client-side traffic. By default, it runs against traffic on ports 111 and 32771.
Option | Description |
---|---|
alert_fragments | Alert on any fragmented RPC record. |
no_alert_multiple_requests | Don't alert when there are multiple records in one packet. |
no_alert_large_fragments | Don't alert when the sum of fragmented records exceeds one packet. |
no_alert_incomplete | Don't alert when a single fragment record exceeds the size of one packet. |
preprocessor rpc_decode: <ports> [ alert_fragments ] \ [no_alert_multiple_requests] [no_alert_large_fragments] \ [no_alert_incomplete]
This preprocessor measures Snort's real-time and theoretical maximum performance. Whenever this preprocessor is turned on, it should have an output mode enabled, either ``console'' which prints statistics to the console window or ``file'' with a file name, where statistics get printed to the specified file name. By default, Snort's real-time statistics are processed. This includes:
The following options can be used with the Performance Monitor:
preprocessor perfmonitor: time 30 events flow file stats.profile max \ console pktcnt 10000 preprocessor perfmonitor: time 300 file /var/tmp/snortstat pktcnt 10000
HTTPInspect is a generic HTTP decoder for user applications. Given a data buffer, HTTPInspect will decode the buffer, find HTTP fields, and normalize the fields. HTTPInspect works on both client requests and server responses.
The current version of HTTPInspect only handles stateless processing. This means that HTTPInspect looks for HTTP fields on a packet-by-packet basis, and will be fooled if packets are not reassembled. This works fine when there is another module handling the reassembly, but there are limitations in analyzing the protocol. Future versions will have a stateful processing mode which will hook into various reassembly modules.
HTTPInspect has a very ``rich'' user configuration. Users can configure individual HTTP servers with a variety of options, which should allow the user to emulate any type of web server. Within HTTPInspect, there are two areas of configuration: global and server.
The global configuration deals with configuration options that determine the global functioning of HTTPInspect. The following example gives the generic global configuration format:
preprocessor http_inspect: global \ iis_unicode_map <map_filename> \ codemap <integer> \ [detect_anomalous_servers] \ [proxy_alert]
You can only have a single global configuration, you'll get an error if you try otherwise.
This is the global iis_unicode_map file. The iis_unicode_map is a required configuration parameter. The map file can reside in the same directory as snort.conf or specified via a fully-qualified path to the map file.
The iis_unicode_map file is a Unicode codepoint map which tells HTTPInspect which codepage to use when decoding Unicode characters. For US servers, the codemap is usually 1252.
A Microsoft US Unicode codepoint map is provided in the snort source etc directory by default. It is called unicode.map and should be used if no other codepoint map is available. A tool is supplied with Snort to generate custom Unicode maps-ms_unicode_generator.c, which is available at http://www.snort.org/dl/contrib/.
NOTE
This global configuration option enables generic HTTP server traffic inspection on non-HTTP configured ports, and alerts if HTTP traffic is seen. Don't turn this on if you don't have a default server configuration that encompasses all of the HTTP server ports that your users might access. In the future, we want to limit this to specific networks so it's more useful, but for right now, this inspects all network traffic.
This enables global alerting on HTTP server proxy usage. By configuring HTTPInspect servers and enabling allow_proxy_use, you will only receive proxy use alerts for web users that aren't using the configured proxies or are using a rogue proxy server.
Please note that if users aren't required to configure web proxy use, then you may get a lot of proxy alerts. So, please only use this feature with traditional proxy environments. Blind firewall proxies don't count.
preprocessor http_inspect: global iis_unicode_map unicode.map 1252
This configuration supplies the default server configuration for any server that is not individually configured. Most of your web servers will most likely end up using the default configuration.
preprocessor http_inspect_server: server default profile all ports { 80 }
preprocessor http_inspect_server: server 10.1.1.1 profile all ports { 80 }
Important: Some configuration options have an argument of `yes' or `no'. This argument specifies whether the user wants the configuration option to generate an httpinspect alert or not. the `yes/no' argument does not specify whether the configuration option itself is on or off, only the alerting functionality. in other words, whether set to `yes' or 'no', HTTP normalization will still occur, and rules based off HTTP traffic will still trigger.
Users can configure HTTPInspect by using pre-defined HTTP server profiles. Profiles allow the user to easily configure the preprocessor for a certain type of server, but are not required for proper operation.
There are three profiles available: all, apache, and iis.
The ``all'' profile is meant to normalize the URI using most of the common tricks
available. We alert on the more serious forms of evasions. This is a great
profile for detecting all types of attacks, regardless of the HTTP server.
``profile all'' sets the configuration options described in Table .
Option | Setting |
---|---|
flow_depth | 300 |
chunk encoding | alert on chunks larger than 500000 bytes |
iis_unicode_map | codepoint map in the global configuration |
ascii decoding | on, alert off |
multiple slash | on, alert off |
directory normalization | on, alert off |
apache whitespace | on, alert off |
double decoding | on, alert on |
%u decoding | on, alert on |
bare byte decoding | on, alert on |
iis unicode codepoints | on, alert on |
iis backslash | on, alert off |
iis delimiter | on, alert off |
webroot | on, alert on |
non_strict URL parsing | on |
The ``apache'' profile is used for Apache web servers. This differs from the
``iis'' profile by only accepting UTF-8 standard Unicode encoding and not
accepting backslashes as legitimate slashes, like IIS does. Apache also
accepts tabs as whitespace. ``profile apache'' sets the configuration
options described in Table .
Option | Setting |
---|---|
flow_depth | 300 |
chunk encoding | alert on chunks larger than 500000 bytes |
ascii decoding | on, alert off |
multiple slash | on, alert off |
directory normalization | on, alert off |
webroot | on, alert on |
apache whitespace | on, alert on |
utf_8 encoding | on, alert off |
non_strict url parsing | on |
The ``iis'' profile mimics IIS servers. So that means we use IIS Unicode
codemaps for each server, %u encoding, bare-byte encoding, double decoding,
backslashes, etc. ``profile iis'' sets the configuration options described in
Table .
Option | Setting |
---|---|
flow_depth | 300 |
chunk encoding | alert on chunks larger than 500000 bytes |
iis_unicode_map | codepoint map in the global configuration |
ascii decoding | on, alert off |
multiple slash | on, alert off |
directory normalization | on, alert off |
webroot | on, alert on |
double decoding | on, alert on |
%u decoding | on, alert on |
bare byte decoding | on, alert on |
iis unicode codepoints | on, alert on |
iis backslash | on, alert off |
iis delimiter | on, alert on |
apache whitespace | on, alert on |
non_strict URL parsing | on |
The default options used by HTTP Inspect do not use a profile and are described
in Table .
Option | Setting |
---|---|
port | 80 |
flow_depth | 300 |
chunk encoding | alert on chunks larger than 500000 bytes |
ascii decoding | on, alert off |
utf_8 encoding | on, alert off |
multiple slash | on, alert off |
directory normalization | on, alert off |
webroot | on, alert on |
iis backslash | on, alert off |
apache whitespace | on, alert off |
iis delimiter | on, alert off |
non_strict URL parsing | on |
preprocessor http_inspect_server: server 1.1.1.1 profile all ports { 80 3128 }
ports port
port
This is how the user configures which ports to decode on the HTTP server. Encrypted traffic (SSL) cannot be decoded, so adding port 443 will only yield encoding false positives.
iis_unicode_map map_filename
codemap
integer
The IIS Unicode map is generated by the program ms_unicode_generator.c. This program is located on the Snort.org web site at http://www.snort.org/dl/contrib/ directory. Executing this program generates a Unicode map for the system that it was run on. So, to get the specific Unicode mappings for an IIS web server, you run this program on that server and use that Unicode map in this configuration.
When using this option, the user needs to specify the file that contains the IIS Unicode map and also specify the Unicode map to use. For US servers, this is usually 1252. But the ms_unicode_generator program tells you which codemap to use for you server, it's the ANSI code page. You can select the correct code page by looking at the available code pages that the ms_unicode_generator outputs.
flow_depth integer
This specifies the amount of server response payload to inspect. This option significantly increases IDS performance because we are ignoring a large part of the network traffic (HTTP server response payloads). A small percentage of Snort rules are targeted at this traffic and a small flow_depth value may cause false negatives in some of these rules. Most of these rules target either the HTTP header, or the content that is likely to be in the first hundred or so bytes of non-header data. Headers are usually under 300 bytes long, but your mileage may vary.
This value can be set from 0 to 1460, with a value of 0 inspecting all HTTP server payloads (note that this will likely slow down IDS performance). Values above 0 tell http_inspect the number of bytes to inspect in the first packet of the server response.
ascii yes
no
The ascii decode option tells us whether to decode encoded ASCII chars, a.k.a %2f = /, %2e = ., etc. It is normal to see ASCII encoding usage in URLs, so it is recommended that you disable HTTPInspect alerting for this option.
utf_8 yes
no
The utf-8 decode option tells HTTPInspect to decode standard UTF-8 Unicode sequences that are in the URI. This abides by the Unicode standard and only uses % encoding. Apache uses this standard, so for any Apache servers, make sure you have this option turned on. As for alerting, you may be interested in knowing when you have a UTF-8 encoded URI, but this will be prone to false positives as legitimate web clients use this type of encoding. When utf_8 is enabled, ASCII decoding is also enabled to enforce correct functioning.
u_encode yes
no
This option emulates the IIS %u encoding scheme. How the %u encoding scheme works is as follows: the encoding scheme is started by a %u followed by 4 characters, like %uxxxx. The xxxx is a hex-encoded value that correlates to an IIS Unicode codepoint. This value can most definitely be ASCII. An ASCII character is encoded like %u002f = /, %u002e = ., etc. If no iis_unicode_map is specified before or after this option, the default codemap is used.
You should alert on %u encodings, because we are not aware of any legitimate clients that use this encoding. So it is most likely someone trying to be covert.
bare_byte yes
no
Bare byte encoding is an IIS trick that uses non-ASCII characters as valid values when decoding UTF-8 values. This is not in the HTTP standard, as all non-ASCII values have to be encoded with a %. Bare byte encoding allows the user to emulate an IIS server and interpret non-standard encodings correctly.
The alert on this decoding should be enabled, because there are no legitimate clients that encode UTF-8 this way since it is non-standard.
base36 yes
no
This is an option to decode base36 encoded chars. This option is based off of info from http://www.yk.rim.or.jp/ shikap/patch/spp_http_decode.patch.
If %u encoding is enabled, this option will not work. You have to use the base36 option with the utf_8 option. Don't use the %u option, because base36 won't work. When base36 is enabled, ASCII encoding is also enabled to enforce correct behavior.
iis_unicode yes
no
The iis_unicode option turns on the Unicode codepoint mapping. If there is no iis_unicode_map option specified with the server config, iis_unicode uses the default codemap. The iis_unicode option handles the mapping of non-ASCII codepoints that the IIS server accepts and decodes normal UTF-8 request.
You should alert on the iis_unicode option, because it is seen mainly in attacks and evasion attempts. When iis_unicode is enabled, ASCII and UTF-8 decoding are also enabled to enforce correct decoding. To alert on UTF-8 decoding, you must enable also enable utf_8 yes.
double_decode yes
no
The double_decode option is once again IIS-specific and emulates IIS
functionality. How this works is that IIS does two passes through the request
URI, doing decodes in each one. In the first pass, it seems that all types of
iis encoding is done: utf-8 unicode, ascii, bare byte, and %u. In the second
pass, the following encodings are done: ascii, bare byte, and %u. We leave out
utf-8 because I think how this works is that the % encoded utf-8 is decoded to
the Unicode byte in the first pass, and then UTF-8 is decoded in the second stage.
Anyway, this is really complex and adds tons of different encodings for one
character. When double_decode is enabled, so ASCII is also enabled to enforce correct decoding.
non_rfc_char byte
byte ...
This option lets users receive an alert if certain non-RFC chars are used in a request URI. For instance, a user may not want to see null bytes in the request URI and we can alert on that. Please use this option with care, because you could configure it to say, alert on all `/' or something like that. It's flexible, so be careful.
multi_slash yes
no
This option normalizes multiple slashes in a row, so something like: ``foo/////////bar'' get normalized to ``foo/bar.''
If you want an alert when multiple slashes are seen, then configure with a yes, otherwise, use no.
iis_backslash yes
no
Normalizes backslashes to slashes. This is again an IIS emulation. So a
request URI of ``/foobar'' gets normalized to ``/foo/bar.''
directory yes
no
This option normalizes directory traversals and self-referential directories.
The directory:
/foo/fake\_dir/../bargets normalized to:
/foo/bar
The directory:
/foo/./bargets normalized to:
/foo/bar
If you want to configure an alert, specify ``yes'', otherwise, specify ``no.'' This alert may give false positives, since some web sites refer to files using directory traversals.
apache_whitespace yes
no
This option deals with the non-RFC standard of using tab for a space delimiter. Apache uses this, so if the emulated web server is Apache, enable this option. Alerts on this option may be interesting, but may also be false positive prone.
iis_delimiter yes
no
This started out being IIS-specific, but Apache takes this non-standard delimiter was well. Since this is common, we always take this as standard since the most popular web servers accept it. But you can still get an alert on this option.
chunk_length non-zero positive integer
This option is an anomaly detector for abnormally large chunk sizes. This picks up the Apache chunk encoding exploits, and may also alert on HTTP tunneling that uses chunk encoding.
no_pipeline_req
This option turns HTTP pipeline decoding off, and is a performance enhancement if needed. By default, pipeline requests are inspected for attacks, but when this option is enabled, pipeline requests are not decoded and analyzed per HTTP protocol field. It is only inspected with the generic pattern matching.
non_strict
This option turns on non-strict URI parsing for the broken way in which Apache
servers will decode a URI. Only use this option on servers that will accept
URIs like this: "get /index.html alsjdfk alsj lj aj la jsj sn". The
non_strict option assumes the URI is between the first and second space even if
there is no valid HTTP identifier after the second space.
allow_proxy_use
By specifying this keyword, the user is allowing proxy use on this server. This means that no alert will be generated if the proxy_alert global keyword has been used. If the proxy_alert keyword is not enabled, then this option does nothing. The allow_proxy_use keyword is just a way to suppress unauthorized proxy use for an authorized server.
no_alerts
This option turns off all alerts that are generated by the HTTPInspect preprocessor module. This has no effect on HTTP rules in the ruleset. No argument is specified.
oversize_dir_length non-zero positive integer
This option takes a non-zero positive integer as an argument. The argument specifies the max char directory length for URL directory. If a url directory is larger than this argument size, an alert is generated. A good argument value is 300 characters. This should limit the alerts to IDS evasion type attacks, like whisker -i 4.
inspect_uri_only
This is a performance optimization. When enabled, only the URI portion of HTTP requests will be inspected for attacks. As this field usually contains 90-95% of the web attacks, you'll catch most of the attacks. So if you need extra performance, enable this optimization. It's important to note that if this option is used without any uricontent rules, then no inspection will take place. This is obvious since the URI is only inspected with uricontent rules, and if there are none available, then there is nothing to inspect.
For example, if we have the following rule set:
alert tcp any any -> any 80 ( msg:"content"; content: "foo"; )
and the we inspect the following URI:
get /foo.htm http/1.0\r\n\r\n
No alert will be generated when `inspect_uri_only' is enabled. The `inspect_uri_only' configuration turns off all forms of detection except uricontent inspection.
webroot
This option generates an alert when a directory traversal traverses past the web server root directory. This generates much fewer false positives than the directory option, because it doesn't alert on directory traversals that stay within the web server directory structure. It only alerts when the directory traversals go past the web server root directory, which is associated with certain web attacks.
preprocessor http_inspect_server: server 10.1.1.1 \ ports { 80 3128 8080 } \ flow_depth 0 \ ascii no \ double_decode yes \ non_rfc_char { 0x00 } \ chunk_length 500000 \ non_strict \ no_alerts preprocessor http_inspect_server: server default \ ports { 80 3128 } \ non_strict \ non_rfc_char { 0x00 } \ flow_depth 300 \ apache_whitespace yes \ directory no \ iis_backslash no \ u_encode yes \ ascii no \ chunk_length 500000 \ bare_byte yes \ double_decode yes \ iis_unicode yes \ iis_delimiter yes \ multi_slash no preprocessor http_inspect_server: server default \ profile all \ ports { 80 8080 }
The asn.1 detection plugin decodes a packet or a portion of a packet, and looks for various malicious encodings.
The general configuration of the asn.1 rule option is as follows:
asn1: [keyword [argument]], . . .
Multiple keywords can be used in an 'asn1' option and the implied logic is boolean OR. So if any of the arguments evaluate as true, the whole option evaluates as true.
The ASN.1 keywords provide programmatic detection capabilities as well as some more dynamic type detection. Most of the keywords don't have arguments as the detection is looking for non-configurable information. If a keyword does have an argument, the keyword is followed by a comma and the argument is the next word. If a keyword has multiple arguments, then a comman separates each.
The bitstring_overflow option detects invalid bitstring encodings that are known to be remotely exploitable.
The double_overflow detects a double ASCII encoding that is larger than a standard buffer. This is known to be an exploitable function in Microsoft, but it is unknown at this time which services may be exploitable.
This detection keyword compares ASN.1 type lengths with the supplied argument. The syntax looks like, ``oversize_length 500''. This means that if an ASN.1 type is greater than 500, then this keyword is evaluated as true. This keyword must have one argument which specifies the length to compare against.
This is the absolute offset from the beginning of the packet. For example, if you wanted to decode snmp packets, you would say ``absolute_offset, 0''. absolute_offset has one argument--the offset value. Offset may be positive or negative.
This is the relative offset from the last content match or byte_test/jump.
relative_offset has one argument--the offset number. So if you
wanted to start decoding and ASN.1 sequence right after the content ``foo'',
you would specify 'content:"foo"; asn1: bitstring_overflow, relative_offset, 0'
. Offset values may be positive or negative.
The following rules use ASN.1 decoding options:
alert udp any any -> any 161 (msg:"Oversize SNMP Length"; \ asn1: oversize_length, 10000, absolute_offset, 0;) alert tcp any any -> any 80 (msg:"ASN1 Relative Foo"; content:"foo"; \ asn1: bitstring_overflow, print, relative_offset, 0;)