|
NIDS v1 Test Details
Contents:
A) Device Integrity Checking (sensor)
A.1. Listening Service Inventory
A.2. Known holes check of all listeners
A.3. SNMPv1 - Protos SNMP Ttests
A.4. To: Routable ISIC/TCPSIC/UDPSIC/ICMPSIC
A.5. Through: Routable ISIC/TCPSIC/UDPSIC/ICMPSIC
A.6. To: Unfiltered ISIC/TCPSIC/UDPSIC/ICMPSIC
A.7. Through: Unfiltered ISIC/TCPSIC/UDPSIC/ICMPSIC
A.8. TCP / ISN generation test
B) Signature Baseline Tests
B.1. Mainstream attack baseline
B.2. Modified attacks (2 attacks)
C) State Tests
C.1. State Confirmation Test
C.2. Tool Dry-Run
C.3-C10. Session Stress Tests
D) Discard Tests
D.1. Tool Dry-Run
D.2-D6. Bogus Port Tests
D.7-D11. Valid Port Tests
D.12. Invalid Traffic (64-byte frames)
E) Engine Flex Tests
E.1. Tool Dry-Run
E.2,3,5,6,and 8. HTTP background traffic and injected attacks
E.4 and 7. HTTP background traffic and injected attacks (536 byte MSS)
F) Evasion Tests
F.1. Basic IP Fragmentation (ordered 8-byte) [fragrouter test F1]
F.2. Basic IP Fragmentation (ordered 24-byte) [fragrouter test F2]
F.3. Complex IP Fragmentation (ordered 8-byte IP fragments, one out of order) [fragrouter test F3]
F.4. Complex IP Fragmentation (ordered 8-byte IP fragments, one duplicate) [fragrouter test F4]
F.5. Complex IP Fragmentation (out of order 8-byte IP fragments, one duplicate) [fragrouter test F5]
F.6. Complex IP Fragmentation (ordered 8-byte IP fragments, marked last frag first) [fragrouter test F6]
F.7. Basic TCP segmentation (3-whs, ordered 1-byte segments, one out of order) [fragrouter test T8]
F.8. Basic TCP segmentation (3-whs, bad TCP checksum FIN/RST, ordered 1-byte segments) [fragrouter test T1]
F.9. Complex TCP segmentation (3-whs, ordered 1-byte segments, one duplicate) [fragrouter test T3]
F.10. Complex TCP segmentation (3-whs, ordered 1-byte segments, one overwriting) [fragrouter test T4]
F.11. Complex TCP segmentation (3-whs, ordered 2-byte segments, fwd-overwriting) [fragrouter test T5]
F.12. Complex TCP segmentation (3-whs, ordered 1-byte segments, interleaved null segments) [fragrouter test T7]
F.13. Complex TCP segmentation (3-whs, out of order 1-byte segments) [fragrouter T9]
F.14. Complex TCP segmentation (3-whs, ordered 1-byte segments, interleaved SYN) [fragrouter C2]
F.15. Complex TCP segmentation (ordered 1-byte null segments, 3-whs, ordered 1-byte segments) [fragrouter C3]
F.16. Complex TCP segmentation (3-whs, RST, 3-whs, ordered 1-byte segments) [fragrouter R1]
F.17. Delayed injection @ 100,000 sessions
F.18. Delayed injection @ 250,000 sessions
F.19. Delayed injection @ 500,000 sessions
F.20. HTTP obfuscation (hex encoding)
F.21. HTTP obfuscation (double hex encoding)
F.22. HTTP obfuscation (Unicode / UTF-8 encoding)
F.23. HTTP obfuscation (self-referential directories) [whisker -I 2]
F.24. HTTP obfuscation (premature URL ending) [whisker -I 3]
F.25. HTTP obfuscation (prepend long string) [whisker -I 4]
F.26. HTTP obfuscation (fake URL parameter) [whisker -I 5]
F.27. HTTP obfuscation (case sensitivity) [whisker -I 7]
F.28. HTTP obfuscation (Windows directory syntax) [whisker -I 8]
F.29. HTTP obfuscation (session splicing) [whisker -I 9]
F.30. HTTP obfuscation (connection reuse)
F.31. HTTP Engine Verification (HTTP version 0.9)
F.32. HTTP Engine Verification (HTTP version 1.0)
F.33. HTTP Engine Verification (HTTP version 1.1)
G) Inline/Tap Engine Tests
G.1. Tool Dry-Run
G.2. HTTP background traffic and injected attacks
Section Goals:
To verify that the sensor itself is not easily subject
to compromise or Denial of Service (DoS). These tests will become deeper
over time, addressing more protocol layers and common failure conditions.
These tests are mandatory, and vendors may not elect to leave any portions
of the "A" section unverified or unreleased.
Goals:
List the services the product introduces to the environment.
If installed on an end-user administrable, commercial, off-the-shelf platform,
list the services the product requires that cannot be filtered by the device
itself or turned off - in general, the services the software adds to the COTS
platform. If installed on a turnkey or appliance system, list all services
that are available to the network.
What to look for:
On the OSEC Verification Certificate:
Just the additional or mandatory listening services.
The format lists the port number followed by the base IP protocol. Thus,
443/tcp would be port 443, protocol TCP. If the higher-layer protocol is
identified, that will be listed as well above the port number.
In the OSEC full information spreadsheet:
All services found to be listening to the network
on the device as tested.
Goals:
No product should go to market with a service with
a published security issue without full disclosure. List the services from
A.1. that are identified to have known vulnerabilities.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if no services have published security issues at the date of
the testing. "F" for "fail" if any services from A.1. have published security
issues at the date of the testing.
In the OSEC full information spreadsheet:
The services with P or F each, and the version number of any service with
a published issue. Where available, a CVE reference will be included.
Goals:
No product should go to market with a service with a published security test,
and not have been subjected to it. SNMPv1 issues have been widely publicized,
and open source, tested implementations are available. Every product using
SNMP should be capable of standing up to the Protos SNMP test suite.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the SNMPv1 services survive the Protos run without fault.
F for "fail" if the SNMPv1 services or the device fail during the test run.
A dash (-) will be displayed if the product simply has no SNMPv1 services
available.
In the OSEC full information spreadsheet:
A "P" only if the product passed. If the product failed any of the Protos
SNMPv1 test suite, the test on which the product failed, and the nature of
the failure will be indicated.
Goals:
No product should go to market with a weak or unstable
stack. ISIC runs are performed, sequentially, through a L3 router, against
the sensor's administrative interface. The L3 router (a Cisco 3640 with a
version of IOS demonstrably stable when faced with ISIC) is used to expose
the product under test solely to packets that can be routed across an enterprise
or the Internet. The tests use a randomized source address and port, randomized
destination port (or protocol in the case of isic) and 25% randomness on each
ISIC component's options. Each test run is conducted for a total of 5.4 million
packets, or 1 hour, at an average pps rate of 1500, or approximately 8Mbps.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the stack survives the ISIC suite run without fault. F for
"fail" if the stack, any service, or the device fails during the test run.
In the OSEC full information spreadsheet:
A "P" only if the product passed. If the product
failed any of the ISIC test suite, the test on which the product failed, and
the nature of the failure will be indicated.
Goals:
No product should go to market with a weak or unstable
stack. ISIC runs are performed, sequentially, through a L3 router, past the
sensor's sniffing interface. The L3 router (a Cisco 3640 with a version of
IOS demonstrably stable when faced with ISIC) is used to expose the product
under test solely to packets that can be routed across an enterprise or the
Internet. The tests use a randomized source address and port, randomized
destination port (or protocol in the case of isic) and 25% randomness on each
ISIC component's options. Each test run is conducted for a total of 5.4 million
packets, or 1 hour, at an average pps rate of 1500, or approximately 8Mbps.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the sniffing device stack survives the ISIC suite run without
fault (i.e., it is still able to detect attacks, and remains up throughout
the testing). F for "fail" if the stack, any service, or the device fails
during the test run.
In the OSEC full information spreadsheet:
A "P" only if the product passed. If the product
failed any of the ISIC test suite, the test on which the product failed, and
the nature of the failure will be indicated.
Goals:
No product should go to market with a weak or unstable
stack. ISIC runs are performed, sequentially, against the sensor's administrative
interface. The tests use a randomized source address and port, randomized
destination port (or protocol in the case of isic) and 25% randomness on each
ISIC component's options. Each test run is conducted for a total of 5.4 million
packets, or 1 hour, at an average pps rate of 1500, or approximately 8Mbps.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the stack survives the ISIC suite run without fault. F for
"fail" if the stack, any service, or the device fails during the test run.
In the OSEC full information spreadsheet:
A "P" only if the product passed. If the product
failed any of the ISIC test suite, the test on which the product failed, and
the nature of the failure will be indicated.
Goals:
No product should go to market with a weak or unstable
stack. ISIC runs are performed, sequentially, past the sensor's sniffing
interface. The tests use a randomized source address and port, randomized
destination port (or protocol in the case of isic) and 25% randomness on each
ISIC component's options. Each test run is conducted for a total of 5.4 million
packets, or 1 hour, at an average pps rate of 1500, or approximately 8Mbps.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the sniffing device stack survives the ISIC suite run without
fault (i.e., it is still able to detect attacks, and remains up throughout
the testing). F for "fail" if the stack, any service, or the device fails
during the test run.
In the OSEC full information spreadsheet:
A "P" only if the product passed. If the product
failed any of the ISIC test suite, the test on which the product failed, and
the nature of the failure will be indicated.
Goals:
No product should go to market with weak TCP initial
sequence number generation for its listening TCP services, or initiated TCP
connections. Weak TCP ISNs can allow an attacker to spoof or hijack sessions
to or from the device. Ensure the device has at minimum a good pseudo-random
ISN generation not subject to easy modeling and spoofing. We anticipate this
test will become more stringent over time as further research in ISN statistical
modeling is published.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device demonstrates statistically strong ISN generation.
F for "fail" if the device has statically increasing, or easily-predictable
ISN generation.
In the OSEC full information spreadsheet:
A "P" only if the product passed with a brief description
of the ISN generation. A scatter plot of the ISNs may be made available on
the OSEC web site. If the product failed the ISN testing, the nature of the
ISN generation and a description of the weakness will be indicated.
Section Goals:
To verify that the sensor picks up the basic attacks
used throughout the testing under minimal background traffic conditions.
Since all OSEC testing uses real attacks against victim hosts, a failure is
a signature failure. While the OSEC NIDS test suite is not a signature validation
regime, a failure to pass baselines for the selected signatures should be
considered a serious gap in signature provisioning of the NIDS device.
Goals:
Verify that the sensor picks up the basic attacks
used throughout the testing under null background traffic conditions. Signature
test.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on each of the attacks. F for "fail" if
the device misses an attack. "N/A" if the vendor elects not to test or not
to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device misses an attack, and the specific attack missed. "N/A" if
the vendor elects not to test or not to claim validation of that capability.
Goals:
Verify that the sensor picks up the attacks used
in the testing when they differ from publicly-available exploits, under null
background traffic conditions. Signature test.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on each of the attacks. F for "fail" if
the device misses an attack. N/A if the vendor elects not to test or not
to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device misses an attack, and the specific attack missed. N/A if the
vendor elects not to test or not to claim validation of that capability.
Section Goals:
To verify that the sensor has a stable state table
reasonably sized for the traffic levels it is designed to monitor. A network-based
IDS should be able to track a number of "sessions" at setup and tear-down
rates equivalent to what a stateful inspection firewall in a similar speed
class can track. In addition, validate that state is correctly monitored
and preserved both when inferring state from extant traffic, and when under
stress.
Goals:
Verify that the sensor does not infer state from
traffic flows that contain attacks and nothing else.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device does not alert on each of the attacks.
F for "fail" if the device reports an attack where no state has been created
(and thus could not succeed). N/A if the vendor elects not to test or not
to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device alerts on an attack with bad state, and the specific attack
on which it alerted. N/A if the vendor elects not to test or not to claim
validation of that capability.
Goals:
Verify that the sensor does not report false positives
from the session generation tool, other than alerts about traffic or session
floods. Tuning exercise - make sure sensor isn't displaying false positives.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device does not alert on the traffic, other than
to report traffic or session floods. F for "fail" if the device reports an
attack where no attack was made (and thus could not succeed). N/A if the
vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device alerts on an attack other than traffic or session floods, and
the specific attack that it reported. N/A if the vendor elects not to test
or not to claim validation of that capability.
Goals:
Verify that the sensor does not fail to detect attacks
(or to detect them correctly) while under load from the session generation
tool. Session concurrency, rate of setup and tear-down, and IP addresses
are adjusted from low levels, to very high levels in eight tests.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Section Goals:
To verify that the sensor does not expend significant
resources while handling traffic that does not match any monitoring rule in
its pre-parser's sniffing rules. Use a combination of stateless TCP ACKs
and Layer-4 valid TCP traffic as background for injected attacks. None of
the background traffic is valid at Layers 5-7. To achieve high speeds, most
NIDS sensors employ a quick discard mechanism for traffic that falls outside
their signature set. This suite tests the ability of the sniffing portion
of the IDS to hand off possibly-significant traffic to the signature or rule
processing portion of the solution while under various less-significant traffic
loads.
Goals:
Verify that the sensor does not report false positives
from the traffic generation tools, other than alerts about traffic or session
floods. Tuning exercise - make sure sensor isn't displaying false positives.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device does not alert on the traffic, other than
to report traffic or session floods. F for "fail" if the device reports an
attack where no attack was made (and thus could not succeed). N/A if the
vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device alerts on an attack other than traffic or session floods, and
the specific attack that it reported. N/A if the vendor elects not to test
or not to claim validation of that capability.
Goals:
Verify that the sensor does not fail to detect attacks
(or to detect them correctly) while under load from the traffic generation
tool. Traffic is generated to and from ports with no published services and
no matching rule in the NIDS. Session BPS is adjusted from low levels, to
very high levels in five tests.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor does not fail to detect attacks
(or to detect them correctly) while under load from the traffic generation
tool. Traffic is generated to or from ports with published services and a
matching rule in the NIDS (e.g., port 80/tcp). Session BPS is adjusted from
low levels, to very high levels in five tests.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Test the sensor at the highest discard traffic level
for which it passes verification, using non-layer-3 valid junk traffic frames.
Verify that the sensor does not fail to detect attacks (or to detect them
correctly) while under load from the traffic generation tool. Session BPS
is set at the highest level for which the sensor passes verification in the
preceding discard tests. Maximally stress the frame-checking routines of
the first sniffing mechanism.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Section Goals:
To verify the sensor's ability to recognize attacks
when under maximal legitimate traffic stress. These tests are designed for
stressing specific inspectors/decoders with background traffic that is layer
7 valid on regularly monitored ports (e.g., 80/tcp), while valid attacks are
injected.
Goals:
Verify that the sensor does not report false positives
from the traffic generation tools, other than alerts about traffic or session
floods. Tuning exercise - make sure sensor isn't displaying false positives.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device does not alert on the traffic, other than
to report traffic or session floods. F for "fail" if the device reports an
attack where no attack was made (and thus could not succeed). N/A if the
vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device alerts on an attack other than traffic or session floods, and
the specific attack that it reported. N/A if the vendor elects not to test
or not to claim validation of that capability.
Goals:
Verify that the sensor does not fail to detect attacks
(or to detect them correctly) while under load from the traffic generation
tool. Traffic is Layer-7 valid HTTP traffic using a combination of session
sizes averaging 4.5K per transaction with an MTU of 1500. Client connections
simulate a mix of client browsers using HTTP 1.0 and 1.1 requests. Session
BPS is tested at five different levels, from 80Mbps to 750Mbps, depending
on the sensor's claimed capabilities.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor does not fail to detect attacks
(or to detect them correctly) while under load from the traffic generation
tool. Traffic is Layer-7 valid HTTP traffic using a combination of session
sizes averaging 4.5K per transaction with an MTU of 576 (MSS 536). Client
connections simulate a mix of client browsers using HTTP 1.0 and 1.1 requests.
Session BPS is tested at two different levels, 80Mbps and 500 Mbps, depending
on the sensor's claimed capabilities.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Section Goals:
To verify the sensor's ability to recognize attacks
that are sent through various obfuscation or evasion mechanisms. These tests
are designed to verify the sensor's ability to deal with published means to
evade network-based IDS sensors. Attacks are generated when no background
traffic is on the wire and are real attacks against real hosts. No fragmentation
or other attacks are used in which the attack is not successful against a
real target host.
Goals:
Verify that the sensor can perform basic, ordered
reassembly. Attacks are fragmented and sent in ordered 8-byte increments.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform basic, ordered
reassembly. Attacks are fragmented and sent in ordered 24-byte increments.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. Attacks are fragmented and sent in ordered 8-byte increments
with one fragment sent out of order.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. Attacks are fragmented and sent in ordered 8-byte increments
with one fragment (the second to last in each packet) duplicated.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. Attacks are fragmented and sent in disordered 8-byte increments
with one fragment (the second to last of each packet) duplicated.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. Attacks are fragmented and sent in ordered 8-byte increments
with the last portion (fragment) of the fragmented packet sent first.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform basic TCP reassembly.
The attack connection 3-way-handshake is completed normally, and then the
attack is fragmented and sent in ordered 1-byte increments with one fragment
sent out of order.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform basic TCP reassembly.
The attack connection 3-way-handshake is completed normally, a simulated disconnection
is made (with bad TCP checksum), and then the attack is fragmented and sent
in ordered 1-byte increments. Tests the state engine by verifying whether
the NIDS is watching for remote acknowledgment of disconnection.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is completed normally,
and then the attack is fragmented and sent in ordered 1-byte increments with
one fragment (the second to last of each packet) duplicated.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is completed normally,
and then the attack is fragmented and sent in ordered 1-byte increments with
one fragment (the second to last of each packet) repeated, but with null data
content.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is completed normally,
and then the attack is fragmented and sent in ordered 2-byte increments.
Each fragment is preceded by a 1-byte, null data segment that overlaps the
latter 2-byte segment.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is completed normally,
and then the attack is fragmented and sent in ordered 1-byte increments.
Each fragment is followed by a 1-byte, null data segment that has a far out-of-window
sequence number. In essence, a sequence number sanity check.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is completed normally,
and then the attack is fragmented and sent in ordered 1-byte increments.
Each fragment is followed by a 1-byte, null data segment that has a far out-of-window
sequence number. In essence, a sequence number sanity check.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is completed normally,
and then the attack is fragmented and sent in ordered 1-byte increments.
Each fragment is followed by a repeat SYN for the connection. In essence,
a connection-tracking sanity check. The IDS should not lose track of the
ongoing session merely because of a fresh SYN with the same connection parameters.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is preceded by null data
sent in ordered 1-byte segments as if a connection had already completed,
then the handshake occurs with the same connection parameters; finally, the
attack is fragmented and sent in ordered 1-byte increments. In essence, a
connection-tracking sanity check. The IDS should not be confused by trash,
connection-less data preceding the legitimate connection with the same connection
parameters.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can perform more complex
reassembly. The attack connection 3-way-handshake is completed and immediately
closed with a valid RST. Another connection with different ISN is made otherwise
with the same parameters, and the attack is sent in ordered 1-byte segments.
In essence, a connection-tracking sanity check. The IDS should not be confused
by the preceding session open and RST preceding another, similar connection
and attack.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability without missing the attack.
Goals:
Verify that the sensor can handle session/state
table entry quantities without losing track of an extant connection upon which
an attack occurs. The attack connection 3-way-handshake is completed, then
100,000 L7 valid HTTP sessions are opened and closed past the sensor. Upon
completion of the 100,000 sessions, the attack is sent on the already-open
connection. The IDS should not lose track of the attack session, or should
be able to infer a session from the attack and response by the victim. Any
NIDS claiming T3 speeds or greater should be able to handle a 100,000 session
table roll.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can handle session/state
table entry quantities without losing track of an extant connection upon which
an attack occurs. The attack connection 3-way-handshake is completed, then
250,000 L7 valid HTTP sessions are opened and closed past the sensor. Upon
completion of the 250,000 sessions, the attack is sent on the already-open
connection. The IDS should not lose track of the attack session, or should
be able to infer a session from the attack and response by the victim. Any
NIDS claiming to exceed full-duplex fast ethernet speeds should be able to
handle a 250,000 session table roll without missing the attack.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can handle session/state
table entry quantities without losing track of an extant connection upon which
an attack occurs. The attack connection 3-way-handshake is completed, then
500,000 L7 valid HTTP sessions are opened and closed past the sensor. Upon
completion of the 500,000 sessions, the attack is sent on the already-open
connection. The IDS should not lose track of the attack session, or should
be able to infer a session from the attack and response by the victim. Any
NIDS claiming gig speeds or greater should be able to handle a 500,000 session
table roll without missing the attack.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can handle URL encoded attacks,
much as the target web server would decode them. The IDS should identify
the attack correctly (and perhaps also alert on the use of URL encoding for
non-binary characters in the attack).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can handle URL encoded attacks
that are themselves URL encoded (just the % is doubly-encoded), much as the
target web server would decode them. The IDS should identify the attack correctly
(and perhaps also alert on the use of URL encoding for non-binary characters
in the attack).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can handle Unicode / UTF-8
encoded attacks, much as the target web server would decode them. The IDS
should identify the attack correctly (and perhaps also alert on the use of
the encoding for non-binary characters in the attack).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor can handle self-referential
directory (./././ etc.) evasion attacks, much as the target web server would
decode them. The IDS should identify the attack correctly (and perhaps also
alert on the use of the self-referential directory).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse URI requests
that contain an apparent request ending (HTTP/1.0\r\n etc.) followed by headers
that in fact continue the request, followed by a "real" ending. The IDS should
identify the attack correctly (and perhaps also alert on the use of the embedded,
distracting request ending).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse URI requests
that contain a long line of garbage text, followed by the content that in
fact is the attack request. The IDS should identify the attack correctly
(and perhaps also alert on the use of the prepended long string).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse URI requests
that contain fake parameter passing, followed by the content that in fact
is the attack request. The IDS should identify the attack correctly (and
perhaps also alert on the use of the bogus parameter).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse all-caps
or mixed-caps URI requests to known case-insensitive targets. The IDS should
identify the attack correctly (and perhaps also alert on the use of the alternate
capitalization).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse HTTP
requests to Windows servers, employing backslashes (\) as internal separators
rather than the forward slashes (/) that the RFCs require. The IDS should
identify the attack correctly (and perhaps also alert on the use of the backslash).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse HTTP-based
attacks that are intentionally spread across multiple packets with 1-3 bytes
of the request per packet. The IDS should identify the attack correctly (and
perhaps also alert on the use of the session fragmentation).
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse HTTP-based
attacks that occur after an innocuous request and a short pause, all within
a single kept-alive HTTP 1.1 session. The IDS should identify the attack
correctly.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse HTTP-based
attacks that occur in a versionless GET request. The IDS should identify
the attack correctly.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse HTTP-based
attacks that occur in an HTTP 1.0 request. The IDS should identify the attack
correctly.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Goals:
Verify that the sensor will properly parse HTTP-based
attacks that occur in an HTTP 1.1 request with full headers. The IDS should
identify the attack correctly.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
Section Goals:
To verify the sensor's ability to recognize attacks
when reading traffic in-line or from a tap. These tests verify the engine's
ability to re-integrate directional streams and to cope with traffic that
exceeds half-duplex fiber speeds. These tests are designed for stressing
specific inspectors/decoders with background traffic that is layer 7 valid
on regularly monitored ports (e.g., 80/tcp), while valid attacks are injected.
Goals:
Verify that the sensor does not report false positives
from the traffic generation tools, other than alerts about traffic or session
floods. Tuning exercise - make sure sensor isn't displaying false positives.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device does not alert on the traffic, other than
to report traffic or session floods. F for "fail" if the device reports an
attack where no attack was made (and thus could not succeed). N/A if the
vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device alerts on an attack other than traffic or session floods, and
the specific attack that it reported. N/A if the vendor elects not to test
or not to claim validation of that capability.
Goals:
Verify that the sensor does not fail to detect attacks
(or to detect them correctly) while under load from the traffic generation
tool. Traffic is Layer-7 valid HTTP traffic using a combination of session
sizes averaging 4.5K per transaction with an MTU of 1500. Client connections
simulate a mix of client browsers using HTTP 1.0 and 1.1 requests. Session
BPS is tested at 1500 Mbps.
What to look for:
On the OSEC Verification Certificate:
P for "pass" if the device alerts on the attacks.
F for "fail" if the device fails to report or misreports an attack. N/A if
the vendor elects not to test or not to claim validation of that capability.
In the OSEC full information spreadsheet:
A "P" only if the product passed. F for "fail"
if the device fails to alert or misreports an attack, and the specific attack
on which the problem occurred. N/A if the vendor elects not to test or not
to claim validation of that capability.
|
