Discover how an SOC helps you fight intruders from the confines of a safe house.

You wouldn't go into battle without a trusted group of generals strategizing your best attacks and defenses, so why try to secure your data without an infosec team and plan? If you don't have a dedicated security operations center and staff, you'll be scrambling to shore up your defenses, even as the bad guys are invading your system.

A SOC can be as simple as a set of offices or cubicles next to each other, or as sophisticated as a standalone complex with extra-large displays, two-factor physical security and a budget to match. Typically, only the largest companies have the resources to build and staff a dedicated SOC. In a recent survey of Secure Enterprise readers, 72 percent of respondents with fewer than 5,000 employees had no plans to build a SOC. Among the 28 percent who have a SOC or plan to build one, 53 percent will collocate in the network operations center, which makes sense because an existing NOC provides the framework to build in the additional functionality required for a SOC. The rest plan to house the SOC in a separate location, either a building (25 percent) or a room (22 percent).

Most Wanted Skills Click to Enlarge

The tasks the security operations center handles can range from typical event management and incident response to account administration, investigations and forensics. Some companies choose to outsource their SOCs, because they want the expertise and 24-hour monitoring of a dedicated security team without staffing and building a SOC. For many, it makes sense to maintain an internal SOC, especially when a NOC already exists. See "How To Set Up a War Room", for a checklist of the essentials. Make sure you've got the budget to build out and hire. But remember, not everyone can speak Perl and decode a TCP/IP packet in three seconds. In "Most-Wanted Skills," we outline the various skill sets needed.

SOC It to Them

The SOC facility, like any other critical operations center, must be secure--both electronically and physically. Building a separate infrastructure is expensive and probably not worth the effort. Instead, try to use an existing, secure infrastructure. In many cases, the data center is a good fit, because it already has manned guard stations, cameras, security clearance and sign-in/sign out requirements and other physical security controls. Everyone who enters and leaves the SOC must be tracked to provide an access audit trail. Even a simple login sheet is better than nothing.

Latest Issue of Secure Enterprise Magazine Read more >>

And only specific employees should have access. Network engineers, application developers and business partners, for example, don't need physical access to the SOC. The SOC manager, or leader, provides access approval. While the CISO or CIO is too high a position to make day-to-day decisions, he or she would still be responsible for the governing policies.

Of course, the greatest threat to your SOC will come from malicious users gaining access to it over the network. Your SOC will access key security systems, such as firewalls, IDSs, IPSs (intrusion-prevention systems), and antivirus- and event-management consoles. It will also store confidential data, such as configuration information, trouble tickets and event logs.

So your security controls must ensure the integrity and confidentiality of the data and systems. Common and successful approaches to this end include having highly restrictive firewall policies for the SOC and placing an IDS--or better yet, an IPS--with restrictive policies inline between the SOC and the rest of the company network. A nonroutable internal address space inside the SOC will give it an additional level of obscurity. Also, make sure your firewall rules lock down the environment. If remote access to the SOC is needed from within the company network, require a VPN connection.

Devices within the SOC should be under the management of the SOC leader, and all changes to these devices must be tested and audited before being made. In addition, you might want to set up a VLAN (virtual LAN) to provide access to key network and security devices in the event the production network is unavailable or saturated with traffic so you can't get to it.

Network Connections

But it's not all about hiding the SOC. An additional network connection will give your SOC personnel an outsider's view of your network. This link could be a T1 line or even an inexpensive DSL connection, preferably from an ISP other than the one providing your primary Internet connections. With this external view, you can perform tests to determine vulnerabilities in the event of a perimeter compromise. And if your primary or secondary network connection goes down, the ancillary network connection can be used for sending e-mail, downloading updates and performing investigations.

Ideally, the account won't be tied to the primary company, nor will DNS entries indicate who owns the connection. This way, when investigating a suspected intruder or attacker, you can visit the Web site or server using an IP address that isn't traceable to your company so as not to let the culprit know you're aware of the attack.

This ancillary network connection must not be tied to your production network. If possible, use an air-gap approach where there's no physical connection between the two networks. You also can set up different VLANs on a single switch or different interfaces on a firewall with a strict set of rules in place. Undoubtedly, you'll need a wireless network in the SOC so workers can roam between conference rooms and offices. However, wireless access should be limited to only specific users and systems. One possible solution is to have wireless users access the SOC network over a VPN requiring two-factor authentication. The wireless network should have virtually no access to the SOC except over the VPN, ensuring that no wireless user can gain access to critical systems.

Coverage requirements are a key factor when proposing and designing a SOC. If you're seeking coverage past normal business hours, consider setting up a SOC in a remote time zone so its staffers can serve your after-hours needs without working the graveyard shift themselves. For some companies, it may make sense to maintain one SOC in North America and another in Asia or Europe, though additional security controls would have to be put in place to safeguard information flowing between the two.

No SOC is complete without a place to analyze suspicious programs and test possible solutions. This is especially true for organizations unlucky enough to be hit by a zero-day or unknown exploit, worm or virus. Most of the time, even unknowns are a variant of some known worm. The variant may use a different payload or a new propagation method. Still, it may be difficult to determine what's occurring and what you must do to stop it. Enter the sandbox.

A sandbox is a set of systems--or even a single system--that's separate from any other network in your organization and used to analyze the behavior of suspicious applications. You can also use a single system running a virtual machine application, such as VMWare. Set up multiple VMWare OSs to share a private network segment--for instance, you can have Windows System A as the infected system and Windows System B as the target system. Run a packet-capture tool, such as Ethereal, on System A to capture all packets going in and out of the system, then run Ethereal on System B and compare the two. Other useful tools for sandboxing, including regmon, filemon and diskmon, are available from Sysinternals. Of course, similar solutions can be found for your favorite Unix OS, too.

A SOC also demands a large display to monitor the health of network traffic and load, routers and firewalls, Active Directory, DNS, production-application servers and other key items.

A Series of Unfortunate Events

Security events can be generated from many devices and applications, including firewalls, routers, VPN concentrators, Web and antivirus applications, and IDS/IPS devices. The sheer number of events that many companies must collect, analyze and store can be staggering. To help, many SOC teams are turning to SEM (security event management) tools. Arcsight, GuardedNet and Network Intelligence are the leaders in this area. It's not uncommon to hear that some SEM tools process several thousand events per second.

But not every event must be captured and analyzed. Use aggregation and selectively choose which types of events get captured to reduce the volume of data. You can limit your analysis to only events coming from critical servers and systems, for instance. A SEM product working with a vulnerability assessment (VA) tool can help you identify those events that are targeted toward systems vulnerable to the attacks. Make sure your VA data is up-to-date and scans are done frequently. This can be challenging when dynamic IP addressing is used, because many VA products use only the host IP address as the unique identifier. Thus, if the workstation's IP changes over time, the VA solution may have difficulty tracking it and redundant scan information will be included.

As for storing the data generated from these events, the length of time to keep the logs depends on various government and industry regulations and standards, such as HIPAA, SOX and ISO 17779. Some of these don't define a retention time, but others specify a minimum of seven years. NIST, in its "Computer Security Incident Handling Guide, Special Publication 800-61," recommends retaining at least several months of data. The North American Electric Reliability Council says audit logs must be kept for three years. To help reduce log storage, some SEM tools can save log data by type. You may want to retain your IDS data for 60 days, for instance, but keep VPN logs for 120 days and application logs for years.

Any SEM tool--or any event collection/aggregation tool--must be able to keep working during abnormally high event counts. IDS/IPS devices and firewalls can generate an enormous number of events during a virus or worm outbreak. You can increase the SEM's availability with multiple horizontally partitioned databases. One database can be used for events generated recently while another database is used for historical analysis. The database used for correlation and reporting may be different from the database used for the short-term events. If everything is stored on a single database, the additional load on the server will hamper its ability to run reports.

You also can ensure information availability using other methods. A fault-tolerant design, for instance, keeps your system operational in the event of a network or physical infrastructure failure. With a fully redundant design, events from each device are sent to primary and secondary event collectors, which then send data to aggregation engines. Ideally, from a network-design perspective, these should all be located near the event source. Additionally, you could have two event-management databases--one at a primary SOC and the other at a secondary SOC.

Log Jam

Most security devices can send their logs to a syslog server, which acts as an event collector for these various devices, but the real integration challenge comes when integrating incident logs with application logs, such as those from a Web server or Web application (for example, someone logging in). The vendors of enterprise-class SEMs offer agents or wizards to help simplify this integration, but you must decide which data elements should be captured and written to logs. Some key data to include are date and time, IP address, action performed and user name.

Information on an event must be analyzed in order to be categorized as an incident. Once an event becomes an incident, the ability to track, monitor and document it becomes critical. You can record all this information on a spreadsheet or in an Access database, but these aren't scalable. Much better is a robust management program from the likes of Remedy.

Policies and Processes

Because the SOC will be the first place security events are collected and managed, there must be a clear and consistent set of processes through which to execute your organization's policies and procedures. (This consistent methodology will satisfy documenting requirements specified by SOX and other regulations.)

Incident-response guidelines dictate how incidents are prioritized, who performs what tasks and when to declare an enterprise-level incident. There are many models a company can use to build an incident-response process, including those from SANS, NIST, Carnegie Mellon University, IETF and RFC 2350.

The basic components of an incident-response process are identification, containment, removal, recovery and post-incident analysis. Most companies tend to do the first four steps very well, but truncate the last or never bother to implement the fixes recommended by post-incident analysis. Thus, companies will find themselves dealing with the same problems again and again.

Part of the process is identifying and training an incident-response team, which will usually consist of dedicated and nondedicated personnel. Having an incident-response team reduces chaos, provides consistency and ensures that all process elements, especially notification and documentation, are followed. In addition to the usual techies, such as firewall and server administrators and network operators, include a representative to communicate to the business side. This will guarantee that accurate and timely information is delivered to system users so backup and contingency plans can be implemented.

Jay Milne, CISSP, is a senior security consultant for a large healthcare provider in Northern California. He has 15 years of IT security experience and has consulted for companies such as Chevron and Genentech. Write to him at jmilne@nwc.com.

It's 6 p.m. All is quiet until you notice a growing number of hits in your IDS. Then you see a mess of trouble tickets from end users complaining about slow network performance. Resources dwindle and become unavailable. No one can get to the intranet or Internet. Servers crash. You've just been hit with an enterprise scope worm. Management, of course, wants answers--now.

Bringing together network staff, security folks, local IT support and management is a challenge, but you're prepared: You have a "war room" designed for such a crisis. The resources are available, and all your "generals" know where it is and how to get to it. Here are the bare essentials:

• Dedicated phone lines. The number of lines you need depends on the size of your SOC. Make sure you have at least two dedicated lines, one for conference calls and the other to make secondary calls.

• A fax line. When my company network went down because of the SQL Slammer worm, we used the fax machine to get remediation instructions out to the field offices.

• A dedicated printer with a local connection.

• Plenty of whiteboards and a large projector screen. Make sure everyone can see what needs to be done.

• A network connection to the Internet separate from your corporate network.

• A secure wireless network. Limit the range of connection, and only allow authorized access. When the war room isn't in use, disconnect the wireless network from the rest of the network.

• A small refrigerator stacked with Red Bull or Mountain Dew. No joke.

1. Does your organization have a SOC?
   
   

2. How well is your SOC staffed?
   
   

3. Does your SOC have additional Internet connections
   outside the corporate network?
   
   

top

top

top

 

Introduction

Computer security technology is still in its infancy. Technologies such as firewalls, antivirus, and IDS have migrated from research labs into production networks, and have become required mainstays both as essential defenses and as legally mandated compliance systems. Computer security systems are complex devices that need to meet a variety of conflicting goals: high performance, fault tolerance, easy administration – and rigorous security processing. Some vendors have staked their claim based on speed, others on cost, and still others on the defensive posture and security of their products. Unfortunately, it’s extremely difficult for the customer to sort through marketing fluff and dubious benchmarks, to determine which products actually work and which merely appear to work. Few customers are sufficiently sophisticated or willing to take the time to do their own testing and most are forced to rely on published results from trade magazines, recommendations from consultants, or industry analysts. Sadly, few of the trade magazines or analysts have the sophistication or time to perform adequate testing, either.

The author’s experience indicates that large numbers of products on the market have excellent and attractive user interfaces, good performance, reasonable costs, rave reviews from loyal fans – and have taken shortcuts in their design that make them significantly less secure than other alternatives. This is not a recent development; it dates back to the early days of the firewall "market." In this article, we examine the evolution of packet filtering firewalls and their current incarnation as "Deep Inspection" firewalls. We compare the fundamental design philosophies of packet filtering firewalls with proxy gateways, and will conclude with a few historical observations regarding the relative effectiveness of conservative design philosophies when compared to their less-rigorous counterparts.

Early Firewalls, Packet Filtering Firewalls and "Stateful Firewalls"

The first firewalls were based on either a proxy design or a simple packet filtering ruleset. The proxy firewall operates by interposing itself in the middle of the application protocol and interpreting it while applying security controls to the application commands and data, where appropriate. The original value proposition of a proxy firewall is that the proxy is essentially a security-oriented reference implementation of the application protocol – in some cases omitting dangerous operations entirely, or providing additional controls on certain security-critical commands. Proxies have always been considered a conservative security design because the proxy reduces the likelihood of protocol backdoors or side-effects since the proxy’s designer is effectively performing a security assessment of the application protocol’s features prior to implementing them. Early packet filter firewalls implemented a simple policy-table lookup based on { source-ip, destination-ip, source-port, destination-port, SYN-seen yes/no } permit or deny. Consequently, packet filters were extremely fast since they did very little computation. They were also extremely easy to implement since they required virtually no security expertise. The simple compute requirements of packet filters, and the fact that they required no security knowledge-base, made them easy to implement in silicon so they quickly became a feature of most routers. From the beginning, proxy firewalls were recognized as being more secure, because they effectively are implementing a correctness check upon the application protocols they gateway. This is still an important property of proxy firewalls. For example, when the author first implemented the FTP proxy in the DEC SEAL firewall, he simply left out unused FTP protocol commands that allowed users to issue remote commands to the FTP server. Years later, when hackers discovered those commands and attempted to exploit them, they simply did not work against proxy-protected networks because the proxy refused to gateway the command through to the target. Sites behind packet filtering firewalls were vulnerable, if the reachable systems behind the firewall were themselves vulnerable.

In 1993, "stateful" firewalls appeared on the market. The first popular stateful firewall, Checkpoint’s Firewall-1, implemented a simple connection-origin table that tracked whether a connection had originated behind the firewall and permitted response packets for that connection. A layer-7 hook to parse FTP PORT commands and update the state table allowed FTP to work transparently through the firewall. Subsequent versions of the stateful firewall added TCP sequence number interpretation, and DNS query/response matching to ensure that return packets were only allowed in response to queries that had originated from the inside. It is important to note that stateful firewalls added these features to overcome vulnerabilities in their design – attacks such as TCP RST flood attacks and DNS cache poisoning. Proxy firewalls never had these kinds of vulnerabilities. Stateful firewalls have continued to evolve; often in response to new types of hacking techniques as they have been discovered. Proxy firewalls have evolved, as well, but mostly in response to ever-higher requirements for performance and transparency.

Near-Term History of Intrusion Detection, Intrusion Prevention, and Firewalls

Intrusion detection technologies (IDS) have enjoyed a spectacular rise and fall. Since Dorothy Denning first proposed the concept in 1986, there were relatively few products until 1997, when a number of vendors began offering commercial network monitoring IDS. The IDS market grew rapidly through 2002, when Gartner researchers announced "IDS is the ‘pet rock’ of computer security" and recommended their customers focus instead on the up-and-coming new technology termed "Intrusion Prevention" (IPS). According to Gartner analyst Richard Stiennon, IDS was a failure for two reasons. First off, IDS generated too many "false positives" – alerts in which the IDS mistook legitimate traffic for an attack. Secondly, IDS didn’t actually do anything but generate an alert when the attack was seen. Stiennon rightly pointed out that, if you can correctly detect an attack, it would be nice to prevent it. By the end of 2003, virtually every IDS vendor was re-branding their product as an IPS, by adding in-line blocking modes to their sensors. Initially, virtually no customers used the in-line blocking modes of their IPS, because they were afraid of traffic interruptions.

Some of the first IPS on the market were not effective security devices: they used simple signature-matching techniques and "knew" how to recognize and block only a few dozen worms and popular attacks. They did not sell particularly well until the massive worm outbreaks of 2004: corporations were desperate to contain worms in their networks, and worms are sufficiently easy to recognize that the IPS did a decent job blocking worm traffic on the network. IPS began to sell. The feature-similarity between IPS, IDS, and firewalls as well as single-purpose spam and anti-virus gateways, has triggered a move to consolidate these technologies into a single platform. A number of vendors have embarked on a course of technology acquisition, with the intent of embedding IPS/IDS/Firewall/AV/VPN termination capabilities into switching platforms. These combined systems offer an attractive feature-set, high performance, and a good price point when compared to having to integrate a series of separate products. The strengths and weaknesses of such systems will depend on the quality of the underlying detection and firewall engines and its IDS knowledge-base.

Enter "Deep Inspection"

According to David Flynn, NetScreen’s Vice President of Marketing, a survey of 1000 network managers indicated their biggest security concern was "depth of protection" against worms, trojans, email viruses, and exploits against software vulnerabilities. Flynn is quoted as noting that these types of attacks "can traverse a traditional stateful firewall even if the firewall is deployed and working as it should be." Both NetScreen and CheckPoint have bolstered their stateful firewalls by adding application-oriented checking logic into their processing modules – essentially merging IDS signatures into the firewall traffic-processing engines of their products. What is "Deep Inspection"(DI) anyhow? According to NetScreen, DI firewalls "use an Attack Object Database to store protocol anomalies and attack patterns (sometimes referred to as signatures), grouping them by protocol and security level (severity)" Packet processing is typically described as "performing application level checks as well as stateful inspection." In other words, a "Deep Inspection" is just a catchy marketing term for a stateful packet inspection firewall with some IDS signatures and some application protocol anomaly detection rules. It is important to understand that a "Deep Inspection" firewall is going to provide all of the protections of a stateful firewall, as well as whatever signatures are loaded into it.

In the case of a NetScreen (circa 2003) the firewall "is designed to provide application layer protection for the most prevalent internet-facing protocols, such as HTTP, SMTP, IMAP, POP, FTP, and DNS." It is ironic to remember that one of the early complaints about proxy firewalls was that they only supported a small set of application protocols – what good is a DI firewall that only does DI on 6 of the "most prevalent" internet-facing protocols? That may be "deep" inspection but it’s certainly not very broad. When the NetScreen product was initially released, it included signatures to detect "over 250 application attacks." As far as an IDS signature set, this does not compare favorably to an IDS such as the open-source Snort, which at that time had signatures to detect over 3,000 attacks. Much like the early versions of the Snort IDS, NetScreen’s DI engine uses regular expression pattern matching to specify its signatures – a technique that most serious IDS product designers stopped using years ago because it is too simplistic to represent complex network/application states effectively. Regular expressions, however, are easily processed in silicon, using off-the-shelf regex engines that provide hardware speedups for matching, so it makes sense for a switch-based DI firewall to rely upon them.

Designers of DI firewalls will probably not be able to provide sophisticated IDS engines in their products because of the performance overhead necessary to execute complex signature analysis and state tracking. The history of the IDS market shows that IDS has always been a performance-sensitive proposition – which is why most line speed IDS appliances are using multiprocessor main boards to keep up with the load. The author suspects that, unlike IDS, the DI firewalls will only be looking for a small subset of the attacks that are known at any given time, due to performance concerns. Even the "only look for a few attacks" strategy will eventually fail; Gartner reported that last year that 70 new vulnerabilities were identified per week, or over 3,500 per year. If even 10% of those are vulnerabilities that replicate across network data, the firewalls are going to very quickly have to handle very large signature-sets. Hardware acceleration will help somewhat, but typical hardware implementations in the fast-path of a switch are going to have built-in limitations on how many signatures they can effectively load at one time. DI firewalls will eventually have to deal with all of the complexity issues that caused Gartner to declare commercial IDS a "dead end."

Protocol Anomaly Detection Meets The Proxy

Protocol anomaly detection is an IDS technique in which rules are written that track the states of an application protocol and look for deviations from a proper state. A simple example of a protocol anomaly detection rule for FTP might be "if the user name given during the login process is longer than 45 characters, alert that as suspicious." It is possible to very tightly track the process of a protocol, at a cost of increased computing and memory use. For example, the NFR IDS protocol anomaly detector for SMTP tracked the entire SMTP command series from start to finish and would generate alerts if the client issued a "MSG From:" command before a "RCPT To:" command, or issued a "HELP" command, etc. Such tight protocol tracking is extremely powerful, because normal applications always send a message using the exact same sequence of steps but human hackers sometimes get the order wrong or attempt to trigger errors by issuing commands deliberately out of sequence. Protocol anomaly detection is a very powerful technique that is used to great effect in all commercial IDS and most DI firewalls. For detection accuracy, protocol anomaly detection pales in comparison to the processing that is performed during the normal functioning of a proxy firewall. This is because the proxy, as part of its normal operation executes the protocol. Instead of having to follow along and try to figure out what the application protocol is doing, the proxy is the application protocol: protocol anomalies represent error conditions that the proxy detects. When the author implemented the first proxy firewall, the FTP proxy logged and error and shut down if, at any time, it received commands that were out of sequence or were incorrectly formatted.

Another topic of debate in the IDS and DI firewall community is the question of "detecting the exploit" or "detecting the vulnerability." When "detecting the exploit" the signature-writer develops a signature that will identify the particular attack tool. For example, the TESO mass-rooter for FTP daemons uses a specific set of buffer overruns against a variety of FTP servers. Coding to detect TESO, the signature-writer would write a minimal signature to identify the first steps of a TESO probe and generate an alert. To "detect the vulnerability" the signature writer would research the entire set of vulnerabilities that particular FTP servers might have, and then would develop a signature set that detected any one of them being exploited. Detecting the vulnerability takes considerably more research since it involves a larger number of signatures to develop. For the proxy firewall developer, this is a complete non-issue. If a new vulnerability is identified in an application, the proxy-writer assesses whether the attack vector is "legal" in the protocol, and, if it is not, the proxy wouldn’t have let it through in the first place. In the case where the attack vector is encompassed in the protocol, the proxy can be updated with a protocol-specific signature at the exact point of the protocol processing where it’s appropriate. In the long run, this approach is much more efficient and takes less work – both in terms of effort to keep the security knowledge-base up to date, but also in terms of processing speed. The proxy only needs to execute protocol-specific checks at the exact part of the protocol where they are needed, whereas a regular-expression based signature engine must compare each expression against each input as it arrives.

Some Philosophy About Security Design

Fundamentally, the tension between proxy firewalls and stateful filter + signature firewalls is a reflection of the deepest philosophic division that in security. It is a division that goes back to what we used to call "stance" back in the early days of firewalls:

• That which is not expressly prohibited is permitted (default permit)
• That which is not expressly permitted is prohibited (default deny)

The default deny stance is the default policy that virtually every credible firewall product executes unless told otherwise. In security, however, there are many places where we do not follow the rule of default deny. For example, antivirus products implement a "default permit" – "if you don’t know it’s a virus, then it’s OK to run it." IPS implement default permit – "if it doesn’t look like an attack, permit it." Fundamentally, default permit is not good security. A proxy firewall implements default deny. Unless the application that is talking to the proxy complies exactly with the protocol as executed by the proxy, the traffic is not going to go through. It’s not perfect, of course, since there might be an attack that is valid within the protocol, but is still damaging. That is why most proxies also add additional application specific signatures.

For example, the TIS Firewall Toolkit’s HTTP proxy not only mediated the entire HTTP transaction it performed URL sanitization and did not permit unusual characters in URLs. Several of the authors’ friends who still use components of the firewall toolkit were highly amused when a piece of software that was coded in 1992 successfully defeated worms in 2002 – 10 years before the worms had been written. Similarly, the DEC SEAL’s FTP proxy from 1990 defeated FTP bounce scans when the hacker community discovered them in 1995. Peter Tippett, the author of Norton Antivirus and founder of ICSA/TruSecure once famously commented "The current philosophy of antivirus is to know the 50,000 bad things that can happen to you. It’s much more sensible to enumerate the 50 things that you want to allow to happen, but the industry just isn’t ready to think that way, yet." Perhaps that is true in the antivirus arena, but the proxy firewall’s approach has always been:

• Nothing gets through unless it looks right

Rather than:

• Everything gets through as long as it doesn’t look bad

Security products that adopt the default permit philosophy will always be fundamentally at risk from new attacks. A signature-based DI firewall can only block attacks that are known by its signature set. When a new worm breaks out that takes advantage of a never-before-seen attack, the signature-based DI firewall will let it through. Once the signature set is updated, the firewall will be able to block the attack, however, the time between the release of the attack and the development of the signature might be a matter of hours or even days. While that window of vulnerability is open, the customer will have no alternative but to either patch host software or disable that service.

Historical Performance of Security Solutions

The history of computer security shows that many customers do not want to take the time to fully understand what they are buying. In the mid 1990’s the author was selling proxy firewall products that had a superlative history of resisting attack; yet the market leading products were simplistic "stateful" packet filters that were sold based on the fact that they were faster, cheaper, and more forgiving. Put differently: they didn’t perform as rigorous checks, so they could be fast. They were easier to code, so they were cheaper. They were more forgiving, because they were more permissive. There is ample anecdotal evidence that default deny security survives better in the long term than default permit. There is ample anecdotal evidence that default deny security is less susceptible to nasty surprises, and requires less patching. Systems that are based on the model "recognize a vulnerability and block it" inherently require "upgrade to fix the vulnerability of the week."

Customers need to understand their objectives and requirements, so they can best select technology that facilitates their mission. There are situations and circumstances where less-rigorous security systems can get the job done safely. The author believes, however, that few vendors take the time to explain the design tradeoffs inherent in their products to their prospective customers. With security being such a hot topic, it’s hard for a customer to make sense of a plethora of products, all of which are asserted to be "secure." Analogies are dangerous, but choosing a security product is similar in many ways to buying a car: the customer needs to have a good idea of what kind of on-road properties are needed. Does it need to be economical and corner well, or does it need to be able to tow a 6 horse trailer safely? Or, perhaps it needs treads and 3" thick armor. What is frustrating to the author is that in the computer security space, there are some vendors asserting that what they offer is a general-purpose high fuel-efficiency sports-car main battle tank. Can you believe that?

mjr.
The speaker break room, Interop (Mandalay Bay Hotel) and McCarran Airport Gate 33, Las Vegas, May 6, 2005

---

Appendix 1: Summary Matrixes

Table 1: Pros and Cons of ASIC-based platforms V. General Purpose mainboard
Packet Processing speed	Advantage: ASIC Platform Notes: Generally ASIC-based systems take advantage of off-the-shelf silicon for "fast-path" packet processing. Fast-path processing represents "best case" throughput; real throughput is often dramatically slower when running application logic (such as security processing beyond simple packet-filtering and state table updates)
Matching performance	Advantage: ASIC Platform, up to a point Notes: off-the-shelf regular expression processing silicon is available and offers dramatic performance speed-ups for simple pattern matching. With most such hardware implementations the pattern-matching engine provides acceleration for a limited number of expressions in parallel; once this number is exceeded performance drops to un-accelerated levels because the higher-level controls need to "swap" in and out pattern-match sets. When this occurs it is usually no longer possible for the matching engine to run in the "fast path" so not only does matching performance suffer, packet processing performance can be degraded as well. Recommendation: Always evaluate such products using a ruleset that is slightly larger than the expected set of real-world signatures. A realistic test of an ASIC-based packet searcher should include ~3000 rules – approximately ½ the rules in the current Snort IDS signature set. Be extremely wary of performance figures in which the ruleset size may have been adjusted to fit within an optimal silicon space for the processor.
Signature set size	Advantage: General-purpose mainboard Notes: Devices based on general-purpose mainboards are generally more tolerant of larger rulesets since they can be expanded with additional RAM as necessary. General-purpose operating systems usually include virtual memory management algorithms and paging kernels, which allow virtual extremely large datasets and rulesets to be resident without performance impact.
Signature update	Advantage: General-purpose mainboard Notes: Devices based on general-purpose mainboards usually can take advantages of the presence of hard disk interfaces or removable media and are not space-constrained when storing administrative configuration data.
Search Space size	Advantage: General-purpose mainboard Notes: ASIC-based systems generally are designed without a hard-disk-backed virtual memory architecture, and consequently are unable to do long-term storage of information relevant to detection signature logic. This sometimes reveals itself in the form of hard capacity limits in ASIC-based devices that are not present in general purpose O/S-based systems. Such capacity limits might present themselves as hard limits on number of simultaneous sessions that can be tracked, maximum number of out-of-sequence packets that will be correctly re-sequenced, etc.
Storage	Advantage: General-purpose mainboard Notes: Most security products based on a general-purpose mainboard take advantage of the presence of a hard disk to store log data, event records, and security alert information. Persistent hard disk storage prevents loss of event/alert information if a network is partitioned temporarily.
Software Update	Advantage: General-purpose mainboard Notes: General purpose operating systems usually include a notion of software component architectures, allowing easy update of program logic, installation of patches, and operating system updates.
Hardware Update	Advantage: General-purpose mainboard Notes: The upgrade path for most ASIC-based systems consists of "buy a new one" while general-purpose mainboard based systems can be component-upgraded with additional RAM, hard disk, or a faster processor. Because ASIC-based systems can seldom be component-upgraded customers may encounter a "hard limit" in which one component of the system reaches capacity and the entire system must be replaced. I.e.: "the signature that broke the camel’s back."

 

Summary observations: ASIC-based solutions are appealing to the customer for whom raw speed is a paramount consideration. Generally, where there is a choice between "doing things fast" and "doing things securely" the design decision of ASIC-based products is to "do things fast." This, necessarily, affects the system’s quality from a security standpoint. The author’s experience is that custom hardware-based security products tend to look more like "a switch with a bit of security bolted on" rather than "a security product that is really fast."

 

Table 2: Pros and Cons of Signature Security Processing V Proxy Security
Signature count	Advantage: Signature-based system Notes: Unlike signature-based systems, proxies do not derive their security through enumeration of attacks or vulnerabilities. Many proxy firewalls support signature/expert rule detection in addition to the proxy’s correctness check. Signature counts are, in other words, not applicable to a proxy firewall. However, many customers attempt to make the comparison.
Protecting Against Unknown Attacks	Advantage: Proxy Notes: Signature-based systems utterly fail to prevent unknown attacks. They may prevent new versions of previously known attacks, or new attacks that happen to trigger an existing signature. Signature-based IDS may detect protocol anomalies, but signature-based firewalls (AKA "deep inspection" firewalls) do not automatically block all the things they prevent; they generally only block things they can be relatively certain represent hostile traffic. Observation: It is important for customers to understand the difference between "signatures the signature-based firewall detected" and "signatures the firewall was confident enough to block traffic based upon." For example, when the Intruvert IPS first came out, it shipped with only 50 signatures (!) of which none were configured to block traffic. The product’s designers were so concerned that false positives would cause the system to automatically shut down legitimate traffic that they only eventually enabled automatic blocking on fewer than 1 dozen high profile and easy-to-detect threats.
Performance	Advantage: neither Notes: As signature-based systems begin to load up on signatures their performance can rapidly degrade. Proxies do not have signature loading properties, so it’s difficult to make a comparision. Recommendation: Customers should perform carefully constructed head-to-head comparisons based on real loads and realistic, useful signature set-loads.
Service Support	Advantage: neither Notes: One of the historical complaints against proxy firewalls is that they do not support as wide a set of services as a packet screening "stateful" firewall. With the addition of signatures to stateful firewalls, the stateful firewalls do not have signatures that perform meaningful security on any but a handful of protocols (e.g.: HTTP, FTP, DNS) The same complaint that used to be levelled against proxy firewalls now applies to "Deep Inspection" firewalls.

  Summary observations: Proxy firewall technologies have proven time and again to be more secure than "stateful" firewalls and will prove to be more secure than "deep inspection" firewalls. The main comparison point between stateful firewalls and proxy firewalls has typically been on performance, which is an orthogonal property to security. High-performance proxy firewalls are available, and some are capable of easily handling gigabit-level traffic. Customers need to be realistic about whether performance is really an issue in the expected deployment, and should be prepared to do operational testing. The author has seen many customers cheerfully settle for inferior security because of "performance concerns" that did not actually apply.