March 2008 Archives

If I were to start over with computing, I would make a number of fundamental changes:

1) ALL hardware should be hot-pluggable. I don't mean just USB and the like. All memory, expansion cards, storage, and other parts of the computing architecture should be able to be swapped out without powering off the system. In my thinking, this would require a small core system to be at the center of everything. It would manage all hardware and provide whatever OS with a basic functionality at incredible speeds. The base system should be nearly unusable as a full computer, but divert processing power, I/O, and memory storage appropriately to whatever modules are installed.

2) The architecture should allow for multiple processors of different types. Today, the x86 processor is nearly ubiquitous and is pretty much the most generic or general-purpose processors. However, in the average PC today there exists a number of speciality processors that help accelerate other tasks. The most notable of these is the GPU, which is effectively a processor many times more efficient than the main CPU, but useful only for a handful of commands. In future computers, I would like to see all processors treated equally through a processor abstraction layer, which allows for an arbitrary number of different types of processors, with a scheduler dividing work appropriately among the processors most suited for the work at hand. I would like to see this include FPGAs, which would be dynamically reprogrammed based on what work is being done. An average system would consist of the core components and processor (which would act as the manager/scheduler), a faster general-purpose CPU (possibly multi-core), and one or more FPGA or other specialty processors.

3) The whole platform should never need to be turned off or rebooted. The core system should be able to manage changes to hardware seamlessly and drivers should work at an abstracted layer which doesn't require the whole system to start over whenever a change is detected. Even in the case where processors are swapped out, everything should fall back to the core system whenever modules are removed. This doesn't mean that you shouldn't properly prepare the system for a change. It is unreasonable for the system to catch a physical disconnect and handle it for some components. For example, in the case where RAM is being swapped out, the contents of that memory need to be pulled and placed elsewhere before the system could handle such a change.

4) Everything should be abstracted. Absolutely every piece of everything should run through an abstraction layer. This layer should be running mostly in the aforementioned core system. This allows the OS developers to concentrate on developing a secure, extendable, and usable OS. APIs should be provided for hooking at all different levels so that third party providers can enhance most everything. Ultimately, tasks that, today, require a complete rebuild, should require almost no effort. For example, if I were to desire to move from a single HD to multiple HDs in RAID 5, I would have almost no choice but to start over from scratch. Similarly, upgrading a motherboard is usually catastrophe without reinstalling the OS. This should all be as simple as swapping out a keyboard or mouse is today.

5) Security. TPM, despite many cries to the contrary, is a pretty good idea in terms of keeping unwanted code from running on your system. Unfortunately, the execution this far has been poor, and not much has been done to fix it. Code signing and strict policy (followed up with enforcement) can help significantly cut down on malware. The issue is, they system needs to do several things to keep up. Firstly, it needs to be flexible enough to adapt quickly to emerging threats. Secondly, system needs to be set up with the right tools (like TPM) which enable it to effectively secure itself. Lastly, the system needs sane defaults. Most systems historically ship with defaults which leave them incredibly exposed. This is usually done with the excuse of "we want it to work with everything out of the box". That line of reasoning is not necessarily at odds with security, and vendors need to embrace security as a feature that users expect to work out of the box just as much as any other.

6) Automatic updating of everything. While this has been around for years in the BSD and Debian worlds, it's not something that many other platforms have picked up. The reality is that all software these days is constantly being updated for security and functionality issues. Updating software outside of the main OS in Windows, most unicies and OS X is hit and miss, with many vendors doing their own thing, or nothing at all. This kind of inconsistency leads to either a lot of effort going towards keeping things current, or to a lack of updates. Some OSs (like Debian-based Ubuntu) do a relatively good job, keeping everything installed through the package system (which is usually almost everything) updated automatically. For any new platform, I'd say this is a must. As a side note, all software should be in package form. Installing and uninstalling should be as simple and consistent as possible. Uninstalling should actually result in a clean system, and should be friendly to configurations.

7) A mouse-and-window-based GUI is pretty much standard these days, and there's not much on the horizon to be supplanting that for foreseeable future systems. But, as with developments in the last several years, there have been enhancements in usability. Probably the most widely usable OS out there is Mac OS X. Sure, argue with me if you will, but of the major OSs, I think OS X is the most friendly. There are certainly things that could be done better, and future systems should really be built around them. Firstly, I should never, ever, ever be able to outrun any UI I'm using. There's just no excuse for that. Secondly, windowing needs to work in a predictable way, and that model should never be broken. All too often, focus is stolen or ordering refuses to operate the way it should. Consistency here is of the utmost importance, and should be enforced.

8) Backups. Apple wins again at making backups easy with Time Machine in OS X 10.5, but more needs to be done. I need to be able to take an entire system and move it through time as a whole or in pieces, all the way back to its first boot. Backups should be able to be placed on any storage, whether directly connected, or on a network or the Internet. Backups should be able to be reloaded on different hardware in case of emergency. The backup images should be encrypted so that they may reside anywhere without concern for compromise.

9) Mobility. Systems of the future should allow you to move anywhere and do anything. Ultimately, I'd like to be able to feel as if I'm using the same computer anywhere I go. This can be accomplished using Terminal Services or a similar type of screen-sharing architecture, but that has serious limitations when it comes to certain applications. What I'd really like is for the OS and the hardware to become user centric.

10) Ubiquitous, high-speed, wireless internet access. Whatever system I'm using, it should always be connected, and always at a high speed. This seems like it might be coming to fruition with the recent developments in the 700mhz and TV white space bands which Google and the like are trying to leverage for wide area data use. Such a network, with speeds approaching those of wired broadband, would enable future systems to be constantly connected, which directly supports my previous item.

After all of my above ranting, what would my dream system look like? Well, let's start with my iPhone. This resembles what I would call a core system. It carries with it enough storage and processing power to be useful, but is quite weak compared to that of a desktop. When I go through my day, I transition from one to the other, to another. I'm spread across several machines, and have to put a decent amount of effort in to keeping them in sync. My perfect system would be a single device, which I would carry with me. Wherever I desire to do some work, I would simply plug it in to a dock. Through the dock, the core system would access more storage, faster network connectivity, enhanced processing power, and better displays. All in all, it would be the same as using a desktop, but I would be able to pick it up and walk away with it, without having to worry about anything. When I get home, I would take the core system, and plug it in to a laptop-shaped dock, which would provide additional power, memory, processing power, and display.

Perhaps I'll see my fantasy come true as a whole some day, but for now, I'm thrilled to see many of these developments on the horizon.

You very simply can't fix what you can't see. This is something that nearly everyone in the IT field struggles with at some point. This is largely due to the lack of proper tools being used to manage the network (see part 1 about consistency). This is also usually the fault of network structure.

Data flows around a network can be difficult to observe and control except on the edges, where there are devices like firewalls and routers which are made for this exact purpose. Monolithic (or flat) networks are particularly bad in this case because of the lack of borders between parts of the network which should be separate. When dealing with such a network, you only have a few options if you are interested in watching the traffic flowing internally. The first is to have a large number of span/monitor/mirror ports which copy the traffic from the ports to which your network devices are connected. Unfortunately, this breaks down rather quickly when the number of used ports and the bandwidth used gets to be too high. The second option is to use Cisco's NetFlow (or similar). Of the networks I've seen, many do run Cisco switches, so this is a viable option. For those running lesser switches, you're pretty much stuck with mirroring ports, or with nothing. Of course, there is always the option of installing a physical tap for every single port, but that's pretty well unreasonable for the average organization.

In my opinion, the best way to increase security and visibility simultaneously is to separate your monolithic networks into smaller, more manageable subnets. Generally, these subnest should be cut along the lines of security, physical, and logical lines. Which mix of these will work best for your organization is something you'll need to consult a security professional to find out. In any case, once the network has been divided, the only way data should flow between any two subnets is through a firewall. Most modern firewall appliances come with the tools necessary to watch and manage the data flowing through them. Some of the better ones will provide tools for visualization and reporting to make understanding these flows easier and more efficient. Honestly, I can't get enough when it comes to tools which crunch otherwise boring data into neat little pictures, so long as detail is available when I want it.

Another incredibly helpful pile of data that your network generates is log data. The problem is that logs are tedious and boring. That's why there are log analyzers like Splunk which give you a nice web interface and search/index functionality which helps turn your log data into something useful. One of the best uses I've heard of for a log analyzer is anomaly reporting. Basically, you run your network for a while and let the analyzer collect log entries. After enough has been accumulated, you go through and mark all the normal data as being so. Once you have tagged the data which you expect to see, you can very easily filter for what you don't expect. This effectively sifts out errors and other unexpected messages from a sea of information.

Of course, doing any of the above assumes that you have time in the day to make it happen. Unfortunately, it seems to be incompatible with IT departments which work behind the curve of break/fix. 

Most organizations don't start out with the resources or immediate need to properly roll out imaged workstations or manage their networks using high-level tools. What happens more often than not is that the network is a collection of very different PCs, all of which have been built and configured individually. As a result, there are as many variations in loadout as there are workstations. As the organization grows, this practice is scaled to meet the need, and results in an IT department which works mainly in fire-extinguishing mode.

There is a certain point at which more consistent control over the network is an absolute must. The problem is that, in my experience, many medium-sized networks are being run without the proper tools. Management tools, strict domain policy, and imaging are all incredibly useful when combatting sprawl and growing pains.

Consistent network management not only makes IT staff more productive, but also allows for self-auditing and quick response to security events. Patching becomes easier (and can be more comprehensive than just WSUS or Windows Update), which allows the very common issue of third-party software vulnerabilities to be addressed in a more elegant and effective way.

Let's take a look at the basic functions that these tools need to help accomplish in order to be useful. The first is enumeration, cataloging, and inventory control. Basically, you need to know what each device is, does, and has on it. By the same token, you need to know what is on your network that shouldn't be. In addition to a simple understanding of the devices on your network, a good management system will keep track of when devices were purchased, what kind of warranty/service plan they have, and what their serial numbers are. Having all of this information in one place is the first step towards getting your network under control to the point where securing it becomes an easy task.

The second tool in your arsenal should be a package/file management tool. In order to properly manage software updates, it's important to know the loadout of each machine, to be able to verify that ONLY the software you approve is installed, to be able to perform updates to your software, and to be able to remove unauthorized installs. Many of the issues I see in security assessments come down to good patch management. I'd estimate that at least half of the vulnerabilities are a result of old software which has updates available. For those thinking that Windows Update or WSUS will cover all of these issues, you are sorely mistaken. Many of the vulnerabilities I find in assessments aren't in Microsoft products, simply because they do get automatically updated. More often than not, it's old antivirus or backup software which runs at system privileges that provides a way in. 

Overall, being consistent usually comes down to putting a bit of initial effort in to a structure, then being disciplined enough to follow it. Automation of the boring, repetitive tasks makes this worlds easier. Manually going to every device in your organization and checking for updates is very tedious and gets frustrating rather quickly for the high-IQ types who usually make up an IT department. This leads to mistakes being made and eventually to the practice being missed or avoided.

Find some good tools, become intimately familiar with their innerworkings and behavior, then use them as an extension of yourself in confidently controlling your network. Get things to be consistent across the organization, then you can begin to manage one set of common machines instead of many individuals.

I'll use, as a base, the definition that the FFIEC publishes:

More often than not, we (White Badger Group, specifically our sales people) end up having to reeducate potential clients as to what a "penetration test" is. Why is this? Well, the problem is twofold.

Firstly, we deal mainly with financial institutions. These organizations are governed and audited by the FDIC, NCUA, OCC, and other agencies at the federal and state level. If you press any of these agencies for details on what kind of security assessment they look for, and the details on what such an assessment should include, they all refer to the FFIEC. The FFIEC publishes a handy series of booklets which details the process of auditing banks and credit unions. As a matter of fact, I often refer to their IT Information Security Examination booklet (p. 89) when asked to define the different kinds of assessments. The problem comes in when the FDIC and NCUA request banks and credit unions to have an assessment performed. Here's an example of one of the NCUA's documents telling a credit union to get a "comprehensive attack and penetration audit":

I looked through what documentation the NCUA and FFIEC make available publicly, and found that term nowhere. If anyone from the NCUA is reading this, I'd be very interested in why this term was chosen over the ones which are clearly defined by the FFIEC. From talking to many banks and credit unions, it seems that the issue is very consistently inconsistent across the industry. One auditor will ask for a "penetration test," while another will send out a document like the one above.

The other issue is that the vendors offering security services are generally unqualified, and use the incorrect terminology. We have come across many IT hardware/software/service companies offering "penetration tests" in a bundle of 4 for $1200. What they're really selling is a vulnerability scan, with zero human interaction (except for printing the report and putting it in the mailbox). This ends up being a great disservice to organizations in need of real security help, and muddies the waters for those of us out there who are trying to do things correctly.

When it comes down to it, the only people who should be performing a security assessment are:

All too often, we come in behind other IT firms who do general IT work (install servers, set up networks, etc.) who had performed a security assessment. Absolutely every time, we have found major issues which should have been found by the other firm. Some of these have been really, really bad; the kind of bad that puts the organization out of business if it had been exploited.

Know what you're buying. A penetration test is a very involved and expensive process when done properly. You can't get one done for $500, no matter how hard you try. If someone claims to be able to do so, then it isn't a penetration test.

In the course of doing business and keeping ourselves current, we do a good deal of internal training, research, and testing. We also come across many real-world scenarios which can be learned from. This blog will act as an outlet for our staff to comment on our findings as well as current events in the industry. We also hope that this will act as an open forum for feedback on our musings from which everyone can benefit.

None of us here at White Badger Group are professional writers/bloggers, and this is the first attempt any of us have made at such an endeavor. Constructive criticism and comments on our structure/style/content is greatly appreciated.

For those interested in our services, please visit our main site: http://www.whitebadger.com