iSCSI Security Part 2

To tie down my iSCSI test environment I've implemented IPsec between client and server. This allows me to encrypt either the traffic or headers of my IP stream. I chose ESP (Encapsulating Security Payload) as this gives full confidentiality to my data, rather than Authentication Header (AH) which provides integrity as to the source of the data.

Implementation on my Netapp Simulator and Windows client was easy; the ipsec command, a file setting on the filer and configuration of the IPsec settings through MMC on Windows. Once I'd dropped and relogged in the iSCSI targets, I did some testing.

Now before I go further, let me stress that this testing was only slightly scientific. Everything I'm using is virtual and on the same physical machine with one hard disk. The figures I'm quoting are indicative of the performance I received and not directly comparable with real-world systems, although you'll get the idea.

So I tested with HDTach and no IPsec enabled. Response time for a standard test was about 0.2ms and throughput ranged from 2-32MB/s over the course of the test (an average of around 17MB/s). With IPsec enabled, response time doubled to 0.4ms and throughput dropped significantly to 2-7MB/s with an average of 4.2MB/s. I repeated the tests a number of times with IPsec on and off and saw consistent results.

Just to be sure, I checked for other bottlenecks in my system. The virtualising PC was not bottlenecked, neither the filer or client. I can therefore put the performance change down to simply "more work to do".

Unsurprisingly, I couldn't check the IP packets with IPsec enabled. Although this is secure, it presents issues when diagnosing problems. I don't know whether tools exist to enable encrypted streams to be analysed, but Ethereal couldn't do it. My IPsec implementation was also rather simple - just a shared key for encrypting the traffic. If I was implementing this for real, I'd be discussing with the Network guys the best way to implement security.

iSCSI Security Part 1

I've done some more work on iSCSI. That meant rebuilding one of my iSCSI client environments as I totalled it by trying to add another drive to the VM guest. Anyway, panic over. I've got the client working again and I've been running traces to see how easy it is to locate relevant data in an unprotected iSCSI stream.

I mentioned before that I am using iSCSI on Win2K3 talking to a Netapp filer (simulator) with Ethereal on the host Win2K3 box to monitor I/O activity. I tried today to write to a test file with Ethereal running and tracing I/O activity. The tool is splendid in its interpretation of the data it sees, formatting the packets on the basis of the content. My save of a file containing some test data yielded some interesting results.

Firstly, I found Ethereal detected the logfile entries for NTFS being saved before my actual file. These are RCRD and RSTR records which NTFS uses to recover the filesystem should anything untoward happen before the data is actually committed to disk.

After this, I tracked the MFT entries being written. These are the actual file saves which contain my data. Non-encrypted I can see the content. Using Ethereal and a tool to map NTFS records, I could easily spy on data being stored on iSCSI volumes. Here are some screenshots:

The first shows the MFT entry that NFS stores for the file save. I used Winhex to examine the blocks on disk. Note the content of the file "The Quick Red.......", plus the file name "TestFile1.txt" in unicode, so using double characters. Contrast this to the Ethereal output which was scanning the network. This shows the same data captured by scanning the network. Ethereal is generous enough to even format the data to allow me to locate the iSCSI data. This is no good for any organisation which must offer data security. Part II will discuss what can be done.

iSCSI Continued (2)

After my previous post on iSCSI testing I promised some more detail. So my test environment is based on the Netapp Simulator version 7.2.1, which you can download if you're a Netapp customer - not sure if it's available to non-customers (it should be as it is a great marketing tool) but I guess if you want to find out you could ask Dave.

Netapp filers export LUNs as iSCSI devices. As far as I can tell, the LUN implementation on ONTAP is effectively a qtree (i.e. a share) based on the way I created it. Anyway, once created, I associated the LUN with an initiator group and the initiator group has the access associated with it (hope you're all following this). The screenshot here shows the output from the LUN show command and igroup show command which list the LUNs I created and their association. You can see the igroup can be used to provide access to a number of servers based on the iqn, iSCSI Qualified Name, which is used to reference an iSCSI target or initiator device. The iqn seems to be a "gentlemens agreement" format, based on the reverse DNS of the server on which the iSCSI device resides, plus the date and month that domain was registered. In this case I registered the test server I have the iSCSI initator on and I got "iqn.1991-05.com.microsoft" followed by the specific server identifier of "vmware2.vmware.brookend.com", the name of my server itself.

My first inclination (being a hacker of old) was to spoof this, so I configured another server (XP this time) with the iSCSI initiator and changed its iqn. Voila I can access the same disks, albeit from another IP address. Being iSCSI and block data, the filer simulator didn't care about multiple access (which is fine) and I spent some time trying to break the shared LUN writing and reading data from both sources.

Needless to say this example highlighted how simple security does (or doesn't) work. I know CHAP authentication is available and I suspect there are many more security options that I need to investigate, so that's my next thing. Getting standards and security right I think will be more important than making sure the network performs.

More details to come.