Technically, it’s a marvel. The geek in me loves it.

However, I have the feeling it’s not yet ready for general use. Don’t get me wrong, it’s wonderful as it is for education, but I think a lot of people are going to use them as general purpose computers, media centers, etc.

The main reason is software availability. While there is a graphics accelerator build into the board, the drivers for the windowing system don’t exist yet. That means that for web browsing, and other general tasks, the graphical use interface is very sluggish. A typical user would get very frustrated trying to browse the web. And there’s not Flash yet, but Adobe is rumoured to be working on it.

Also, to use it as a media centre, XBMC have made good progress on getting hi-definition video working on it, and I’ve seen a 1080p stream running on it. It’s impressive, but XBMC is only just out in beta, so it’s still rough around the edges.

But for the hobbyist, there’s a world of possibilities. GPIO boards are starting to appear, so it’s only a matter of time before we see Raspberry Pi’s controlling a whole world of gadgets….

Here’s some more ramblings on 10G Networking on the Cheap.

Having gone to all the trouble of getting a 900MB/sec SRP target working, I was interested to go back and see how a samba mount would perform now that I’d sorted out all the cards into their optimum PCI-express slots.

So, I did the same setup as the 900MB/sec SRP test, Ramdisk to Ramdisk. This time, I set up a 2Gig ramdisk as a samba mount, and copied large files to and from that ramdisk from a 4G Ramdisk on the Windows side. ATTO disk benchmark doesn’t test network drives, so I had to make do with copying large files and keeping an eye on the throughput with dstat.

I initially measured the IPoIB throughput using netperf, than this gave me about 6Gbits.

daveh@raid:~$ netperf -H 10.4.12.2
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0
AF_INET to 10.4.12.2 (10.4.12.2) port 0 AF_INET : demo
Recv   Send    Send
Socket Socket  Message  Elapsed
Size   Size    Size     Time     Throughput
bytes  bytes   bytes    secs.    10^6bits/sec

 8192  16384  16384    10.00    6022.50

Based on that number, I was not expecting anywhere close to the 900MB/sec throughput I was getting with SRP. Anyway, just to compare samba throughput with SRP, I pressed on.

While copying a 1 GB video file to
the ramdisk on the linux box (write)

usr sys idl wai hiq siq| read  writ| recv  send
  3  57  31   0   0   9|   0     0 | 191M  565k
  3  61  28   0   0   8|   0     0 | 188M  556k
  4  56  31   0   0   9|4096B   40k| 192M  568k
  4  59  32   0   0   6|   0     0 | 163M  485k
  5  56  29   0   0   9|   0     0 | 186M 6419k
  3  61  29   0   0   8|   0     0 | 187M 6519k
  2  64  25   0   0   9|   0     0 | 183M  543k

 

While copying a 1 GB video file from
the ramdisk on the linux box (read)

usr sys idl wai hiq siq| read  writ| recv  send
  3  40  54   0   0   4|   0  4096B| 863k  251M
  6  42  47   1   0   5|   0    52k| 910k  263M
  5  44  48   0   0   5|   0     0 | 816k  233M
  5  42  46   0   0   6|   0     0 | 812k  231M
  4  45  48   0   0   4|   0     0 | 826k  236M
  3  44  49   0   0   5|   0     0 | 819k  234M
  4  44  48   0   0   5|   0     0 | 832k  236M

I know it’s not very scientific, and I didn’t spend any time tuning samba. But the 900MB/sec throughput is practically out-of-the-box for SRP, so why not test samba at it’s out-of-the-box settings.

So, even with netperf showing a 6Gbit throughput, I was only seeing around 190-235MB/sec throughput over samba (at 40-60% CPU usage).  So I think I’ll stick with my SRP target setup rather than going back to samba. With SRP, I know that whatever disk subsystem I put into the linux box, it probably won’t be able to keep up with the 900MB/sec  that the link is capable of. Unless, that is, I buy a very expensive RAID card and a bunch more disks. And with the way disk prices have gone over the last few months with the floods in Thailand, it’ll be a while until I even think about doing that.

Here’s a few acronyms:

SCSI – we should know what this one is (Small Computer Systems Interface)
iSCSI – SCSI protocol over a network.  (Internet SCSI).
SCST – SCSI Target subsystem for linux with target drivers for iSCSI, Fibre Channel, SRP, SAS, FCoE, etc. We’re most interested in the SRP target.
RDMA – Remote Direct Memory Access – fast way of copying chunks of memory from one machine to another.
SRP – SCSI RDMA Protocol – wraps it all up using RDMA over SCSI protocol.

Stage 1 – Setting up IPoIB

This section shows how to get the basic fabric up and running with TCP/IP over Infiniband (IPoIB). Once you’ve done this, you’ll be able to send pings between the machines, ssh from one to the other across the fabric, etc.

For the windows 7 machine, it’s a simple case of installing the OFED drivers from  openfbrics.org. While installing the windows drivers, make sure you select the SRP options, as later in this guide we’ll be setting up the RAID box as a drive on the Windows box using SRP (iSCSI RDMA Protocol). So,  install the OFED driver package.

To enable improved throughput, enable connected mode on the link by updating the “Connected Mode” and “Connected Mode Payload Mtu size” in the properties of the interface. Open up “Device Manager” and find your Infiniband adapter:

Right clock on the adapter and bring up the properties dialog:

So, on to the linux box. I started with a fresh install of Ubuntu 11.10.

Firstly Install Ubuntu. Update all the packages to the latest using

sudo apt-get update
sudo apt-get upgrade

Everything below is done as root, this avoids having to type ‘sudo’ before every command, so I just call “sudo bash”.

Note: No need to edit the udev rules any more as per the Ubuntu 10.10 HOWTO, as they are correctly set by the kernel 3.0.0 and newer.

Edit /etc/modules and add the following modules:

ib_sa
ib_cm
ib_umad
ib_addr
ib_uverbs
ib_ipoib
ib_ipath
ib_qib

Next,

apt-get install opensm

This will install the subnet manager and all the relevant dependencies, libibverbs, etc.
Then add the relevant entries for the interface into /etc/network/interfaces file:

auto ib0
iface ib0 inet static
address 192.168.1.1
netmask 255.255.255.0

Then reboot. This will create the relevant infiniband entries in /sys, load the ipoib modules, and bring up the infiniband port with an ip address. You should now have a functioning infiniband port on your Ubuntu machine, and you should be able to ping the remote machine.

The next thing is to enable connected mode for the infiniband connection. I found that this increased the tcp/ip netperf benchmarks from 3gbps tp 7gbps.

root@raid:~# echo connected >`find /sys -name mode | grep ib0`
root@raid:~# echo 65520 >`find /sys -name mtu | grep ib0`

To make this happen when the ib0 interface is brought up, modify the /etc/network/interfaces file as follows:

auto ib0
iface ib0 inet static
address 10.4.12.1
netmask 255.255.255.0
up echo connected >`find /sys -name mode | grep ib0`
up echo 65520 >`find /sys -name mtu | grep ib0`

Stage 2 – Setting up SRP Target

The next step, should you wish to take it, is to set up the SRP targets. This is much better option than using samba shares, as it gives you a massive boost in throughput, and uses much less CPU. For example, on my setup using IPoIB and samba shares, I was able to achieve no more than 125MB/sec. Using a ramdisk set up as an SRP target, I was able to achieve 900MB/sec between my two machines.
So, first get a few packages:

apt-get install libmthca1
apt-get install iscsitarget
apt-get install open-iscsi
apt-get install lsscsi
apt-get install scsitools

change /etc/default/iscsitarget to:

ISCSITARGET_ENABLE=true

We now need to get scstadmin. I typically pull the latest version from subversion, so we first need to get svn. apt-get install subversion
There’s a choice to me made now. According to the guide at http://iscsi-scst.sourceforge.net/iscsi-scst-howto.txt , you get a slight performance increase by patching and re-compiling the kernel. That’s a lot of work, so I skipped that step for this guide. I went for the easier option, which is just to run the few commands below. I’m not sure what the difference in performance is, but I can still pull 900MB/sec across the fabric from a ramdisk set up as a an SRP target. And not having to rebuild the kernel makes this a 15 minute procedure rather than a 3 hour one.

cd ~
svn co https://scst.svn.sourceforge.net/svnroot/scst/trunk scst
cd scst
make scst scst_install
make iscsi iscsi_install
make scstadm scstadm_install
make srpt srpt_install

N.B.: You’ll need to rebuild these each time Ubuntu upgrades the kernel, as the modules get out of sync. Just re-run the make’s again.

Lets now use scstadmin to create an SRP target:

First we modprobe a couple of kernel modules. That will make the relevant drivers and targets available to scstadmin. We should then be able to see thse drivers/targets with a few scstadmin ‘list’ commands:

modprobe scst_vdisk
modprobe ib_srpt
root@raid:~# scstadmin -list_handler

Collecting current configuration: done.
 Handler
 -------------
 vdisk_fileio
 vdisk_blockio
 vdisk_nullio
 vcdrom

All done.
root@raid:~# scstadmin -list_driver

Collecting current configuration: done.

 Driver
 -------
 ib_srpt

All done.
root@raid:~# scstadmin -list_device

Collecting current configuration: done.

 Handler           Device
 ------------------------
 vdisk_nullio     -
 vdisk_fileio     -
 vdisk_blockio    DISK01
 vcdrom           -

All done.
root@raid:~#
So, if you can see those handlers, targets, and drivers, your good to go with the next step.

Summary:

The following command list is the quick list to use (and modify) if you’ve done this kind of thing before. The detail section below gives a bit of explanation on each of the lines.

scstadmin -disable_target ib_srpt_target_0 -driver ib_srpt
scstadmin -clear_config -force
scstadmin -open_dev DISK01 -handler vdisk_blockio -attributes filename=/dev/sda
scstadmin -set_dev_attr DISK01 -attributes t10_dev_id=0x2346,threads_num=4
scstadmin -add_group HOST01 -driver ib_srpt -target ib_srpt_target_0
scstadmin -add_lun 0 -driver ib_srpt -target ib_srpt_target_0 -group HOST01 -device DISK01 -attributes read_only=0
scstadmin -add_init 0x0002c9020021f9fc0002c902002200bc -driver ib_srpt -target ib_srpt_target_0 -group HOST01
scstadmin -enable_target ib_srpt_target_0 -driver ib_srpt
scstadmin -write_config /etc/scst.conf

Detail:

Clear the current config (only if that’s ok, and you don’t have any other config that you want to keep. I do this because I’m starting with a clean slate).

scstadmin -clear_config -force

Create DISK01, assigning it to a partition (/dev/sdg1, /dev/md0p1, etc., etc.). I’m using a disk partition in this example.

scstadmin -open_dev DISK01 -handler vdisk_blockio -attributes filename=/dev/sdg1

Now set the drive attributes.

scstadmin -set_dev_attr DISK01 -attributes t10_dev_id=0x2345

Now add a group

scstadmin -add_group HOST01 -driver ib_srpt  -target ib_srpt_target_0

Add a LUN to the group, assigning it to DISK01

scstadmin -add_lun 0 -driver ib_srpt -target ib_srpt_target_0 -group HOST01 -device DISK01 -attributes read_only=0

Add an initiator to the group, allowing the initiator to connect to our new target. I got this from watching /var/log/messages while I was disabling and enabling the Infiniband SRP miniport in device manager on the Win7 box. This caused the SRP miniport to attempt to connect to the Ubuntu target, and that attempt is shown in the /var/log/messages file along with the initiator ID.

scstadmin -add_init 0x0002c9020021f9fc0002c902002200bc -driver ib_srpt -target ib_srpt_target_0 -group HOST01

Finally enable the target

scstadmin -enable_target ib_srpt_target_0 -driver ib_srpt

And write the config.

scstadmin -write_config /etc/scst.conf

At this point you should see a new drive appear on the Win7 box, and asking you to format it. If not, you could try disabling and enabling the SRP miniport driver again, and take a look at /var/log/messages to see what’s happening.

Once the new drive appears on the Windows host, you should be able to format it and start using it. If you don’t see it immediately, have a look in “Disk Management”.

I found that the //etc/init.d/scst script was not getting called at boot, so I added a softlink in /etc/rcS.d

cd /etc/rcS.d
ln -s ../init.d/scst S26scst

This is so it will start after scsitools, opensm and open-iscsi. Using the /etc/init.d/scst script also resolved a problem I’ve had for a long time, in that I used to have some modules in /etc/modules, so the SRP miniport would attempt to connect too early in the linux boot sequence, and fail, so that I had to go into the device manager and disable/enable to get it working. Now it pops up perfectly every boot of the linux box, and I  don’t have to go into device manager on the Win7 box any more.

I’d suggest you run a benchmark on it also just to see what kind of speed you’re getting out if it in real-world usage. There’s a very handy (free) benchmarking took available from attotech.com. Oh, and please do drop a comment below if you do get decent speeds. I’m always interested to hear what people are getting. Oh, and also, comment if you find this guide useful!

References:

SCST – http://scst.sourceforge.net/

Installing iSCSI-SCST – http://iscsi-scst.sourceforge.net/iscsi-scst-howto.txt

Performance Testing.

If you have plenty of memory in your linux machine, you might like to do a test of a ramdisk SRP target over the fabric.

There’s a ramdisk set up by default in Ubuntu at /dev/ram0, but it’s quite small, at 64K. I like to bump it up to 1Gig or 2Gig, as I have 4Gig in my linux box and it does need all that for normal operation. Adding an extra parameter to the kernel line in /boot/grub/grub.cfg does the trick nicely.

ramdisk_size=1024000 for a 1Gig ramdisk at /dev/ram0

ramdisk_size=2048000 for a 2Gig ramdisk at /dev/ram0

so the full line would look like:

linux   /boot/vmlinuz-3.0.0-12-generic root=UUID=39331d95-e9ba-48a5-9dd9-09978e0503c4 ro   quiet splash vt.handoff=7 ramdisk_size=1024000

Then using scstadmin, we add another disk to the script above, but we use vdisk_fileio rather than vdisk_blockio.

scstadmin -open_dev DISK02 -handler vdisk_fileio -attributes filename=/dev/ram0

Then when you open disk manager in windows, you should see a disk requesting a new MBR, which you go ahead and add, then format and run a few benchmarks on it.

In previous posts, I’d identified several bottlenecks, and as I got around each one, I slowly got the speed up. However, everything seemed to max out at 200-220MB/sec. I’d identified PCI-express slots as one potential problem but when I set up a ramdisk on the linux box and exported that as an SRP target, I was still only getting 200MB/sec. Then I thought – could it be my i7 Win7 box? Surely not. That’s sooo much more powerful than the athlon64 x2 3GHz box that I’d linux installed on. But for a laugh I had a look at the PCI express slot that I’d inserted the Mellanox Infiniband card into. Even though it’s a 16x sized slot, my jaw dropped when I saw PCIEx4 written on the motherboard beside it. But the Mellanox card is a PCIEx8 card! So I moved the card into the spare PCIEx16 slot, and re-ran my tests, only to smash all previous records, achieving almost 950MB/sec for some packet sizes.

Just to put this into context:

Machine A:

• 3GHz Athlon 64 dual core, Asus A8N-SLI Premium, 4Gig ram, Mellanox MHEA28-XTC 10Gbps Infiniband card.
• Running Ubuntu 11.10, 2Gig Ramdisk set up as an SRP target

Machine B:

• 3.20GHz Intel i7 quad core, Gigabyte GA-X58-USB3, 12Gig ram, Mellanox MHEA28-XTC 10Gbps Infiniband card
• Running Windows 7 Home Premium, OFED infiniband drivers package, SRP Initiator. ATTO Disk Benchmark utility.

Both machines are connected directly together, no switch was needed.

Consumer grade hardware for the most part, with the magical Mellanox ingredient that pushes this beyond what I previously though possible.

Also, it’s worth noting that this is a stock Ubuntu 11.10 install, a custom kernel was NOT required. I’m working on a HOWTO to get the Infiniband fabric up and SRP targets configured, I’ll be posting that soon.  I reckon it’ll be possible to configure an Ubuntu box up to the 900MB/sec speeds in under 15 minutes.

Here’s the HDTune benchmark of the same setup. Not quite 900MB/sec, but not far off….

“You can learn from my mistakes” (putting the infiniband card in the wrong slot)  :)

More to come soon…..

Additional Notes:

Here’s a shot of the Linux Box. It’s changed a bit since this photo was taken, there’s now a raid card in there where the graphics card is shown, but it give you and idea of the “consumer grade” hardware I’m doing this with….

I ended up getting it for €31 including shipping to Ireland from Finland. Most of these cards go for three times that price. I was just lucky that the auction ended at 3am, and I dont think that  many people bothered to stay up to keep an eye on it.

About a week later, the card arrived, along with the seperate order of a couple of SFF8484 to 4-way SATA cables, I was ready to go.

I’d already freed up one of the PCI-e 16x slots in by Ubuntu box, so the Perc 5/i slotted right in. I hooked up the 5 x 1TB drives that I was configuring in a RAID5 volume, and booted up.

Upon boot, the raid card was detected, and I hit CTRL-R to enter the setup. It was fairly straightforward, I selected the 5 drives that I’d connected up to it, and initialised a new RAID5 volume.

Anyway, the drive then appeared to Linux as a 4TB volume, which I used parted to partition into one big volume.

There were a couple of problems once I’d installed the drive,  read speed and write speed. I was getting 200MB/sec reads and 50MB/sec writes. I did a bit of googling and found an excellent thread on the Dell PERC 5/i controller (ref1), so I flashed the firmware up to the very latest from LSI. I initially did not have a battery, but the latest firmware allowed me to force Write-Back and Adaptive Read-Ahead (not safe settings without battery, but it’d be ok till the battery arrived, another eBay purchase), which should improve the throughput. Also, I found that when you do a quick init of the aray, it says that it completes very quickly, but it’ll go ahead and do a full background init afterwards thereby confusing my throughput results. I found that the best thing to do was to do a full init in the RAID configuration utility and let it sit there for 2.5 hours to complete the full init of 4TB (5x1TB). That way I’d know that the initialisation process would not be skewing my measurements later. Before I did the full init, though, I was getting 170MB read/write (across the infiniband fabric) even when the background init was in progress, so the firmware upgrade was definitely a step in the right direction. I was looking forward to full speed measurements in a couple of hours.

Once the init finished, I was disappointed to find that I was only getting just under 200MB/sec read/write.

So, I tore down the array again and went back to a 5 disk RAID0 array, for max speed. This gave me the following:

This was only marginally better than the RAID5 readings, and worse read speed than using software RAID and motherboard SATA connectors. So to eliminate my Linux box motherboard and CPU, I took the RAID controller out of the Linux box and interted it into my i7  desktop (i7-950@3.07GHz 12G RAM), leaving the drives in the linux box, and running the SATA cables between the two machines. The throughput from the raid card was quite different.

So, now I was getting 300-350MB/sec writes, and up to 600MB/sec reads, with 500MB/sec reads quite common above block size of 64K. That proved that my linux box motherboard is now the bottleneck. Maybe the fact that I’ve two PCIe-8x cards plugged into the two PCIe-16x slots intended for graphics cards is a problem?

At least now I knew what the Perc 5i RAiD controller was capable of, I just had to get the machine that it was hosted in up to spec to be able to drive it at full speed. Maybe then we’ll get more out of the infiniband link between the two machines. I’ll look at re-organising the cards in the machine, maybe look at getting a new motherboard with faster PCI-express lanes.

I’m rapidly coming to the conclusion that trying to re-cycle 5-year-old components to get uber-speed throughput is a lot more difficult than it seems at first. Might be something to do with the hardware not being up to the task when pushed to the limit. The motherboard I’m using in the Linux box with the RAID controller is an Asus A8N-LSI-Premium, which was a top-end consumer-grade motherboard in 2005. With two PCI-e 16x slots, each theoretically capable of 2GB/sec (PCI-e 1.0 8 lanes) in SLI mode, yet I’m only able to get 200-300MB/sec through them? What’s with that? Surely there’s gotta be a helluva lot more data going through those slots when there’s two big fat graphics cards in them.

Stay posted for more updates soon….

Update: Just to help narrow down the bottleneck, I managed to get a 1Gigabyte ramdisk set up as an SRP target. Here’s the throughput across the fabic:

So, The infiniband card in the PCI-express socket with 8 lanes assigned is only getting 200MB/sec through it from RAM. CPU was about 25% busy through the whole test.

References:

http://www.overclock.net/t/359025/perc-5-i-raid-card-tips-and-benchmarks

Anyway, welcome back all….

Regards,

Dave.

While building a linux box as a hard drive farm, I wanted to get the fastest transport possible between the machines using my existing Infiniband fabric (10Gbps), and all research lead to methods using RDMA. I had to rule out NFS/RDMA, as there is no NFS client delivered with Windows 7 Home Premium. I looked at iSCSI, and there is an implementation which uses RDMA called SRP. Having tried and initially failed, I gave up for a few months and was happy enough with a samba link over IPoIB. That was giving me about 130MB/sec reads. I still would have love to get SRP working, as that should theoretically be a lot faster.

Then, someone dropped me a comment on one of my Infiniband blog articles about how they’d got SRP working between a Windows 7 machine and a SLES target. Since this confirmed that Windows 7 was capable of SRP, I had another go. I had thought that it was a limitation of the OFED drivers on Windows 7, but it turns out that I was incorrect,  it was mis-configuration on the Ubuntu side. This article take you through that setup, after which we’ll have an SRP target on the Ubuntu box, accessed from an initiator on a Windows 7 box.

Firstly, here’s some acronyms:

SCSI – we should know what this one is (Small Computer Systems Interface)
iSCSI – SCSI protocol over a network.  (Internet SCSI).
SCST – SCSI Target subsystem for linux with target drivers for iSCSI, Fibre Channel, SRP, SAS, FCoE, etc. We’re most interested in the SRP target.
RDMA – Remote Direct Memory Access – fast way of copying chunks of memory from one machine to another.
SRP – SCSI RDMA Protocol – wraps it all up using RDMA over SCSI protocol.

This article assumes that you’ve already got Infiniband working. See link to “Enabling Infiniband on Ububtu 10.10″ article below which guides you though that.

I found that the easiest way to get SCST (SCSI Target) with SRP (SCSI RDMA Protocol) was to patch the ubuntu kernel source and run a custom kernel. I haven’t tried the latest Ubuntu installs, hopefully it’s included in those and that a kernel rebuild is not necessary. For the moment, I patched my kernel and booted it.

Patch broke kernel. I had to add

#define REQ_WRITE 0×4
to block/blk-map.c

Your kernel might be ok, but that fix allowed my compile to complete.

Once the kernel was up and running, there’s various packages to be installed. I didn’t keep track of these, apologies. However, I do intend to do a complete rebuild soon, so I’ll update the main article with the complete list from start to finish, including the basic infiniband setup all the way to setting up SRP targets so you can have a clean install. Ubuntu 11.10 might even have the scst stuff built into the kernel, eliminating the need to build a custom kernel.

The srp_daemon was complaining on startup about missing /dev/class/infiniband/uverbs* files, so again the udev rules had to be edited to create these. Also, the libmthca1 package needed to be installed to provide user-space drivers for the Mellanox HCAs for use with the ibverbs library.

Now, onto the SCST (SCSI target subsystem) setup. We should have the ib_srp driver installed and available as a target to scstadmin. We can verify this by using “scstadmin -list_target” go show the ib_srpt target is available.

scstadmin -clear_config -force
scstadmin -open_dev DISK01 -handler vdisk_blockio -attributes filename=/dev/sdg1
scstadmin -set_dev_attr DISK01 -attributes t10_dev_id=0x2345
scstadmin -add_group HOST01 -driver ib_srpt  -target ib_srpt_target_0
scstadmin -add_lun 0 -driver ib_srpt -target ib_srpt_target_0 -group HOST01 -device DISK01 -attributes read_only=0
scstadmin -add_init 0x2c902002200bc -driver ib_srpt -target ib_srpt_target_0 -group HOST01
scstadmin -add_init 0x0002c9020021f9fc0002c902002200bc -driver ib_srpt -target ib_srpt_target_0 -group HOST01
scstadmin -enable_target ib_srpt_target_0 -driver ib_srpt
scstadmin -write_config /etc/scst.conf

scstadmin -clear_config -force

scstadmin -open_dev DISK01 -handler vdisk_blockio -attributes filename=/dev/sdg1

scstadmin -set_dev_attr DISK01 -attributes t10_dev_id=0x2345

scstadmin -add_lun 0 -driver ib_srpt -target ib_srpt_target_0 -group HOST01 -device DISK01 -attributes read_only=0

scstadmin -add_init 0x0002c9020021f9fc0002c902002200bc -driver ib_srpt -target ib_srpt_target_0 -group HOST01

scstadmin -enable_target ib_srpt_target_0 -driver ib_srpt

scstadmin -write_config /etc/scst.conf

At this point I saw a new drive appear on the Win7 box, and it was asking me to format it. I formatted it with NTFS, 16K blocks, then did some benchmarks. 250MB/sec reads and 50MB/sec writes. The 50MB/sec writes were down to the speed of the software RAID on linux. Once I split the drives into a couple of RAID0 arrays, the writes went up to 200MB/sec (two drives in RAID0).

So, the SRP issue is resolved, now I need a hardware RAID card with enough horsepower to push the drives at full speed. I’m looking at an 8-port RAID card with PCI-e 4x or 8x. I’ll soon have a spare PCI-e 8x slot in my motherboard once I move the graphics card to a 1x slot. I’ve hacked the 1x slot so I can fit a larger card into it. I don’t need a fast gfx card, and I’ve even disabled the graphical desktop on the machine, all I need is a text-based interface. So a PCI-e 1x graphics card should be fine. It’s also interesting to note that my motherboard has 20 PCI-express lanes in total, 16 for the two PCI-e 16x slots, but the 16 lanes are shared.  It’s either 16x + 1x or 8x + 8x, depending on the SLI settings in the BIOS. That leaves 3 or 4 lanes for the PCI-e 4x slot (2 lanes)  and 1 or two left for the motherboard RAID controller. Not enough really, so I want to get the new RAID controller into one of the PCIe 16x slots using 4-8 PCIe lanes on the mobo. Who’d have known that the slot sizes don’t actually have that amount of lanes assigned. That’s one of the cheats that manufacturers use to bring down the price of motherboards.

More to come soon….

Edit: See what happened when I got the RAID controller…

References:
SCST - http://scst.sourceforge.net/
Rebuild a new linux kernel https://help.ubuntu.com/community/Kernel/Compile
Installing iSCSI-SCST - http://iscsi-scst.sourceforge.net/iscsi-scst-howto.txt

Hi all,

Ive just installed a new theme for the overall domain “www.davidhunt.ie”, and in the process I’ve moved the blog up to the root directory of the server. When exported the content from here and imported into the new blog, the page numbers changes, so I can’t redirect traffic from here to the new site automatically. So I’ll leave this WordPress installation here so anyone who’s put links on their site to here will still work (and I know there’s a lot of you from the stats).

So, the blog is now at www.davidhunt.ie, and the old one will remain dormant at www.davidhunt.ie/wp .

If you were an RSS subscriber, it might be an idea to re-subscribe to the new blog: http://davidhunt.ie/?feed=rss2

Rgds,

Dave.

The nice thing about this design, is that it distributes the strobes quite evenly around the centre point, so that when an umbrella is inserted (that’s what the hole in the middle is for), it illuminates it nicely.

Here’s a closer look:

And a front view:

Also, here’s the original flash/brolly holder before modification. You can see the piece I removed in order to attach the new acrylic pieces.

In order to make the acrylic pieces, I cut a length of acrylic, and made a notch in it. Then I used cyano-acrylate (super glue) to glue the notched piece onto another flat piece of acrylic. The bond of the cyano-acrylate seems to be very strong, very good for acrylic. Maybe the hint is in the “acrylate”

Anyway, I ended up with an 8-inch lenght of hotshoe-holder, which could then be cut it to length to make several hotshoe holders, and I could bolt them onto the brolly holder at various angles, as can be seen in the second image above.

As for the setup time – with the previous design, I had to set up 3 brolly adapters on the “cross”, then attach three strobes. Now it’s pop the modifued brolly adapter onto the stand, and then the three strobes. About half the setup time. And it’s neater, and I think much cooler looking with the transparent acrylic

The three strobes in the top image are triggered by a single trigger using a 4-way cable. You can see the DIY sync ports in the muddle of the red window, as the IR modules have been removed.

The iPad Camera Connection kit gives you two dongles, one with a slot for an SD card, and the other with a USB connector, for direct connection to your camera via USB cable.

The initial thing I discovered (I have a habit of reading the manual after I’ve tried things out), was that this is not really a tethering solution. It’s more of a backup of the images on your camera, which you can then go through, categories, apply ratings, keywords, etc, then later import into lightroom. There is no option to send direct to the iPad as your shooting. I was kinda disappointed at that.
Also, the other similar application I have on the iPad, called Shuttersnitch, only works wirelessly, not over the USB cable, so does not work with the Camera Connection Kit. Now there’s something for either team of developers to look at, proper tethering via USB cable. There’s several reasons why this would be a good idea.

1. A WFT-E4 wireless grip for my Canon EOS5DII is phenomenally expensive.
2. Eye-Fi with any Compact Flash based camera is notoriously difficult to get working, I’m still trying to find a suitable CF-SD adapter so my Eye-Fi X2 Pro will work in my 5DII.
3. Large RAW files transfer MUCH more quickly over USB cable, so if your in the habit of shooting your subjects in burst mode, that kinds rules out Wifi tethering.

Anyway, the general mode of operation is as follows:

1. Take bunch of pictures.
2. Connect your camera to the iPad via the USB cable and the Camera Connection Kit.
3. The iPad photo app will kick off and ask you which images you want to import. Import your selected images.
4. THEN, you start up Photosmith. You can browse through your imported images, tagging, rating, assigning colour codes, etc. just like Lightroom.
5. Once you get back to your PC you can then import the images using the Import wizard in lightroom to get the images from your iPad to your PC/Mac.
6. All the ratings are imported along with the imported images.

One thing I found is that the default connection method is wifi. This takes absolutely AGES when working with RAW files that are up to 30Mb each. So slow in fact, that I almost gave up at that stage. But I persisted, and found an “advanced import” method where you connect your iPad via USB, import the images into lightroom with the normal method, then use the Photosmith sync to get the ratings metadata. 3 seconds per image as opposed to 30. And the metadata was applied to the correct files even though they were imported via an alternative method to the Photosmith sync.

So while it’s not a tethering solution, it’s a great way of having a backup of your images in the field. Even if you were only to sync your camera to your iPad every few dozen images or so, you’ll have a backup on the iPad in case of flash card disaster. And, if you’ve got your images on the iPad, you can categorise them and tag them as you shoot in the field, or on the trip home, which will save you some time when you get back to your main post-processing computer.

Now that I have it, and have sorted out the “advanced import” option, it’s certainly a usable and useful addition to my list of iPad photography apps, and makes the iPad more useful in the field. If only there was an option to display images as you shoot them!

References:

Apple iPad Camera Connection Kit:
http://store.apple.com/us/product/MC531ZM/A
Photosmith Website: http://www.photosmithapp.com/