Build a cheap and fast RAID 5 NAS (1)

URL: http://www.tomsnetworking.com/2006/08/01/cheap_fast_diy_raid_5_nas/

Introduction

RAID ("Redundant Arrays of Independent Disks"), especially raid level 5, is participant ly compelling to geeks, even wannabe geeks like me. What's not to love? You lose one disk, throw in another and your data rocks on! This is the RAID 5 dream. RAID 5 reality, however, is not so compelling. In fact, please people who know they shocould be using RAID 5 systems do not or cannot use the technology.

The purpose of this article is to build an inexpensive RAID 5 network attached storage (NAS) system. i'll then compare the home-built system to off-the-shelf products to see how they compare in terms of performance. if we cut enough cost and produce enough value, we might convert some of the RAID 5 procrastinating masses into RAID 5 users.

For this project, I chose a RAID card fromLSI logic's megaraidFamily of RAID 5 adapters. I found the megaraid family based on a Tom's hardware review (cheap and reliable RAID 5 Storage compared) of the SATA 4 port adapter; I have been using these adapters since late 2004. after building my first RAID 5 system on the megaraid SATA 4, I noticed on the 'Net that the older IDE/ATA generation of megaraid cards was available on eBay at a fraction of the price of the SATA card. (At the time of this writing, I can buy an ATA card for $80 on eBay, while the LSI megaraid SATA card on froogle.google.com is $260 .)

Those of you who are very frugal may wonder why I'm not using a software RAID 5 solution. Well, I tried to use software RAID 5 and even bought the O 'Reilly bookManaging raid on LinuxIn order to learn how. recovery is the single most important aspect of RAID 5 to be sure you understand before implementing RAID 5. but even with the O 'Reilly book, I cocould not figure out how I wocould recover from a drive failure, step by step. so instead, it's a hardware RAID 5 solution for me.

After using hardware-based raid for two years, I 've successfully recovered from drive failures, but just as importantly, I 've learned That hardware raid gives you a degree of freedom in designing your NAs. A hardware RAID card insulates you from operating systems and their issues. since I use a well-supported hardware RAID card, I have installed Windows 2003, freenas, Ubuntu, clarkconnect, and Suse, on the same hardware, without having to learn each operating system's setup for raid and how to perform raid recovery.

 

The LSI logic megaraid card

The LSI logic megaraid adapters are ROM-based. so the usage model is to boot the computer and then start the Controller's software with a control-Key sequence. the Controller's software is a BIOS-style menu-driven interface that allows you to set up, upgrade, or repair your RAID 5 array. when your computer boots up, you will see the LSI logic adapter's control instructions flash on your monitor, as shown in figure 1.

Figure 1: Splash Screen

When you press <Ctrl> + m in response, the LSI logic megaraid will start its software from Rom to allow you to manage your RAID 5 array. figure 2 shows the main menu of the megaraid software.

Figure 2: megaraid Main Menu

Figure 3 shows how menus pop up in the megaraid software.

Figure 3: megaraid menu pop-ups

Note that in Figure 3 I have moved the cursor to "Configure" and then pressed the <enter> key, to pop up the "Configure" menu. I then moved the cursor to "new configuration" to initialize the RAID 5 array of drives. the megaraid software has then popped open a third menu that says "proceed? ". To finish refreshing a new array I wocould then move the cursor to "yes" and hit <enter>. while the interface is a flashback to the days of MS-DOS, It is functional and reliable.

The megaraid cards allow what I call the "sane" RAID 5 repair procedure:

Step 0: Hear the shrill wailing of the megaraid card
Step 1: identify which hard drive has failed
Step 2: Shut down the raid NAS
Step 3: remove the failed hard drive
Step 4: Get in your car
Step 5: drive to your local Computer Retailer and buy a replacement hard drive
Step 6: return home
Step 7: install the new hard drive
Step 8: kick off the RAID Controller's recovery procedure
Step 9: when the recovery is complete, reboot the server and carry on

I named this the "sane" procedure after looking at software raid and concluding that recovering a drive under software raid was (for me) insane.

Parts list

The hardware for this do-it-yourself project is shown in table 1. to keep this project simple, there are only two hardware options: the default configuration with the motherboard's built-in 10/100 Ethernet, and a secondary configuration with Gigabit Ethernet. if you're a good shopper, you may be able to beat my cost, especially if you use a retired computer that's just gathering dust in your closet.

By way of comparison, Buffalo technology's 1 TB terastation is selling on the web for und $650 and the "pro" version is about $230 more.

	Component	Cost
RAID Controller	LSI logic megaraid I4 (ATA)	$80 to $160 on www.ebay.com
CPU	Sempron 2200 +	$59 (for 2600 +; 2200 + is no longer available)
Motherboard	ECS 741gx-m	$110
Ram	512 mb ddr 400 SDRAM Running at 266 MHz	$40
Power Supply	Coolermaster RS-450-ACLX 450 W	$80
Ethernet 1	100 Mb on ECS 741gx-m	-(Imported ded in motherboard)
Ethernet 2	32-bit 10/100/1000 Trendnet TEG-PCITXR	$19
Hard drives	4 300 gigabyte 7200 ata ide	4x$105 = $420
Case	Cheapest @ compUSA	$30
Cd rom to boot	Asus 52x	$10
OS	Ubuntu 6.06 Desktop	$0
Total system cost		$829-928

Table 1

For this project, the most important thing to be sure of is that the motherboardDoes not mask interrupt 19. Masking interrupt 19 prevents the <Ctrl> + M keystroke from starting the megaraid firmware. if you have a motherboard that blocks interrupt 19, you have to boot the computer to an OS close enough to MS-DOS to allow LSI logic's megaraid.exe program to run. (The megaraid.exe program starts the firmware in the megaraid card, which then allows you to manage your disk array .) this is annoying to do when you are working on a machine for several hours, so a motherboard that does not block interrupt 19 is highly recommended.

The ram for this system is 512 MB of ddr400 SDRAM running at 266 MHz. the ECS motherboard theoretically supports 333 MHz, But I 've never been able to get memory to run that fast on this board. the power supply is a 450 watt coolermaster model. A power level of 450 W is more than is needed-each of my drives draws about 9 watts of power-but because you are building a RAID 5 array, you will need a lot of power connectors. A 450 W supply usually has the two extra power connectors cables needed for a 4 drive array.

I used Ubuntu 6.06 because it instils beautifully. my favorite build of Linux is clarkconnect, But it wocould not install on the LSI logic megaraid I4 controller. this is because clarkconnect is based on Fedora 4, which pulled support for the megaraid I4 cards last year.

Assembly

You need a case that will allow four hard drives with a full hard drive's worth of space between them. figure 4 shows the #1 feature I look for in a case: a drive tower that runs from the floor of the case all the way up to the underside of the externally mountable devices (I. E ., the floppy and optical drives ). I have found that trying to squeeze hard drives into a case that does not have extra space is a sure way to cook them.

Figure 4: Case detail

Next, lay out the RAID 5 components. to make the NAS you need to basically start with a standard desktop computer without a hard drive. in Figure 5 I 've already installed the motherboard, Ram, CPU, optical drive and power supply, to make a standard desktop computer that is ready to go into t for the hard drives.

Figure 5: Ready for assembly

Then, install the hard drives one at a time and hold them in place with one screw. I leave a full drive's worth of space vertically between drives in the case, for better cooling.

Install the power cables to each drive, then install the ribbon cables on the megaraid I4 card and insert the card into the PCI slot. figure 6 shows the four drives installed and powered, And the megaraid card with 4 IDE/ATA ribbon cables hanging off, ready to be attached to the 4 drives.

Figure 6: drives installed waiting for raid cable connection

Tip:I advise installing the cables before you put the megaraid card into the computer. the connectors on the megaraid card can crack if you affect the ribbon cable towards and away from the face of the card. installing the cables while the card is outside of the Case greatly lowers this risk.

After the megaraid I4 card and cables are installed, connect the ribbon cables from the megaraid I4 to the hard drives (see figure 7 ).

Figure 7: drives cabled up

At this point you can add the second screw to each hard drive and button up the case. I 've never set a jumper on a hard drive when using a megaraid card. I simply take the hard drives out of the box, put them into the computer case and don't even look at how the Jumpers are set.

Raid array Configuration

With hardware Assembly out of the way, we now can create the drive array. to do this you need to connect a keyboard, monitor, and power to the computer. boot up the computer, and when you see the message shown in Figure 8 appear on the screen, hold down <Ctrl> + m to kick off the megaraid's Rom software.

Figure 8: megaraid splash screen

I'm going to go through the Raid set up step by step because I found that the biggest stressor in building your own NAS is figuring out the megaraid user interface. I stumbled around for several days before getting my first disk array up and running because I cocould not find any step-by-step instructions. so now there are!

Step 1:Figure 9 shows the megaraid management menu that will pop up on your screen after you hold down <Ctrl> + m while your computer boots. move the cursor to "Configure" on this menu, and press <enter>.

Figure 9: megaraid management menu

Step 2:Your screen will now appear as in Figure 10. move the cursor to "new configuration" and press <enter>.

Figure 10: select new configuration

Step 3:When the "proceed? "Menu pops up (see figure 11) move the cursor to" yes "and then hit <enter> again. the megaraid card will then begin to scan all the hard drives attached to it. after a moment, the screen will go from looking like Figure 11, back to looking like figure 10. if you try to move the cursor, it will seem like your system is locked up. don't panic.

Figure 11: Select yes or no to proceed

The trick is to watch the bottom row of characters on your computer screen. figure 12 shows the messages that flash by on the bottom row. what is happening here is that your megaraid card is scanning the hard drives attached to it.

Figure 12: Status messagesraid configuration, continued

Step 4:After the megaraid software is finished scanning, your screen will look like figure 13.

Figure 13: array selection menu screen

You'll now select the drives for your array. in Figure 13, you can see three of the four hard drives attached to this computer. the first row, with "M" underneath each channel column, shows "master" hard drives. the second row is "S" for "slave" hard drives. because the megaraid I4 is an IDE/ATA device, you can have up to 8 drives per card in a 4 Master, 4 slave configuration. the missing hard drive in Figure 13 is channel-3 (I. E ., the fourth hard drive) which is not shown because an 80-column character-based screen is not wide enough to show four drives.

Now that we see all the drives that are going into our disk array, we need to tell the megaraid software which of these hard drives to use. not all hard drives have to be used in the RAID 5 array. if you have four drives, you can put three drives into a RAID 5 array, and use the fourth drive as a hot-spinning spare for recovery.

To select all the drives for our array, we use the arrow keys in Figure 13 to move the cursor to a hard drive that says ready, and then we hit the "space" bar to toggle the drive's status from "ready" to "onlin" (yes, the "e" is missing !).

Figure 14 shows what the screen looks like after I 've scrolled through each of the hard drives and hit "space" to select each hard drive for the array. it actually is the same as Figure 13, doesn t that the drives have changed from "ready" to "onlin ".

Figure 14: drives selected for the raid Array

To select the hard drive on channel-3 (not shown off to the right of the screen) Just use the right arrow to scroll to the right to find it, and then hit space bar.

As you can see, the megaraid user interface is clunky, but it's not too bad once you get the hang of the implicit rule of this interface:"When in doubt, look at the bottom row of the screen ."The black magic around RAID 5 then disappears, and you can become comfortable with the megaraid.

Once you have all the disks showing a status of "onlin" (as in Figure 14) Look at the bottom row of the screen (see figure 15 ).

Figure 15: Status line after drives are selected

Step 5:Then convert the selected drives into an array. hit "enter" to tell the megaraid software you are done selecting drives. after this, the screen will remain unchanged. next, hit <F10> to configure the RAID 5 array.