Home

Search this site:
 

Horshack’s
very
own site

 

This is Bavaria, Germany!
You will find links to Perl,
bootdisks for system administration,
a lot of Linux,
Windows administration
and some personal belongings.
3Ware RAID-controller with Debian Linux

3Ware RAID-controller with Debian Linux, the whole story

Hardware - 3Ware RAID-controller

We wanted a RAID-controller for Debian Linux which I can control automatically via scripts to feed Nagios. Debian is a problem, because most hardware developers only give closed source software for Linux which is not able to run under Debian. Suse and Redhat get supported everywhere, but we don't use these distributions.

We had pretty hard times to figure out that Adaptec 2410SA does not work with Debian-raidutils, so the disks are not controlable. So, the raidcontroller is not usable in this environment!

We decided to use a:

Hardware-Bay for S-ATA-disks

We found a pretty good and pretty cheap bay to connect the hard-drives:

Initialization of the harddisks

This is no trouble. Press Alt-3 to configure when booting the machine. We decided to use this as our configuration:


All harddisks are shown as SCSI-devices (/dev/sda, /dev/sdb, ...)

Commandline utility (CLI) for controlling the RAID

The commandline utility (CLI) is good to automatically control what is going on. But it's not a good idea to deal with it to build up RAIDs which are broken. You should take the tool 3dmd vom 3ware for that, you can deal with that things from a webbrowser which is muuuuuch easier (and safe in this bad situation when something is broken).

Installing the CLI

I fetched the tgz from http://www.3ware.com (no password, e-mail-address or other crap to input there :-)
kuelwalda:/usr/src# ls -l
-rw-r--r--  1 root src   487250 2006-01-16 08:00 cli_linux.tgz

kuelwalda:/usr/src# tar -xzvf cli_linux.tgz
tw_cli
kuelwalda:/usr/src# mv tw_cli /usr/sbin/
kuelwalda:/usr/src# tw_cli
3ware CLI> info
List of controllers
-------------------
Controller 0: 8506-4LP (4)

3ware CLI> info c0 model
Controller Model = 8506-4LP
Attention! We were told to take the 3ware 9500-series CLI which is compatible.

Asking the CLI questions, when everything is well

kuelwalda:~# tw_cli info c0 allunitstatus
Total Units = 2
Bad Units = 0

kuelwalda:~# tw_cli info c0 u1 status
OK (0xÿ)
That means: there are 2 disks, every disk is ok and in good use of the RAID.

Asking the CLI questions, when something is broken

This is how I broke it. There is only disk1 left!

kuelwalda:~# tw_cli info c0 allunitstatus
Total Units = 1
Bad Units = 1
Status of Unit1, my only RAID-1 array:
kuelwalda:~# tw_cli info c0 u1 status
DEGRADED (0xï)
Human output, take care! Port1 and Port2 tell you that everything is ok, but Port2-disk is dead!
kuelwalda:/etc# tw_cli info c0
Controller: c0
-------------
Driver:   1.02.00.037
Model:    8506-4LP
FW:       FE8S 1.05.00.068
BIOS:     BE7X 1.08.00.048
Monitor:  ME7X 1.01.00.040
Serial #: L17901A5110414
PCB:      Rev5
PCHIP:    1.30-66
ACHIP:    3.20


# of units: 1
Unit 1: RAID 1 76.69 GB ( 160834432 blocks): DEGRADED

# of ports: 4
Port 1: HDS728080PLA380 PFDB32S6RW4ZBM 76.69 GB (160836480 blocks): OK(unit 1)
Port 2: HDS728080PLA380 PFDB32S6RW5YAM 76.69 GB (160836480 blocks): OK(unit 1)
The real information is here:
kuelwalda:/etc# tw_cli info c0 u1
Unit /c0/u1
---------------------
Status:        DEGRADED
Unit Type:     RAID 1
Stripe Size:   N/A
Size:          76.69 GB (160834432 blocks)
# of subunits: 2

    Subunit 0:    CBOD: OK
    Physical Port: 1
    Logical  Port: 0

    Subunit 1:    CBOD: DEGRADED
    Physical Port: 2
    Logical  Port: 1
You can also see the real situation in 3dmd-webgui:

Asking the CLI questions, when rebuild is working

Diskfailures, disk2 is broken, disk3 was broken, too! Rebuild is started via 3dmd, the web-browser commandline interface. CLI is a bad choice to do that, you will not be happy (too complicated).

This is the picture with 3dmd in my webbrowser when rebuilding happens:


kuelwalda:~# tw_cli info c0 allunitstatus
Total Units = 1
Bad Units = 1
Status of Unit1, my only RAID-1 array:
kuelwalda:~# tw_cli info c0 u1 status
REBUILDING (0xß)
How far is the rebuilding in percent?
kuelwalda:/usr/src# tw_cli info c0 u1 rebuildstatus
16
Output for humans:
kuelwalda:~# tw_cli info c0
Controller: c0
-------------
Driver:   1.02.00.037
Model:    8506-4LP
FW:       FE8S 1.05.00.068
BIOS:     BE7X 1.08.00.048
Monitor:  ME7X 1.01.00.040
Serial #: L17901A5110414
PCB:      Rev5
PCHIP:    1.30-66
ACHIP:    3.20


# of units: 1
Unit 1: RAID 1 76.69 GB ( 160834432 blocks): REBUILDING (0%)

# of ports: 4
Port 1: HDS728080PLA380 PFDB32S6RW4ZBM 76.69 GB (160836480 blocks): OK(unit 1)
Port 2: HDS728080PLA380 PFDB32S6RW5YAM 76.69 GB (160836480 blocks): OK(NO UNIT)
Port 3: HDS728080PLA380 PFDB32S6RVTE3M 76.69 GB (160836480 blocks): OK(unit 1)

Rebuild for an 80 GBytes RAID-1 disk took about 40 minutes, which is pretty good.

3dmd web-gui-tool to administrate 3ware-controller

This is a programm you can simple start from the commandline. It sends itself into background and opens a webserver on Port 1080.

Change password for 3dmd

You can lock yourself out very easy! Create a copy of /etc/3dmd.conf before doing that. The old, default password is "3ware".
  1. Give admin a password
  2. Give user a password
  3. Activate password for admin
  4. Activate password for user

Starting 3dmd automatically on boot

kuelwalda:~# cd /etc/init.d
kuelwalda:/etc/init.d# ls -l skeleton
-rwxr-xr-x  1 root root 2111 2004-09-10 17:00 skeleton
kuelwalda:/etc/init.d# cp skeleton 3dmd
kuelwalda:/etc/init.d# vim 3dmd
Change these lines:
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DESC="3Ware RAID monitor"
NAME=3dmd
DAEMON=/usr/sbin/$NAME
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME
Take care that 3dmd start automatically. First find out your current runlevel for multiuser mode (which is the usual mode) and start 3dmd at step 99 (one of the last).
kuelwalda:/etc/init.d# runlevel
N 2
kuelwalda:/etc/init.d# update-rc.d 3dmd start 99 2 .
 Adding system startup for /etc/init.d/3dmd ...
   /etc/rc2.d/S993dmd -> ../init.d/3dmd

Resume

3ware 8506-4 for SATA is the best RAID-controller I have found although I could imagine even better:

Additional infos

Support told us:

3Ware 95xx and Debian Sarge

You can get a Debian-package to install from http://www.debian-unofficial.org

Add this to /etc/apt/sources.list:

# For getting 3dm2-package for 3ware
deb http://ftp.debian-unofficial.org/debian sarge main contrib non-free restricted
and then
# apt-get update
# apt-cache search 3ware
3ware-3dm2-new-binary - 3ware 3DM 2 Disk Management Utility (new)
3ware-3dm2-old-binary - 3ware 3DM 2 Disk Management Utility (old)
# apt-get install 3ware-3dm2-new-binary
Start 3dm2 manually first and a certificate gets created. You will find a process listening on port 888. Navigate to https://your.machine:888,  the password for User and Administrator is "3ware"

Change the passowords for both users immediately with Menu "3dm2-Settings"



Last modified:
04.02.2006 08:14
Impressum
Wozu Witze ausdenken? Das Leben schreibt sie selbst:
Das haben wir jetzt 40 Jahre so gemacht, warum sollten wir das ändern?
Why inventing jokes? Real life does a better job:
We are doing it this way for 40 years, why should we change it?