Tags:
view all tags
Server Documentation:<br /><br />Switches have been stacked, but management interface still considers them as two separate switches with the same IP<br /> *NOTE*: 9/5/2010 Turning on the 2nd Switch causes the blades not to be able to communicate any longer, and the NICs on the machines flash green very rapidly<br /><br />Labelled the cables in the following way:<br /><br />ControllerNode: Main server with fiber connection is node "00"<br /><br />Remaining servers each have 2 ethernet data ports denoted node / 1 and node / 2, where n is the servers 01 to 10<br /> Top Port (eth1) is hotpluggable to an internet connection (X.X.X.142 on our local lab network)<br /> Bottom Port (eth0) is set up as IP 10.10.10.1NODE, so IPs 100-110 are reserved for the Nodes<br /><br />Both disks appear as one disk called: /dev/cciss/c0d0 to the operating system<br /><br />SlaveNode Configuration: <br />Drives are configured as a logical volume via the HP Smart Array P400 RAID controller (<a target="_blank" href="http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=en&cc=us&docIndexId=64179&taskId=101&prodTypeId=329290&prodSeriesId=1157687" title="Manual">Manual</a>), RAID 0 ("fault tolerance")<br />Not sure what the other configuration settings are--ask Sirhan<br /><br />Partition table of logical drive---must be connected using the RAID Controller (press F8 to get into it and set up the logical drives)<br /><br />settings of logical drivess: Bays 1 and 2 or 3 and 4 (paired)<br />Clonezilla will refer to the drives<br />p1 1 2432 primary LINUX<br />p2 2433 7295 primary LINUX<br />p3 7296 10942 primary LINUX<br />p4 10943 end EXTENDED<br />p5 10943 17021 EXTENDED<br /><br />63 sectors<br />121 <br />594 cylinders<br />16065 * 512 = 8225280 bytes<br />-----<br />[original defaults read:<br />32 sectors<br />239389 cylinders 8160 * 612 = 4177920 bytes] ISSUES:<br />RAID controller doesn't like swapping of drives for Logical Volumes<br /><br />NOTE ABOUT LOGICAL DRIVE MANAGEMENT:<br />If there are 2 logical volumes (4 drives) and bay 3 and 4 are removed, and replaced, the HP Smart Array controller will not detect ANY logical volumes, unless the old one is properly deleted before inserting new disks FAILED ATTEMPT TO CLONE:<br />/opt/drbl/sbin/ocs-onthefly -g auto -e1 auto -e2 -j2 -v -f sda -t sdb<br /><br />dd if=/dev/sda of=/tmp/ocx<br /><br />FRESH INSTALLATION ON A NODE:<br /><br />1. SET UP THE RAID DISK ARRAY<br /> - At the HP Smart Array P400 Controller Initialization screen, press F8 (it passes quickly so be ready)<br />2. I Installed the TESTING snapshot of Debian. I use the business card CD, install the base system, and only have the System Utilities and SSH server installed<br />3. configure /etc/hosts /etc/hostname /etc/network/interfaces appropriately (fill this in)<br />4. install the following package: openmpi-common openmpi-bin libopenmpi1.3 libopenmpi-dev emacs23-nox and openssh-server<br />5. I think something needs to be done with SWAP files<br /><br /><br />Currently Nodes 01-03 and 05-10 are set up with EXT4 + LVM + NFS, but this seems to fail with "scattertime" test (NFS hangs)<br /><br />Node 04 is configured with ext3 + nfs and no LVM We noticed that ext4 + nfs + LVM was causing problems (the system was hanging), so we have gone back to the original partition table (above) with ext3 partitions. 9/7/2010 We also had an issue with the speed of the ethernet cards. Rather than running at gigabit speed (1000BaseT), it was running at 10BaseT half duplex. This was determined by examining /var/log/messages file or typing "dmesg" immediately after unplugging and replugging the cable back in. A simple reboot solved the problem, but we are unsure what set it to that mode in the first place. -- Main.StephenFox - 2010-08-23
Edit
|
Attach
|
Watch
|
P
rint version
|
H
istory
:
r4
<
r3
<
r2
<
r1
|
B
acklinks
|
V
iew topic
|
Raw edit
|
More topic actions...
Topic revision: r1 - 2015-11-10
-
TWikiGuest
Home
Site map
Main web
Sandbox web
TWiki web
Main Web
Users
Groups
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
Edit
Attach
Copyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback