Bringing up dart
Hardware
- SuperMicro SC742 4U server case
- SuperMicro X9SRA motherboard
- Intel Xeon E5-2650 Sandy Bridge-EP 2.0GHz (2.8GHz Turbo Boost) LGA 2011 95W 8-Core Server Processor
- SUPERMICRO SNK-P0050AP4 Heatsink for Supermicro X9DR3-F Motherboard
- 32GB ECC registered RAM (4 x 8GB)
- Four 15K 150GB drives HITACHI HUS153014VL
- Corsair Neutron Series GTX CSSD-N240GBGTX-BK 2.5" 240GB SATA III Internal Solid State Drive (SSD)
- 3Ware 9690SA-4I SATA/SAS RAID controller
Drive Config
I ended up doing the entire config using the CentOS installer, but left this information here because going through the steps myself helped me understand how LVM works. I created some virtual drives and played around with them in a VirtualBox installation of CentOS to work in a safe environment where I would not be able to break anything.
Hardware
sda 3Ware 435 (4 x Hitachi 150GB in RAID 5) sdb Neutron SSD 240 sdc Neutron SSD 240 sdd WDC Black 1000 sde Revo 60 sdf Revo 60
2 x Neutrons => partitioned for boot and md0 RAID 1 = operating system 2 x Revos => md1 RAID 0 = fast workspace 3ware RAID => /var and other fast data storage Black = home, bulky data, swap
Volume groups
vg_mirror vg_workspace vg_raid vg_bulk
Logical volumes
lv_root / lv_workspace /workspace lv_var /var lv_home /home lv_swap SWAP
Neutron drive: Build a broken mirror, add second drive later
[See http://www.gagme.com/greg/linux/raid-lvm.php Managing RAID and LVM with Linux] For install onto RAID see also some of the steps here Install on partitionable RAID1
Here are the commands that I used.
# Use the entire drive and put the partitions inside /dev/md_d0 mdadm --create --metadata=0.90 --level=mirror --raid-devices=2 /dev/md_d0 /dev/sda missing # put RAID1 into LVM2 pvcreate /dev/md0 vgcreate vg_mirror /dev/md0
# later, set up second drive as the mirror mdadm --add --verbose /dev/md0 /dev/sdb2 # the mirror should automatically be added to LVM since it's part of /dev/md0 cat /proc/mdstat
Revo drive: completely managed under LVM2
The Revo drive has its own RAID onboard. I set up the two halves as RAID 0 for performance at the cost of potential dataloss. The drive is visible to Linux as /dev/sde and /dev/sdf and also as a RAID at /dev/mapper/sil_bibibgajfbdj (a name only its mother could love.) I set it up under LVM2 thusly
[root@dart ~]# pvcreate -v /dev/mapper/sil_bibibgajfbdj
Set up physical volume for "/dev/mapper/sil_bibibgajfbdj" with 234458624 available sectors Zeroing start of device /dev/mapper/sil_bibibgajfbdj Writing physical volume data to disk "/dev/mapper/sil_bibibgajfbdj" Physical volume "/dev/mapper/sil_bibibgajfbdj" successfully created
[root@dart ~]# vgcreate --verbose vg_workspace /dev/mapper/sil_bibibgajfbdj
Wiping cache of LVM-capable devices Adding physical volume '/dev/mapper/sil_bibibgajfbdj' to volume group 'vg_workspace' Archiving volume group "vg_workspace" metadata (seqno 0). Creating volume group backup "/etc/lvm/backup/vg_workspace" (seqno 1). Volume group "vg_workspace" successfully created
[root@dart ~]# lvcreate --verbose --extents 100%FREE -n lv_workspace vg_workspace
Setting logging type to disk Finding volume group "vg_workspace" Archiving volume group "vg_workspace" metadata (seqno 1). Creating logical volume lv_workspace Creating volume group backup "/etc/lvm/backup/vg_workspace" (seqno 2). Found volume group "vg_workspace" activation/volume_list configuration setting not defined: Checking only host tags for vg_workspace/lv_workspace Creating vg_workspace-lv_workspace Loading vg_workspace-lv_workspace table (253:5) Resuming vg_workspace-lv_workspace (253:5) Clearing start of logical volume "lv_workspace" Creating volume group backup "/etc/lvm/backup/vg_workspace" (seqno 2). Logical volume "lv_workspace" created
[root@dart ~]# mkfs.ext4 /dev/vg_workspace/lv_workspace
/dev/vg_workspace/lv_workspace will be mounted at /workspace
3Ware RAID5: completely managed under LVM2
pvcreate /dev/sd? vgcreate vg_raid /dev/sd? lvcreate -l ????? vg_raid -n lv_var mkfs.ext4 /dev/vg_raid/lv_var
After installing OS, /dev/vg_raid/lvm0 will be mounted at /var
Black drive: completely managed under LVM2
Might also want some SWAP on here?
parted set up one LVM partitions mark partition type as 8e = LVM pvcreate /dev/sdd vgcreate vg_bulk /dev/sdd lvcreate -l ????? vg_bulk -n lv_home lvcreate -l ????? vg_bulk -n lv_swap mkfs.ext4 /dev/vg_bulk/lv_home mkswap /dev/vg_bulk/lv_swap
Will be mounted as /home and swap
Commands to view what we've done
This is how things actually look right now..
[root@dart bwilson]# pvdisplay
--- Physical volume --- PV Name /dev/sdd1 VG Name vg_black PV Size 931.51 GiB / not usable 4.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 238466 Free PE 0 Allocated PE 238466 PV UUID 2ZWwrU-Eyp9-erVR-DaSZ-ld3T-g6LK-8SBJcn --- Physical volume --- PV Name /dev/disk/by-id/dm-name-sil_bibibgajfbdj VG Name vg_workspace PV Size 111.80 GiB / not usable 1.75 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 28620 Free PE 0 Allocated PE 28620 PV UUID F86o1d-tmhZ-4KVW-Ke06-b9iE-7HGW-EjhLrF --- Physical volume --- PV Name /dev/md1 VG Name vg_root PV Size 223.37 GiB / not usable 1.87 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 57183 Free PE 0 Allocated PE 57183 PV UUID drVc1G-694L-6GSl-cTNl-uINo-zbEc-ZBpILd --- Physical volume --- PV Name /dev/sda1 VG Name vg_var PV Size 405.09 GiB / not usable 3.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 103703 Free PE 0 Allocated PE 103703 PV UUID nMZCtR-oZkS-2Ctu-9HwB-Vt5r-7iPD-vf5v7H
[root@dart bwilson]# vgdisplay
--- Volume group --- VG Name vg_black System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 2 Open LV 2 Max PV 0 Cur PV 1 Act PV 1 VG Size 931.51 GiB PE Size 4.00 MiB Total PE 238466 Alloc PE / Size 238466 / 931.51 GiB Free PE / Size 0 / 0 VG UUID fYgPgc-cXaW-6ryU-nU1v-5m3F-Rj0r-Afe9p6 --- Volume group --- VG Name vg_workspace System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 111.80 GiB PE Size 4.00 MiB Total PE 28620 Alloc PE / Size 28620 / 111.80 GiB Free PE / Size 0 / 0 VG UUID 9OkF0O-E5f7-bfvH-qAfb-E7Wh-cOGS-EhVydH --- Volume group --- VG Name vg_root System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 223.37 GiB PE Size 4.00 MiB Total PE 57183 Alloc PE / Size 57183 / 223.37 GiB Free PE / Size 0 / 0 VG UUID 2Q5ooC-B0ZW-BcxA-jufX-80zh-hkQh-qYNm2G --- Volume group --- VG Name vg_var System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 405.09 GiB PE Size 4.00 MiB Total PE 103703 Alloc PE / Size 103703 / 405.09 GiB Free PE / Size 0 / 0 VG UUID mIgZdQ-4tzM-T2KP-S2f7-QA14-iwp9-cTwyly
[root@dart bwilson]# lvdisplay
--- Logical volume --- LV Path /dev/vg_black/lv_swap LV Name lv_swap VG Name vg_black LV UUID 3ZtM9o-a5Z8-iAs9-Sm1K-O1zU-jfLM-iKX7pj LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:26:11 -0800 LV Status available # open 1 LV Size 15.62 GiB Current LE 4000 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1 --- Logical volume --- LV Path /dev/vg_black/lv_home LV Name lv_home VG Name vg_black LV UUID rwvORk-YHTr-jmMR-lxey-Q0Ue-NwTO-mOcrUO LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:26:12 -0800 LV Status available # open 1 LV Size 915.88 GiB Current LE 234466 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2 --- Logical volume --- LV Path /dev/vg_workspace/lv_workspace LV Name lv_workspace VG Name vg_workspace LV UUID wUhofL-LtjP-p6Rr-wMj0-jKRF-X2fB-Bjyy9H LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-09 18:29:17 -0800 LV Status available # open 0 LV Size 111.80 GiB Current LE 28620 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:5 --- Logical volume --- LV Path /dev/vg_root/lv_root LV Name lv_root VG Name vg_root LV UUID 5cCyeM-yuTM-GsfE-M4Xu-em5h-6olD-sMucxV LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:26:00 -0800 LV Status available # open 1 LV Size 223.37 GiB Current LE 57183 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Path /dev/vg_var/lv_var LV Name lv_var VG Name vg_var LV UUID GZFOdu-D6uN-UCAR-Gt3R-Ddni-2dGd-ZNmhNh LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:25:17 -0800 LV Status available # open 1 LV Size 405.09 GiB Current LE 103703 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:3
/etc/fstab
/dev/mapper/vg_root-lv_root / ext4 defaults,noatime,discard 1 1 UUID=488a0c3b-689e-4f2b-8688-d8eefdf0a7f8 /boot ext4 defaults,noatime,discard 1 2 /dev/mapper/vg_black-lv_home /home ext4 defaults,noatime 1 2 /dev/mapper/vg_var-lv_var /var ext4 defaults,noatime 1 2 /dev/mapper/vg_black-lv_swap swap swap defaults 0 0 /dev/mapper/vg_workspace-lv_workspace /workspace ext4 defaults,noatime,discard 1 2 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 # SSD optimizations none /tmp tmpfs defaults,noatime,mode=1777 0 0
CentOS install
Install server system from DVD
vi /etc/sysconfig/network-scripts/ifcfg-eth0 vi /etc/resolv.conf service network restart yum -y install emacs yum -y install openssh-clients yum -y install gcc unzip yum -y install hdparm yum -y install httpd yum update reboot
Basic software install
Install PostGIS 2.0