Bringing up dart
Hardware
- SuperMicro SC742 4U server case
- SuperMicro X9SRA motherboard
- Intel Xeon E5-2650 Sandy Bridge-EP 2.0GHz (2.8GHz Turbo Boost) LGA 2011 95W 8-Core Server Processor
- SUPERMICRO SNK-P0050AP4 Heatsink for Supermicro X9DR3-F Motherboard
- 32GB ECC registered RAM (4 x 8GB)
- Four 15K 150GB drives HITACHI HUS153014VL
- Corsair Neutron Series GTX CSSD-N240GBGTX-BK 2.5" 240GB SATA III Internal Solid State Drive (SSD)
- 3Ware 9690SA-4I SATA/SAS RAID controller
Drive Config
I ended up doing the entire config using the CentOS installer, but left this information here because going through the steps myself helped me understand how LVM works. I created some virtual drives and played around with them in a VirtualBox installation of CentOS to work in a safe environment where I would not be able to break anything.
Hardware
sda 3Ware 435 (4 x Hitachi 150GB in RAID 5) sdb Neutron SSD 240 sdc Neutron SSD 240 sdd WDC Black 1000 sde Revo 60 sdf Revo 60
2 x Neutrons => partitioned for boot and md0 RAID 1 = operating system 2 x Revos => md1 RAID 0 = fast workspace 3ware RAID => /var and other fast data storage Black = home, bulky data, swap
Volume groups
vg_mirror vg_workspace vg_raid vg_bulk
Logical volumes
lv_root / lv_workspace /workspace lv_var /var lv_home /home lv_swap SWAP
Neutron drive: Build a broken mirror, add second drive later
[See http://www.gagme.com/greg/linux/raid-lvm.php Managing RAID and LVM with Linux] For install onto RAID see also some of the steps here Install on partitionable RAID1
Here are the commands that I used.
# Use the entire drive and put the partitions inside /dev/md_d0 mdadm --create --metadata=0.90 --level=mirror --raid-devices=2 /dev/md_d0 /dev/sda missing # put RAID1 into LVM2 pvcreate /dev/md0 vgcreate vg_mirror /dev/md0
# later, set up second drive as the mirror mdadm --add --verbose /dev/md0 /dev/sdb2 # the mirror should automatically be added to LVM since it's part of /dev/md0 cat /proc/mdstat
Revo drive: completely managed under LVM2
The Revo drive has its own RAID onboard. I set up the two halves as RAID 0 for performance at the cost of potential dataloss. The drive is visible to Linux as /dev/sde and /dev/sdf and also as a RAID at /dev/mapper/sil_bibibgajfbdj (a name only its mother could love.) I set it up under LVM2 thusly
[root@dart ~]# pvcreate -v /dev/mapper/sil_bibibgajfbdj
Set up physical volume for "/dev/mapper/sil_bibibgajfbdj" with 234458624 available sectors Zeroing start of device /dev/mapper/sil_bibibgajfbdj Writing physical volume data to disk "/dev/mapper/sil_bibibgajfbdj" Physical volume "/dev/mapper/sil_bibibgajfbdj" successfully created
[root@dart ~]# vgcreate --verbose vg_workspace /dev/mapper/sil_bibibgajfbdj
Wiping cache of LVM-capable devices Adding physical volume '/dev/mapper/sil_bibibgajfbdj' to volume group 'vg_workspace' Archiving volume group "vg_workspace" metadata (seqno 0). Creating volume group backup "/etc/lvm/backup/vg_workspace" (seqno 1). Volume group "vg_workspace" successfully created
[root@dart ~]# lvcreate --verbose --extents 100%FREE -n lv_workspace vg_workspace
Setting logging type to disk Finding volume group "vg_workspace" Archiving volume group "vg_workspace" metadata (seqno 1). Creating logical volume lv_workspace Creating volume group backup "/etc/lvm/backup/vg_workspace" (seqno 2). Found volume group "vg_workspace" activation/volume_list configuration setting not defined: Checking only host tags for vg_workspace/lv_workspace Creating vg_workspace-lv_workspace Loading vg_workspace-lv_workspace table (253:5) Resuming vg_workspace-lv_workspace (253:5) Clearing start of logical volume "lv_workspace" Creating volume group backup "/etc/lvm/backup/vg_workspace" (seqno 2). Logical volume "lv_workspace" created
[root@dart ~]# mkfs.ext4 /dev/vg_workspace/lv_workspace
/dev/vg_workspace/lv_workspace will be mounted at /workspace
3Ware RAID5: completely managed under LVM2
pvcreate /dev/sd? vgcreate vg_raid /dev/sd? lvcreate -l ????? vg_raid -n lv_var mkfs.ext4 /dev/vg_raid/lv_var
After installing OS, /dev/vg_raid/lvm0 will be mounted at /var
Black drive: completely managed under LVM2
Might also want some SWAP on here?
parted set up one LVM partitions mark partition type as 8e = LVM pvcreate /dev/sdd vgcreate vg_bulk /dev/sdd lvcreate -l ????? vg_bulk -n lv_home lvcreate -l ????? vg_bulk -n lv_swap mkfs.ext4 /dev/vg_bulk/lv_home mkswap /dev/vg_bulk/lv_swap
Will be mounted as /home and swap
Commands to view what we've done
This is how things actually look right now..
[root@dart bwilson]# pvdisplay
--- Physical volume --- PV Name /dev/sdd1 VG Name vg_black PV Size 931.51 GiB / not usable 4.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 238466 Free PE 0 Allocated PE 238466 PV UUID 2ZWwrU-Eyp9-erVR-DaSZ-ld3T-g6LK-8SBJcn --- Physical volume --- PV Name /dev/disk/by-id/dm-name-sil_bibibgajfbdj VG Name vg_workspace PV Size 111.80 GiB / not usable 1.75 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 28620 Free PE 0 Allocated PE 28620 PV UUID F86o1d-tmhZ-4KVW-Ke06-b9iE-7HGW-EjhLrF --- Physical volume --- PV Name /dev/md1 VG Name vg_root PV Size 223.37 GiB / not usable 1.87 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 57183 Free PE 0 Allocated PE 57183 PV UUID drVc1G-694L-6GSl-cTNl-uINo-zbEc-ZBpILd --- Physical volume --- PV Name /dev/sda1 VG Name vg_var PV Size 405.09 GiB / not usable 3.00 MiB Allocatable yes (but full) PE Size 4.00 MiB Total PE 103703 Free PE 0 Allocated PE 103703 PV UUID nMZCtR-oZkS-2Ctu-9HwB-Vt5r-7iPD-vf5v7H
[root@dart bwilson]# vgdisplay
--- Volume group --- VG Name vg_black System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 3 VG Access read/write VG Status resizable MAX LV 0 Cur LV 2 Open LV 2 Max PV 0 Cur PV 1 Act PV 1 VG Size 931.51 GiB PE Size 4.00 MiB Total PE 238466 Alloc PE / Size 238466 / 931.51 GiB Free PE / Size 0 / 0 VG UUID fYgPgc-cXaW-6ryU-nU1v-5m3F-Rj0r-Afe9p6 --- Volume group --- VG Name vg_workspace System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 0 Max PV 0 Cur PV 1 Act PV 1 VG Size 111.80 GiB PE Size 4.00 MiB Total PE 28620 Alloc PE / Size 28620 / 111.80 GiB Free PE / Size 0 / 0 VG UUID 9OkF0O-E5f7-bfvH-qAfb-E7Wh-cOGS-EhVydH --- Volume group --- VG Name vg_root System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 223.37 GiB PE Size 4.00 MiB Total PE 57183 Alloc PE / Size 57183 / 223.37 GiB Free PE / Size 0 / 0 VG UUID 2Q5ooC-B0ZW-BcxA-jufX-80zh-hkQh-qYNm2G --- Volume group --- VG Name vg_var System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 2 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 405.09 GiB PE Size 4.00 MiB Total PE 103703 Alloc PE / Size 103703 / 405.09 GiB Free PE / Size 0 / 0 VG UUID mIgZdQ-4tzM-T2KP-S2f7-QA14-iwp9-cTwyly
[root@dart bwilson]# lvdisplay
--- Logical volume --- LV Path /dev/vg_black/lv_swap LV Name lv_swap VG Name vg_black LV UUID 3ZtM9o-a5Z8-iAs9-Sm1K-O1zU-jfLM-iKX7pj LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:26:11 -0800 LV Status available # open 1 LV Size 15.62 GiB Current LE 4000 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:1 --- Logical volume --- LV Path /dev/vg_black/lv_home LV Name lv_home VG Name vg_black LV UUID rwvORk-YHTr-jmMR-lxey-Q0Ue-NwTO-mOcrUO LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:26:12 -0800 LV Status available # open 1 LV Size 915.88 GiB Current LE 234466 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:2 --- Logical volume --- LV Path /dev/vg_workspace/lv_workspace LV Name lv_workspace VG Name vg_workspace LV UUID wUhofL-LtjP-p6Rr-wMj0-jKRF-X2fB-Bjyy9H LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-09 18:29:17 -0800 LV Status available # open 0 LV Size 111.80 GiB Current LE 28620 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:5 --- Logical volume --- LV Path /dev/vg_root/lv_root LV Name lv_root VG Name vg_root LV UUID 5cCyeM-yuTM-GsfE-M4Xu-em5h-6olD-sMucxV LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:26:00 -0800 LV Status available # open 1 LV Size 223.37 GiB Current LE 57183 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 --- Logical volume --- LV Path /dev/vg_var/lv_var LV Name lv_var VG Name vg_var LV UUID GZFOdu-D6uN-UCAR-Gt3R-Ddni-2dGd-ZNmhNh LV Write Access read/write LV Creation host, time dart.alseageo.com, 2012-12-08 02:25:17 -0800 LV Status available # open 1 LV Size 405.09 GiB Current LE 103703 Segments 1 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:3
/etc/fstab
/dev/mapper/vg_root-lv_root / ext4 defaults,noatime,discard 1 1 UUID=488a0c3b-689e-4f2b-8688-d8eefdf0a7f8 /boot ext4 defaults,noatime,discard 1 2 /dev/mapper/vg_black-lv_home /home ext4 defaults,noatime 1 2 /dev/mapper/vg_var-lv_var /var ext4 defaults,noatime 1 2 /dev/mapper/vg_black-lv_swap swap swap defaults 0 0 /dev/mapper/vg_workspace-lv_workspace /workspace ext4 defaults,noatime,discard 1 2 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 # SSD optimizations none /tmp tmpfs defaults,noatime,mode=1777 0 0
CentOS install
Install server system from DVD
vi /etc/sysconfig/network-scripts/ifcfg-eth0 vi /etc/resolv.conf service network restart yum -y install emacs yum -y install openssh-clients yum -y install gcc unzip yum -y install hdparm yum -y install httpd yum update reboot
Basic software install
PostGIS
Install PostGIS 2.0 See PostGIS page
PHP 5
Build php so that you can use your own PostgreSQL build. See Ben Ramsey's page
Using the standard php won't work because it pulls in postgresql 8.4 client library
Likewise this means installing psycpg2 as well.