EVMS
Un article de ChtilandWiki.
[modifier] IntroductionThis page will show how to install a new system using EVMS and the Gentoo 2005.1 install instructions. I will assume that you are familiar with the general gentoo install process, but are new to EVMS. I will talk through the concepts needed, and will assume you are using the NCURSES (evmsn) user interface. Throughout this process you will find the EVMS User Guide essential reading, although it focuses more on the mechanics of making a change. This page will focus instead on the higher level steps you need to accomplish to set up your system. I will also assume you are familar with general raid concepts. If you are not then you really have no business setting up EVMS. Go search for "raid tutorial" first. The EVMS web site has a very good overview paper by Corry and Dobbelstein. Be warned though: some of the concept names that they use are different from the current EVMS names in the UI. This walk through will assume a relatively simple hardware environment: a single server with two identical local disks. There are many, many different ways you could set up a system like this. To simplify this walkthrough I'll be handling a single case, which will illustrate many of the EVMS features:
We will be doing all native EVMS partitions; LVM2 compatibility containers will not be used (EVMS has the ability to talk directly to LVM partitions in a compatibility mode). An earlier version of this walk through used Bad Block Relocation. Comments on the EVMS mailing list and my personal experiences had me stop using BBR - there were reliability problems when using BBR. The EVMS mailing list suggests making a kernel patch on 2.6.12 systems. The fix is integrated with 2.6.13. For more information see this Mailing List Message. (note: 2.6.12-gentoo-r9 has this bug already patched) [modifier] EVMS usage overviewThe EVMS User Guide is extremely detailed about the syntax of each command. What I found difficult (and had to figure out by trial and error) was what sequence of EVMS objects would accomplish what I wanted. This picture really says everything. You can pretty much ignore the rest of the walkthrough if you understand it: Image:Http://gentoo-wiki.com/images/6/63/Evms.png [modifier] Getting StartedBoot 2005.1, using either the CD or network. I used this boot line, but yours may quite possibly differ: Modèle:Commandline My disks are called /dev/sda and /dev/sdb; yours may be the same or /dev/hda and /dev/hdb. Throughout this walkthrough I'll be using sda and sdb. [modifier] Starting EVMSAfter your system reboots you need to start the EVMS subsystem. Beginners may have read right past this step, but it is completely essential: [modifier] Start the EVMS ncurses interfaceYou could use the command line interface instead, but I'm going to demonstrate using the ncurses interface. Its much easier for beginners to figure out what is going on with the sort-of-visual UI. At the end of each section I'll give the equivalent Modèle:Commandline commands. Start the EVMS ncurses interface: I'm not going to spend much time talking through how to drive the interface when the EVMS users guide has all the dirt. But you need to understand that there are several screens that show the current evms status (switch screens using TAB) and that you navigate the menu system using letters, arrow keys, and Enter. After you start evmsn you should see a screen like this: Actions Settings Help 0=Logical Volumes Name Size Modified Active R/O Plug-in Mountpoint ────────────────────────────────────────────────────────────────────────────── /dev/evms/loop0 41.1 MB X /mnt/livecd This is showing the livecd loopback filesystem that was used to boot the system. Tab to the next screen; you should see your two disks: Actions Settings Help 5=Disk Segments Name Size Modified Active R/O Type Offset Plug-in ────────────────────────────────────────────────────────────────────────────── sda_mbr 31 KB Meta Data 0 DosSegM sda_freespace1 74.5 GB Free Spac 63 DosSegM sdb_mbr 31 KB Meta Data 0 DosSegM sdb_freespace1 74.5 GB Free Spac 63 DosSegM If you are not seeing sda_mbr and sda_freespace1 (and the same for sdb) then something has gone terribly wrong; you should fix it before proceeding. You may need to write an empty dos partition table using fdisk or cfdisk. [modifier] Create the boot partitionsGrub and LILO are pretty stupid; they know nothing about EVMS, nor the kernel implementation of the device mapper (which does the actual run-time magic). We therefore need to create a boot partition with the minimum necessary magic for Grub (which I'll be using because the instructions are much simpler; for Lilo see EVMS boot loader page). The basic strategy is that we will be creating a normal dos partition at the start of each disk to hold the boot partition. These partitions will be mirrored to each other, but the underlying raw partitions will be usable by Grub. Because Grub is not EVMS aware, we will not be using any of the more advanced EVMS features like bad block mapping or growable regions for the boot partition. [modifier] Create the raw dos partitionsFirst create a 512 MB dos partition on sda: Repeat the commands above to create a 512 mb boot partition on sdb also. After you have done this you should see the following disk segments (remember to use Tab to switch between screens if you are not seeing screen 5=Disk Segments at the top): Actions Settings Help 5=Disk Segments Name Size Modified Active R/O Type Offset Plug-in ────────────────────────────────────────────────────────────────────────────── sda_mbr 31 KB X Meta Data 0 DosSegM sda1 509.8 MB X Data 63 DosSegM sda_freespace1 74.0 GB Free Spac 1044225 DosSegM sdb_mbr 31 KB X Meta Data 0 DosSegM sdb1 509.8 MB X Data 63 DosSegM sdb_freespace1 74.0 GB Free Spac 1044225 DosSegM And there is a new screen that wasn't available before: screen 1=Available Objects. Tab to it and you should see something like this: Actions Settings Help 1=Available Objects Name Size Modified Active R/O Plug-in ────────────────────────────────────────────────────────────────────────────── sda1 509.8 MB X DosSegMgr sdb1 509.8 MB X DosSegMgr [modifier] Create a RAID-1 MirrorNow link those two raw partitions together to a single raid mirror: Screen 1 (Available Objects) should now show the newly created md0 region: Actions Settings Help 1=Available Objects Name Size Modified Active R/O Plug-in ────────────────────────────────────────────────────────────────────────────── md/md0 509.8 MB X MDRaid1RegMgr [modifier] Create an EVMS volumeYou now need to take the raid region and turn it into an EVMS volume: Screen 0 (Logical Volumes) should show the newly created volume: Actions Settings Help 0=Logical Volumes Name Size Modified Active R/O Plug-in Mountpoint ────────────────────────────────────────────────────────────────────────────── /dev/evms/boot 509.7 MB X /dev/evms/loop0 41.1 MB X /mnt/livecd [modifier] Make a filesystemFinally you have to make a filesystem. I'm going to use ext3: Screen 0 (Logical Volumes) will now show /dev/evms/boot as having an Ext2/3 filesystem on it: Actions Settings Help 0=Logical Volumes Name Size Modified Active R/O Plug-in Mountpoint ────────────────────────────────────────────────────────────────────────────── /dev/evms/boot 509.7 MB X Ext2/3 /dev/evms/loop0 41.1 MB X /mnt/livecd You've now successfully completed the boot partition! Its now time to try something more challenging. [modifier] Quick Command Line VersionHere is the identical series of steps outlined above using the Modèle:Commandline command line program:
Create: Segment,sda_freespace1,size=512MB
Create: Segment,sdb_freespace1,size=512MB
Create: Region,MDRaid1RegMgr={},sda1,sdb1
Create: Volume, "md/md0", Name="boot"
Mkfs: Ext2/3={vollabel=boot}, /dev/evms/boot
Save
Quit
[modifier] Swap spaceRepeat the above steps making a swap partition intead of a boot partition. I created my swap partition using RAID 0 instead of RAID 1; to do this select the RAID 0 region manager instead of the RAID 1 region manager. [Comment from JaredThirsk: Note that the linux kernel automatically stripes swap partitions as long as are mounted at the same priority, so some people believe it is best not to use RAID0 for the swap partitions and let the kernel handle striping. Another approach is to use RAID1 mirroring to ensure swap memory is still valid if a disk goes down.] I'm not going to include all the screen captures; its really pretty much the same sequence of steps [modifier] Quick Command Line VersionHere are the steps to create a RAID 0 set of swap partitions:
Create: Segment,sda_freespace1,size=2GB
Create: Segment,sdb_freespace1,size=2GB
Create: Region,MDRaid0RegMgr={},sda2,sdb2
Create: Volume, "md/md1", Name="swap"
Mkfs: SWAPFS={}, /dev/evms/swap
Save
Quit
[modifier] LVM ContainersThe previous example allowed you to have mirrored boot partitions, but didn't really add much extra functionality. This section will show how to use one of the most useful EVMS features: containers. Basically a container separates the physical disk from logical usage. It allows you to grow (or shrink) a filesystem as needed by dividing the disk into 32 MB chunks; you can then map these chunks at will into particular volumes. Corry and Dobbelstein have a nice overview of this; go to slide 12. They call it Volume Groups; the EVMS interface call it Containers. [modifier] Create a dos partition for the rest of the diskFirst create the raw dos partitions: Repeat that for sdb_freespace1 You should now see two available objects (you may need to hit tab a few times to get to the Available Objects screen): Actions Settings Help 1=Available Objects Name Size Modified Active R/O Plug-in ────────────────────────────────────────────────────────────────────────────── sda3 72.0 GB X DosSegMgr sdb3 72.0 GB X DosSegMgr And these disk segments. Sda3 and sdb3 are the ones that should have just been created: Actions Settings Help 5=Disk Segments Name Size Modified Active R/O Type Offset Plug-in ────────────────────────────────────────────────────────────────────────────── sda_mbr 31 KB X Meta Data 0 DosSegM sda1 509.8 MB X Data 63 DosSegM sda2 2.0 GB X Data 1044225 DosSegM sda3 72.0 GB X Data 5237190 DosSegM sda_freespace1 905 KB Free Spac 156248190 DosSegM sdb_mbr 31 KB X Meta Data 0 DosSegM sdb1 509.8 MB X Data 63 DosSegM sdb2 2.0 GB X Data 1044225 DosSegM sdb3 72.0 GB X Data 5237190 DosSegM sdb_freespace1 905 KB Free Spac 156248190 DosSegM [modifier] Bad Block RelocationI used to show Bad Block Relocation, but the 2.6.12-r6 kernel included on the 2005.1 disk doesn't work with BBR so I've left it out of this walkthrough. [modifier] Mirror the two segments togetherCreate a RAID 1 region of the two segments you just created: You should now see md/md2 on the Available Objects screen: Actions Settings Help 1=Available Objects Name Size Modified Active R/O Plug-in ────────────────────────────────────────────────────────────────────────────── md/md2 72.0 GB X MDRaid1RegMgr [modifier] Create an LVM2 containerNow create the container: Actions Settings Help 3=Storage Regions Name Size Modified Active R/O Corrupt Plug-in ────────────────────────────────────────────────────────────────────────────── lvm2/main/Freespace 72.0 GB LVM2 md/md1 4.0 GB X MDRaid0RegMgr md/md0 509.8 MB X MDRaid1RegMgr md/md2 72.0 GB X MDRaid1RegMgr [modifier] Create Storage RegionsOnce you have a container you need to create the actual storage regions. We will be creating one region: for the root. You could obviously make more, or you could keep your free space available. Unlike non-LVM filesystems you don't have to worry overmuch about how big to make each partition; you can always grow or shrink a filesystem as long as it is allocated from an LVM container. I'm using a root size of 16 gb; feel free to adjust this value to suit your needs. If things went well you should see this available object: Actions Settings Help 1=Available Objects Name Size Modified Active R/O Plug-in ────────────────────────────────────────────────────────────────────────────── lvm2/main/root 16.0 GB X LVM2 [modifier] EVMS VolumesYou now have to package this region up into an EVMS Volume: This new volume should show up on the Logical Volumes Screen: Actions Settings Help 0=Logical Volumes Name Size Modified Active R/O Plug-in Mountpoint ────────────────────────────────────────────────────────────────────────────── /dev/evms/boot 509.7 MB X Ext2/3 /dev/evms/loop0 41.1 MB X /mnt/livecd /dev/evms/root 16.0 GB X /dev/evms/swap 4.0 GB X [modifier] Make FilesystemsFinally tie the root volume to a filesystem: [modifier] Save and Quit out of EVMSNOr, more exactly Quit and Save. Evmsn asks if you want to save when quitting; so select Quit and then Save: You should see some kernel messages about "raid array is not clean -- starting background reconstruction". This is a good thing and means that things were set up correctly. [modifier] Quick Command Line VersionHere are the steps to create a RAID 1 root of 16 gb on an LVM2 container:
Create: Segment,sda_freespace1,
Create: Segment,sdb_freespace1,
Create: Region,MDRaid1RegMgr={},sda3,sdb3
Create: Container,LVM2={name="main"},md/md2
Create: Region, LVM2={name="root", size=16gb},lvm2/main/Freespace
Create: Volume, "lvm2/main/root", Name="root"
Mkfs: ReiserFS={vollabel=root}, /dev/evms/root
Save
Quit
[modifier] Continue with the installYou can now continue on with your install. EVMS has created your partitions for you, so you can pick up at the swapon command in the Preparing the Disks page. swapon /dev/evms/swap mount /dev/evms/root /mnt/gentoo mkdir /mnt/gentoo/boot mount -t ext3 /dev/evms/boot /mnt/gentoo/boot
[modifier] Setting up boot loader[modifier] A. Booting with Gentoo 2005.1 and newer and a recent kernelConcerning booting with EVMS we have two upcoming issues. If you are using a current kernel from 2.6 series (this is default on 2005.1), then your will run into the following trap: The linux kernel locks the whole disk, from which the root file system is mounted. That implies, that if your are booting from for instance /dev/sda2 then evms cannot install it's naming scheme and only /dev/sda2 /dev/vg00/root_fs are possible. It is not possible to use evms in this case. It is neccessary to use an initrd, which issues the command evms_activate PRIOR to mounting ANY root file system -> This makes it possible to use evms normally. The root partition would be called then /dev/evms/sda2. The cheapest way to set this up under Gentoo is: 2) emerge evms and compile a recent kernel with genkernel: USE=static emerge evms emerge genkernel genkernel --menuconfig --evms2 all This compiles a complete kernel plus modules, necessary for evms usage AND prepares a standardized gentoo initramfs, which can issue the famous evms_activate. 3) edit your boot loader to have the following kernel parameters (this is for grub and gentoo-sources 2.6.12-gentoo-r10): title Gentoo 2005.1 booting with evms support kernel /kernel-gentoo-x86-2.6.12-r10 root=/dev/ram0 real_root=/dev/evms/sdb2 init=/linuxrc udev doevms2 initrd /initramfs-gentoo-x86-2.6.12-r10 ATTENTION: Most important is the parameter "doevms2", which triggers evms_activate.
Take care, that /etc/fstab uses completely the evms device naming scheme, i.e.: /dev/evms/md0, /dev/evms/lvm2/vg00/root, etc.... Questions to this procedure please send to cchrist at mcpsoftworks dot com [modifier] B. Gentoo prior to 2005.1 and older kernelsBecause we put the root filesystem on EVMS, we have to take some extra steps to boot the system. The EVMS web site has a page that talks about this. I'll summarize the steps here. The basic strategy is to boot into an initrd image, that will activate your evms root partition. You need to grab the image from the EVMS download page. Grab the image that corresponds to the EVMS version you emerged (at the time this page was written that was 2.5.2). Copy that file to /boot (in our case: /boot/evms-2.5.2-initrd.gz). Set up your grub.conf file: title gentoo on sda kernel (hd0,0)/bzImage root=/dev/evms/root ramdisk=8192 initrd (hd0,0)/evms-2.5.2-initrd.gz title gentoo on sdb kernel (hd1,0)/bzImage root=/dev/evms/root ramdisk=8192 initrd (hd1,0)/evms-2.5.2-initrd.gz Note that things are set up to be able to boot off of either disk drive, in case the first drive is kaput, since grub can't tell when an evms drive is off line but still present. Make sure you install grub on both disks: # grub device (hd0) /dev/sda root (hd0,0) setup (hd0) device (hd0) /dev/sdb root (hd0,0) setup (hd0) Finally, you will need to create an /initrd directory so the initrd ram image can pivot_root to your "real" root: mkdir /initrd [modifier] Setting up fstabSetting up fstab is pretty normal, but the names of the mount points need to be the EVMS mount points: /dev/evms/boot /dev/evms/swap /dev/evms/root
[modifier] Building your kernelFirst of all: when in doubt you could check the EVMS kernel build page, especially if you are running a 2.4 kernel. Note that there is a known kernel bug with RAID1 and RAID 5 in 2.6.12 (fixed since 2.6.12-gentoo-r9): forum post kernel patch You only need the loopback device if you are going to modify the EVMS initrd partition. You only need those raid and device mapper capabilities that you actually are using on your system. The EVMS initrd assumes that all Raid, device mapper, and disk drivers needed to boot are compiled into the kernel; the initrd does not load any modules. If this doesn't work for you there are directions on the EVMS web site for modifying the init-ramdisk image (you will need to scroll the page down...) [modifier] Change checkrootBy default the gentoo startup files start EVMS in the /etc/init.d/checkfs script. Since in our case the root filesystem is on evms this is too late; evms needs to be started in the /etc/init.d/checkroot script. Change the top of the start() function in /etc/init.d/checkroot to this: The line you need to really add is the one that calls start_volumes; this will eventually call evms_activate.
[modifier] Things that should be talked aboutThere are some topics that this page doesn't cover and should be talked about:
|
|
