ASM on Red Hat 7 and OEL 7 with UDEV



I know UDEV, at least the word, since a quite long time but I have never tried it in a concrete example. While discussing in the OCM workshop I came to know that ASMLib was not mandatory to install Automatic Storage Management (ASM). Also Oracle has not been super fast to provide the ASMLib library for Red Hat 7, their initial communication was that they are not going even to provide it on Red Hat 7. Apparently now you can download the one of OEL 7 and install it on Red Hat 7. I’m not even mentioning all blog posts and forum discussions about the obsolescence of ASMLib versus UDEV. Some even mentioned that ASMLib brings nothing versus the native kernel UDEV feature. At least from what I have tested it eases the management of disks.

While trying to implement UDEV I have found tons of blog posts on Red Hat 5 and 6 but nothing really on Red Hat 7, or at least with the latest capabilities of UDEV rules.

Very high level UDEV is a dynamic device manager managing your entities called nodes in /dev directory. What is worth to have in mind is that after a reboot your disk devices may change their names which might be a small problem for your database. This tricky situation can be solved by writing UDEV rules to have a consistent naming convention for your devices.

For licensing reason I have tested using a virtual machine using Oracle Linux Server release 7.1 (equivalent of Red Hat Enterprise Linux Server release 7.1 (Maipo)) and Oracle Grid Infrastructure

UDEV Linux configuration

What I’m preparing in the background is a RAC installation using two virtual machine running under VirtualBox. For this I have attached to my first virtual machine four fixed size of 1GB disks like this:


To attach those disks in another virtual machine go to Virtual Media Manager and change their property to shareable like this:


Once you start your guest OS you should see something:

[root@server3 ~]# fdisk -l
Disk /dev/sdb: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sde: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

In many blog post I have seen creation of a unique primary partition on those disks with:

[root@server3 ~]# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xa4cc3f96.
Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-2097151, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2097151, default 2097151):
Using default value 2097151
Partition 1 of type Linux and of size 1023 MiB is set
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

Or use parted if you want to automate the creation with a shell script:

[root@server3 ~]# parted /dev/sdd mklabel msdos
Information: You may need to update /etc/fstab.
[root@server3 ~]# parted /dev/sdd mkpart primary 1m 100%
Information: You may need to update /etc/fstab.
[root@server3 ~]# fdisk -l /dev/sdd
Disk /dev/sdd: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000df279
   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1            2048     2097151     1047552   83  Linux
[root@server3 ~]# parted /dev/sdb print
Disk /dev/sdb: 1074MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1074MB  1073MB  primary

But honestly I’m wondering the need of this unique primary partition filling entirely the disk. Apparently it is needed only with ASMLib so as not using it decided to give a try without creating them…

So deleted the one I created (fdisk can also be used):

[root@server3 rules.d]# parted /dev/sde rm 1
Information: You may need to update /etc/fstab.

The idea of UDEV is to associate the fixed device id to a more understandable name. On many blog posts you will see the usage of iscsi_id binary. That moved from /sbin in Red Hat 5 and 6 to /usr/lib/udev in Red Hat 7.

[root@server3 ~]# for disk in `ls /dev/sd*`
                    echo $disk
                    /usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=$disk

Which can now be replaced (since Red Hat 6) with udevadm (udevadm info -h for more information):

[root@server3 ~]# udevadm info --query=property /dev/sdb

The target is to write UDEV rules to associate our /dev/sdb to /dev/sde disk devices to symbolic names with a taste of ASMLib i.e. /dev/oracleasm/disks/DISKxx. To have a first insight of how to write UDEV rules you can start with man udev command. When using legacy method it gives below rules that you see on many articles, I have decided to put the rules in Oracle suggested file i.e. 99-oracle-asmdevices.rules located in /etc/udev/rules.d directory:

[root@server3 ~]# head -n 1 /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?", PROGRAM=="/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB420a8d94-2a375f5b", SYMLINK+="oracleasm/disks/DISK01", OWNER="oracle", GROUP="dba", MODE="0660"

I rate it a bit cumbersome and hard to read, with newest UDEV rules it can gives something brighter like below:

[root@server3 ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VB420a8d94-2a375f5b", SYMLINK+="oracleasm/disks/DISK01", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VB63ab3b43-33327cc0", SYMLINK+="oracleasm/disks/DISK02", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VBce02798e-7d13dc9a", SYMLINK+="oracleasm/disks/DISK03", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?", ENV{ID_SERIAL}=="VBOX_HARDDISK_VB9c27c116-7154ebea", SYMLINK+="oracleasm/disks/DISK04", OWNER="oracle", GROUP="dba", MODE="0660"

Easier isn’t it ? Notice the sd? to take all SATA disk, even the system one but as we have not specified its serial id no rule will apply to it. The ENV{ID_SERIAL} match the serial ID of the disk to a generic name, so even if after reboot your devices names have changed the serial ID will not change. The SYMLINK will create a more understandable link to our devices and OWNER, GROUP and MODE gives ownership and permission.

Reload and apply the UDEV rules with:

[root@server3 ~]# udevadm control --reload-rules
[root@server3 ~]# udevadm trigger

Then you can see the link nodes that have been created:

[root@server3 ~]# ll /dev/oracleasm/disks/
total 0
lrwxrwxrwx 1 root root 9 Jul  2 12:33 DISK01 -> ../../sdb
lrwxrwxrwx 1 root root 9 Jul  2 12:33 DISK02 -> ../../sdc
lrwxrwxrwx 1 root root 9 Jul  2 12:33 DISK03 -> ../../sdd
lrwxrwxrwx 1 root root 9 Jul  2 12:33 DISK04 -> ../../sde

You can test your UDEV rules with:

[root@server3 ~]# udevadm test /sys/block/sdb

The rights are set only devices but not on symlinks:

[root@server3 hugepages]# ll /dev/sd*
brw-rw---- 1 root   disk 8,  0 Jul  6 13:04 /dev/sda
brw-rw---- 1 root   disk 8,  1 Jul  6 13:04 /dev/sda1
brw-rw---- 1 root   disk 8,  2 Jul  6 13:04 /dev/sda2
brw-rw---- 1 oracle dba  8, 16 Jul  7 15:45 /dev/sdb
brw-rw---- 1 oracle dba  8, 32 Jul  7 15:46 /dev/sdc
brw-rw---- 1 oracle dba  8, 48 Jul  6 13:27 /dev/sdd
brw-rw---- 1 oracle dba  8, 64 Jul  6 13:27 /dev/sde

ASM configuration with UDEV

Once the UDEV rules have been created you can install Grid Infrastructure and when done use asmca to create your ASM instance and create your first ASM device group on your newly created disks (I have first changed the discovery path to /dev/oracleasm/disks):


I got the “Oracle Grid Infrastructure is not configured properly. ASMCA needs Oracle Grid Infrastructure to configure ASM” error:


What I did not notice when executing Grid Infrastructure is the below message:

To configure Grid Infrastructure for a Stand-Alone Server run the following command as the root user:
/u01/app/12.1.0/grid/perl/bin/perl -I/u01/app/12.1.0/grid/perl/lib -I/u01/app/12.1.0/grid/crs/install /u01/app/12.1.0/grid/crs/install/
To configure Grid Infrastructure for a Cluster execute the following command as oracle user:
This command launches the Grid Infrastructure Configuration Wizard. The wizard also supports silent operation, and the parameters can be passed through the response file that is available in the installation media.

Once script executed the configuration went better. I had few others error message but they were all related to memory allocation of my virtual machine. Oracle Grid Infrastructure in 12cR1 is consuming almost 1 GB of memory so you need a quite big /dev/shm (MEMORY_TARGET parameter).

Once the ASM instance has been created even without any partition created on disks everything looks fine:

SQL> col PATH FOR a30
SQL> SET lines 200
SQL> SELECT name,total_mb FROM v$asm_diskgroup;
NAME                             TOTAL_MB
------------------------------ ----------
DATA                                 3072
SQL> SELECT name,PATH,mode_status,os_mb FROM v$asm_disk
NAME                           PATH                           MODE_ST      OS_MB
------------------------------ ------------------------------ ------- ----------
                               /dev/oracleasm/disks/DISK01    ONLINE        1024
DATA_0000                      /dev/oracleasm/disks/DISK02    ONLINE        1024
DATA_0002                      /dev/oracleasm/disks/DISK04    ONLINE        1024
DATA_0001                      /dev/oracleasm/disks/DISK03    ONLINE        1024


One thought on “ASM on Red Hat 7 and OEL 7 with UDEV

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>