Symantec SmartIO for Oracle databases

Posted on , updated on by Yannick Jaquier
Oracle

Table of contents

Preamble

This blog post is about activating Symantec SmartIO for Solid State Drives and use it to cache important figures of an Oracle database. SmartIO feature (part of storage foundation) aim at caching in read or writeback mode I/O perform on files managed by SmartIO cache. This post is a follow up of Fusion I/O installation and benchmarking post.

Writeback mode allows you to return to application once the I/O has been written on flash cache area. Read mode simply put blocks read on disk at first time for second and further faster access, at each write SmartIO synchronize cache area with master disk information.

Even if attractive I have not planned to writeback mode has it deserves more infrastructure to ensure files are not in an inconsistent state in case of crash (cache replica). Second our target database will be using an Operating System cluster (Veritas Cluster Server) so switching from one node to slave one would trigger additional step to ensure all I/Os have been written on disk…

The test server I have used has a fusion I/O card that we have tested in previous blog post and I have an 11.2.0.4 Oracle enterprise edition database. The recommended feature to implement is called SmartIO caching for Oracle databases on VxFS file systems (that has preference on SmartIO caching for databases on VxVM volumes according to Symantec support).

My test have been done using Symantec Storage Foundation 6.1 running on Red Hat Enterprise Linux Server release 6.4 (Santiago).

SmartIO configuration

Start by adding in /etc/sudoers (visudo) the following line to give access to fscache binary to your Oracle Linux account (oradwh for me):

oradwh ALL=(ALL) NOPASSWD: /opt/VRTS/bin/fscache

Remark
The fscache is not a typo mistake 🙂

I’m starting from a Veritas VxVM device group created on the fusion I/O card. Just to confirm there is nothing yet configured:

[root@server1 ~]# sfcache list
NAME                             TYPE   SIZE       ASSOC-TYPE  STATE      DEVICE

Now we need to create the cache area, I started by providing directly the device group:

[root@server1 ~]# sfcache create --noauto fusionio
SFCache ERROR V-5-2-6281 No valid disk(s) provided to create cache area
[root@server1 ~]# vxdg list
NAME         STATE           ID
vgp3801      enabled,cds          1404731405.184.server1
fusionio     enabled,cds          1405514597.10441.server1

But realized I had to provide a VxVM device group and a logical volume created on it:

[root@server1 ~]# vxassist -g fusionio maxsize
Maximum volume size: 712802304 (348048Mb)
[root@server1 ~]# vxassist -g fusionio make lvol01 712802304
[root@server1 ~]# vxassist -g fusionio maxsize
VxVM vxassist ERROR V-5-1-15809 No free space remaining in diskgroup fusionio with given constraints

Second trial:

[root@server1 ~]# sfcache create --noauto fusionio/lvol01
SFCache ERROR V-5-2-0 Caching is not applicable for specified volume as DG version should be greater than 180

I have to upgrade the device group version:

[root@server1 ~]# vxdg -q list fusionio |grep version
version:   180
[root@server1 ~]# vxdg upgrade fusionio
[root@server1 ~]# vxdg -q list fusionio |grep version
version:   190

Third trial:

[root@server1 ~]# sfcache create -t vxfs --noauto fusionio/lvol01
[root@server1 ~]# sfcache list
NAME                             TYPE   SIZE       ASSOC-TYPE  STATE      DEVICE
fusionio/lvol01                  VxFS   339.89g    NOAUTO      ONLINE     fusionio0_0
[root@server1 ~]# sfcache list -l
Cachearea: fusionio/lvol01
Assoc Type: NOAUTO
Type: VxFS
Size: 339.89g
State: ONLINE
 
FSUUID                                                  SIZE  MODE       MOUNTPOINT
000000000000000000000000000000000000000000000000        0 KB  nocache    /ora_dwh
4edee453af0c0f0064af0000cda6e12ee70300000b82ba53        0 KB  nocache    /ora_dwh/arch
92dee453d3c40b00feaf0000e3077658e70300001283ba53        0 KB  nocache    /ora_dwh/ctrl
99dee4531a54070006b0000052fc9e5ae70300002c83ba53        0 KB  nocache    /ora_dwh/data02
9edee4536a2306000ab000008b76b35be70300002e83ba53        0 KB  nocache    /ora_dwh/data03
a0dee4539a8c08000eb00000c1f0c75ce70300003083ba53        0 KB  nocache    /ora_dwh/data04
a4dee453da5e000012b00000f96adc5de70300001482ba53        0 KB  nocache    /ora_dwh/dump
a8dee4539ab0020016b0000031e5f05ee70300003c83ba53        0 KB  nocache    /ora_dwh/index01
aadee45331eb02001ab00000675f0560e70300004283ba53        0 KB  nocache    /ora_dwh/index02
acdee453b90a01001eb000009dd91961e70300004583ba53        0 KB  nocache    /ora_dwh/index03
aedee4538983030022b00000d3532e62e70300004983ba53        0 KB  nocache    /ora_dwh/index04
b2dee45389fa040026b000000bce4263e70300001c82ba53        0 KB  nocache    /ora_dwh/rbs
b8dee453a1550a002bb00000d2669c64e70300001d82ba53        0 KB  nocache    /ora_dwh/software
bcdee45318b30c002fb000000ae1b065e70300005783ba53        0 KB  nocache    /ora_dwh/sys
c0dee4536066020033b00000425bc566e70300005d83ba53        0 KB  nocache    /ora_dwh/temp01
c3dee453185a060037b0000079d5d967e70300005f83ba53        0 KB  nocache    /ora_dwh/temp02
c7dee45300d507003db00000cb8c7869e703000000fac853        0 KB  nocache    /ora_dwh/data_tmp
fbdbe45314380d00d2ad000010ab5742e7030000f882ba53        0 KB  nocache    /ora_dwh/log
eba6e4535d5d0b004f740000d96e2b3be70300002983ba53        0 KB  nocache    /ora_dwh/data01

No interesting statistics so far:

[root@server1 ~]# sfcache stat fusionio/lvol01
 
TYPE: VxFS
NAME: fusionio/lvol01
               Cache Size:   339.9 GB
        Cache Utilization:    94.5 MB ( 0.03 %)
 File Systems Using Cache:       0
Writeback Cache Use Limit:  Unlimited
  Writeback Flush Timelag:      10 s
 
Read Cache                                Writeback
Hit Ratio    Data Read    Data Written    Hit Ratio    Data Written
 
Total:
   0.00 %         0 KB            0 KB       0.00 %            0 KB

Oracle configuration

My Oracle Linux account (oradwh) has its environment well set up:

[oradwh@server1 ~]$ env |grep ORA
ORACLE_SID=dwh
ORACLE_BASE=/ora_dwh
ORACLE_HOME=/ora_dwh/software

So issuing the documented command:

[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o setdefaults --type=olap
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
INFO: Setting olap policies
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo01.log
UX:vxfs fscache: ERROR: V-3-28050: Failed to set caching mode to /ora_dwh/log/dwh/redo01.log: Inappropriate ioctl for device
SFCache ERROR V-5-2-0 Unknown Error. Please check logs at /etc/vx/log/sfcache.log .

The sfcache.log file is not helping at all…

Investigating why the command is failing (with the help of Symantec support) I tried to set the VxFS option to an Oracle mount point and got below error:

[root@server1 ~]# mount -t vxfs -o remount,smartiomode=read /dev/vx/dsk/vgp3801/lvol13 /ora_dwh/log
UX:vxfs mount.vxfs: ERROR: V-3-28217: smartiomode=read mount option supported only on layout versions >= 10.

So tried to upgrade the VxFS layout:

[root@server1 ~]# /opt/VRTS/bin/fstyp -v /dev/vx/dsk/vgp3801/lvol13 | grep version
magic a501fcf5  version 9  ctime Mon 07 Jul 2014 01:22:32 PM CEST
[root@server1 ~]# /opt/VRTS/bin/vxupgrade
UX:vxfs vxupgrade: INFO: V-3-22568: usage: vxupgrade [-n new_version] [-r rawdev] mount_point
[root@server1 ~]# /opt/VRTS/bin/vxupgrade -n 10 /ora_dwh/log
[root@server1 ~]# /opt/VRTS/bin/fstyp -v /dev/vx/dsk/vgp3801/lvol13 | grep version
magic a501fcf5  version 10  ctime Mon 07 Jul 2014 01:22:32 PM CEST

Now all is working fine:

[root@server1 ~]# mount -t vxfs -o remount,smartiomode=read /dev/vx/dsk/vgp3801/lvol13 /ora_dwh/log
[root@server1 ~]# mount | grep /ora_dwh/log
/dev/vx/dsk/vgp3801/lvol13 on /ora_dwh/log type vxfs (rw,delaylog,largefiles,ioerror=mwdisable,smartiomode=read)

Now the command is going a bit further but still failing for:

[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o setdefaults --type=olap
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
INFO: Setting olap policies
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo01.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo02.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo03.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo04.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo05.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo06.log
INFO: Setting nocache mode to /ora_dwh/software/dbs/arch
UX:vxfs fscache: ERROR: V-3-28032: Failed to stat /ora_dwh/software/dbs/arch: No such file or directory
SFCache ERROR V-5-2-0 Unknown Error. Please check logs at /etc/vx/log/sfcache.log .

Which I corrected, at Oracle level, with (even if my database is in NOARCHIVELOG mode I had to set it):

SQL> ALTER SYSTEM SET log_archive_dest_1='location="/ora_dwh/arch/dwh"' scope=both;
 
SYSTEM altered.

I had again the same Inappropriate ioctl for device error message for another file system:

[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o setdefaults --type=olap
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
INFO: Setting olap policies
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo01.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo02.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo03.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo04.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo05.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo06.log
INFO: Setting nocache mode to /ora_dwh/arch/dwh
INFO: Setting read mode to /ora_dwh/temp01/dwh/temp01.dbf
UX:vxfs fscache: ERROR: V-3-28050: Failed to set caching mode to /ora_dwh/temp01/dwh/temp01.dbf: Inappropriate ioctl for device
SFCache ERROR V-5-2-0 Unknown Error. Please check logs at /etc/vx/log/sfcache.log .

So decided to upgrade all file systems to layout 10 and finally the command worked fine this time:

[oradwh@server1 dbs]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o setdefaults --type=olap
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
INFO: Setting olap policies
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo01.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo02.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo03.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo04.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo05.log
INFO: Setting nocache mode to /ora_dwh/log/dwh/redo06.log
INFO: Setting nocache mode to /ora_dwh/arch/dwh
INFO: Setting read mode to /ora_dwh/temp01/dwh/temp01.dbf
INFO: Setting read mode to /ora_dwh/temp02/dwh/temp_mrs02.dbf
INFO: Setting read mode to /ora_dwh/temp01/dwh/temp_mrs01.dbf
INFO: Setting read mode to /ora_dwh/data01/dwh/hratedata03.dbf
INFO: Setting read mode to /ora_dwh/data04/dwh/cvto_data02.dbf
INFO: Setting read mode to /ora_dwh/index04/dwh/bckidx05.dbf
INFO: Setting read mode to /ora_dwh/data02/dwh/cvto_data01.dbf
INFO: Setting read mode to /ora_dwh/index04/dwh/dwhidx04.dbf
INFO: Setting read mode to /ora_dwh/data03/dwh/svidata_tt01.dbf
INFO: Setting read mode to /ora_dwh/data03/dwh/rbwdata01.dbf
INFO: Setting read mode to /ora_dwh/data01/dwh/dwhdata07.dbf
INFO: Setting read mode to /ora_dwh/data02/dwh/e2dwhdata05.dbf
.
.
.

But if you display the attribute for one datafile (or tablespace) where the caching method is supposed to be read you should see nothing:

[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o list --datafile=/ora_dwh/data01/dwh/users01.dbf
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
FILENAME                        MODE      PINNED    CACHE_USED
--------                        ----      ------    ----------
/ora_dwh/data01/dwh/users01.dbf  nocache   no            0 KB

It is simply because we have created the cache are in noauto mode so you need to enable SmartIO for the file system first:

[root@server1 ~]# sfcache enable /ora_dwh/data01

Now display is correct and SmartIO read is activated:

[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o list --datafile=/ora_dwh/data01/dwh/users01.dbf
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
FILENAME                        MODE      PINNED    CACHE_USED
--------                        ----      ------    ----------
/ora_dwh/data01/dwh/users01.dbf  read      no            0 KB

Please note that even if I activate it for my redo log folder the redo log files will not be in read cache mode. As expected and set by sfcache app command issued by Oracle Linux account:

[root@server1 ~]# sfcache enable /ora_dwh/log
[root@server1 ~]# sfcache list -r /ora_dwh/log
/ora_dwh/log:
READ CACHE    WRITEBACK    MODE       PINNED  NAME
      0 KB         0 KB    nocache    no      /ora_dwh/log/dwh/redo01.log
      0 KB         0 KB    nocache    no      /ora_dwh/log/dwh/redo02.log
      0 KB         0 KB    nocache    no      /ora_dwh/log/dwh/redo03.log
      0 KB         0 KB    nocache    no      /ora_dwh/log/dwh/redo04.log
      0 KB         0 KB    nocache    no      /ora_dwh/log/dwh/redo05.log
      0 KB         0 KB    nocache    no      /ora_dwh/log/dwh/redo06.log
      0 KB         0 KB    read       no      /ora_dwh/log/dwh
      0 KB         0 KB    read       no      /ora_dwh/log

What Symantec claim is that you should pin top read datafiles in cache area. By doing this, free space is tablespaces will not be is SmartIO cache area. The datafiles ordered by IOPS can be easily found in an AWR report. Please note than pinning and loading (sfcache pin with -o load option or sfcache load directly) is not suggested as it would also put empty block in SmartIO cache area. As a reference what says official documentation:

  • The load operation preloads files into the cache before the I/O accesses the files. The files are already in the cache so that the I/Os return more quickly . By default, the files are loaded in the background. Use the -o sync operation to load the files synchronously , which specifies that the command does not return until all the files are loaded. The files that are loaded in this way are subject to the usual eviction criteria.
  • The pin operation prevents the files from being evicted from the cache. You can pin commonly used files so that SmartIO does not evict the files and later need to cache the files again. A pinned file is kept in the cache indefinitely , until it is deleted or explicitly unpinned. If you pin a file with the -o load option, the operation also caches the file contents synchronously . If you do not specify the -o load option, the file contents are cached based on I/O access.
  • The unpin operation removes files from the pinned state. The unpin operation does not cause the file to be immediately evicted. SmartIO considers the file for eviction in the same way as any other file, when space is required in the cache.

So I’m pinning in cache the datafile of the tablespace on which I’m planning to do testing:

[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o pin --datafile=/ora_dwh/data01/dwh/users01.dbf
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
INFO: Setting pin policy to /ora_dwh/data01/dwh/users01.dbf
[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o list --tablespace=USERS
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
FILENAME                        MODE      PINNED    CACHE_USED
--------                        ----      ------    ----------
/ora_dwh/data01/dwh/users01.dbf  read      yes           0 KB

Oracle testing

For testing I created in USERS tablespace the same test table I have used for Bind variables peeking and Adaptive Cursor Sharing (ACS) except that I created it with 50,000,000 rows and no index (total size is 3.5 GB). The plan is to do a count(*) on it to perform a Full Table Scan (FTS) and flush the buffer cache before each run to force Oracle to read blocks (so on disk or on flash cache).

Once the table is created I purge the cache area to start with a clean situation:

[root@server1 ~]# sfcache purge /ora_dwh/data01
[root@server1 ~]# sfcache stat -r /ora_dwh/data01
[root@server1 ~]# sfcache stat /ora_dwh/data01
       Cache Size:   339.9 GB
Cache Utilization:       8 KB ( 0.00 %)
 
Read Cache
Hit Ratio    Data Read    Data Written
 
/ora_dwh/data01:
   0.00 %         0 KB            0 KB
[root@server1 ~]# sfcache list /ora_dwh/data01
/ora_dwh/data01:
READ CACHE    WRITEBACK    MODE       PINNED  NAME
      0 KB         0 KB    read       no      /ora_dwh/data01/dwh
      0 KB         0 KB    read       no      /ora_dwh/data01
[root@server1 ~]# sfcache list /ora_dwh/data01/dwh/users01.dbf
/ora_dwh/data01/dwh/users01.dbf:
READ CACHE    WRITEBACK    MODE       PINNED
      0 KB         0 KB    read       yes

Remark:
To bypass Linux file systems caching I have mounted /ora_dwh/data01 using nodatainlog,mincache=direct,convosync=direct options.

Or with Oracle Linux account:

[oradwh@erver1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o list --tablespace=USERS
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
FILENAME                        MODE      PINNED    CACHE_USED
--------                        ----      ------    ----------
/ora_dwh/data01/dwh/users01.dbf  read      yes           0 KB

Then I created small below SQL script. The script flushes buffer cache (should not be needed for big table), generates a trace file and perform a FTS on my test table by doing a count(*):

SET autotrace traceonly EXPLAIN STATISTICS
ALTER SYSTEM flush buffer_cache;
ALTER SESSION SET events '10046 trace name context forever, level 12';
SELECT COUNT(*) FROM test1;

After first run, needed to load the SmartIO cache area, I got:

[root@server1 ~]# sfcache stat -l /ora_dwh/data01
       Cache Size:   339.9 GB
Cache Utilization:       8 KB ( 0.00 %)
 
Read Cache
Hit Ratio    Data Read    Data Written    Files Cached    Files Pinned    Data Pinned
 
/ora_dwh/data01:
   0.00 %         0 KB            0 KB               0               0           0 KB
[root@server1 ~]# sfcache list /ora_dwh/data01/dwh/users01.dbf
/ora_dwh/data01/dwh/users01.dbf:
READ CACHE    WRITEBACK    MODE       PINNED
  3.029 GB         0 KB    read       yes

Or with Oracle Linux account:

[oradwh@server1 trace]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o list --tablespace=USERS
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO
FILENAME                        MODE      PINNED    CACHE_USED
--------                        ----      ------    ----------
/ora_dwh/data01/dwh/users01.dbf  read      yes       3.029 GB

After second run I got (then third run would set the hit ratio to 66% and so on):

[root@server1 ~]# sfcache stat -l /ora_dwh/data01
       Cache Size:   339.9 GB
Cache Utilization:   3.029 GB ( 0.89 %)
 
Read Cache
Hit Ratio    Data Read    Data Written    Files Cached    Files Pinned    Data Pinned
 
/ora_dwh/data01:
  50.01 %      3.03 GB        3.029 GB              34               1       3.029 GB

I have executed tkprof Oracle tool over two trace files of my two runs and the difference is quite obvious:

First run in 67 seconds (disk access):

OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS
 
call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        5      0.00       0.00          0          0          0           0
Execute      5      0.00       0.00          0          3          3           3
Fetch        4      4.47      27.83     198484     298711          0          10
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total       14      4.47      27.83     198484     298714          3          13
 
Misses in library cache during parse: 1
 
Elapsed times include waiting on following events:
  Event waited on                             Times   Max. Wait  Total Waited
  ----------------------------------------   Waited  ----------  ------------
  SQL*Net message to client                       8        0.00          0.00
  SQL*Net message from client                     8       40.05         40.05
  Disk file operations I/O                        2        0.00          0.00
  db file sequential read                         7        0.01          0.02
  direct path read                             3123        0.07         25.24
  db file scattered read                          1        0.01          0.01
  log file sync                                   1        0.00          0.00

Second run in 51 seconds (cache area access but much less I/O latency, around 4 times faster to perform disk access):

OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS
 
call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        5      0.00       0.00          0          0          0           0
Execute      5      0.00       0.00          0          3          3           3
Fetch        4      3.22       7.01     198484     298711          0          10
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total       14      3.22       7.01     198484     298714          3          13
 
Misses in library cache during parse: 1
 
Elapsed times include waiting on following events:
  Event waited on                             Times   Max. Wait  Total Waited
  ----------------------------------------   Waited  ----------  ------------
  SQL*Net message to client                       8        0.00          0.00
  SQL*Net message from client                     8       44.31         44.32
  Disk file operations I/O                        3        0.00          0.00
  db file sequential read                         7        0.00          0.00
  direct path read                             3123        0.00          4.05
  db file scattered read                          1        0.00          0.00
  log file sync                                   1        0.00          0.00

How to remove

Simple commands to use to remove cache area:

[root@server1 /]# sfcache delete fusionio/lvol01
SFCache ERROR V-5-2-6240 Cache area is not offline
[root@server1 /]# sfcache offline fusionio/lvol01
[root@server1 /]# sfcache delete fusionio/lvol01

Issues encountered

Sfcache stat command not returning anything

[oradwh@server1 ~]$ sfcache app cachearea=fusionio/lvol01 oracle -S $ORACLE_SID -H $ORACLE_HOME -o stat
INFO: Oracle Instance dwh is runnig
INFO: Store DB details at /ora_dwh/sys/dwh/.CACHE_INFO

But still the stat command is empty:

Symantec support informed me that it is a known bug that will be correct by Storage Foundation 6.1.1. In 6.1.1 the command will still have issue if multiple databases are running on the same server. The complete correction of all bugs is expected to come with a later release…

I have dowloaded and applied the 6.1.1 patch using https://sort.veritas.com/patch/finder but it did not solved the issue (was expected to be honest)… Need to wait for 6.2 or future release…

SmartIO meta data cleaning

If when doing multiple test you got a strange error like:

UX:vxfs fscache: ERROR: V-3-28056: Failed to pin the file /ora_dwh/data01/dwh/users01.dbf to the cache: No such device
SFCache ERROR V-5-2-0 Unknown Error. Please check logs at /etc/vx/log/sfcache.log .

Means SmartIO is taking old configuration history and try to apply them to a newly created one, delete the .CACHE_INFO directory. This folder location is display each time you issue sfcache app command.

Sfcache purge command bug

While performing my tests I used the sfcache purge command to remove cache content of my /ora_dwh/data01 file system. The command further forbid anything to go in cache for the purged file system (!!). To recover the situation I tried many option and I have found that only one working is to offline/online the cache area:

[root@server1 ~]# sfcache offline fusionio/lvol01
[root@server1 ~]# sfcache online fusionio/lvol01

SmartIO licensing issue

For further testing I wanted to activate writeback mode but apparently I do now own a vlaid license. The strange thing is that it seems we own a full copy license:

[root@server1 ~]# mount -t vxfs -o smartiomode=writeback /dev/vx/dsk/fusionio/lvol01 /mnt/fioa
UX:vxfs mount.vxfs: ERROR: V-3-25255: vxfs mount:SmartIO: You don't have a license to run this program
[root@server1 ~]# vxdctl license
All features are available:
 Mirroring
 Root Mirroring
 Concatenation
 Disk-spanning
 Striping
 RAID-5
 RAID-5 Snapshot
 VxSmartSync
 DMP (multipath enabled)
 CDS
 Dynamic LUN Expansion
 Hardware assisted copy
 DMP Native Support
 SMARTIO_VMREAD

Or use vxlicrep. I have not investigated further this point as the feature has no immediate interest…

References

About Post Author

Yannick Jaquier

administrator

Find more about me on social media.

See author's posts

Share the knowledge!

Related Posts:

This entry was posted in Oracle and tagged . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>