IT WorldIT World

AWR baselines and performance comparison

Yannick — Wed, 08 May 2013 09:13:06 +0000

Preamble
Good performance
Bad performance
Performance improved
Conclusion
References

Preamble

While reviewing OCM exam objectives I saw that AWR baselines are in to-be-known list…. So decided to have a quick look and at same time understand this SYSTEM_MOVING_WINDOW default baseline that appears in Grid/Cloud Control.

As you can read in blogs of well known Oracle worldwide experts (few references below), increasing performance of an Oracle database is all about reducing DB time metric (that you can find in AWR reports). As a DBA I have quite often been in front of internal customers stating: performance of database were good, we did not change anything and now performance are awful, please correct problem as soon as possible…. I’m sure most of you have faced this in their DBA life…

So come in loop AWR baselines that are really great and can save your day !

For the test case I create the below table and fill it with 10,000 rows:

DROP TABLE test1;
 
CREATE TABLE test1(id NUMBER, descr VARCHAR(200)) TABLESPACE users;
 
DECLARE
i NUMBER;
nbrows NUMBER;
BEGIN
  i:=1;
  nbrows:=10000;
  LOOP
    EXIT WHEN i>nbrows;
    INSERT INTO test1 VALUES(i,TO_CHAR(TO_DATE(i,'j'), 'jsp'));
    i:=i+1;
  END LOOP;
  COMMIT;
END;
/
 
EXEC DBMS_STATS.GATHER_TABLE_STATS(ownname=>'yjaquier', tabname=>'test1', estimate_percent=>NULL, method_opt=>'FOR ALL INDEXED COLUMNS SIZE AUTO', degree=>DBMS_STATS.DEFAULT_DEGREE, CASCADE=>TRUE);

Modus operandi will be multiple select on this TEST1 table using a PL/SQL block, then I will multiply by 10 number of rows in this table and run same PL/SQL block and finally run an optimized (to be proven) PL/SQL block. This is symptomatic of a newly created application that starts with very few rows and get bigger when users fill-in information…

Testing has been done using Oracle Enterprise 11.2.0.3 running on a 16 cores box (not dedicated) with Red Hat Enterprise Linux Server release 5.5 (Tikanga). For testing purpose I have reduced AWR frequency to its minimum i.e. 10 minutes.

Good performance

The PL/SQL block I aim to use is the following. You will notice the hard parsing as no bind variable and as many select as number of rows in TEST1 table that has no indexes:

DECLARE /* Yannick */
  i NUMBER;
  sql_stmt VARCHAR2(200);
	max_id NUMBER;
BEGIN
  SELECT MAX(id) INTO max_id FROM yjaquier.test1;
  DBMS_RANDOM.SEED(SYSDATE);
  FOR i IN 1..max_id
  LOOP
    sql_stmt:='SELECT /* Yannick */ descr FROM yjaquier.test1 WHERE id=' || ROUND(DBMS_RANDOM.VALUE(1,max_id));
    EXECUTE IMMEDIATE sql_stmt;
  END LOOP;
END;
/

With 10,000 rows in the table the PL/SQL block is executed in 39.83 seconds, I take a baseline using below command:

SQL> EXEC DBMS_WORKLOAD_REPOSITORY.CREATE_BASELINE(13264,13265,'Good performance');
 
PL/SQL PROCEDURE successfully completed.
 
SQL> SELECT baseline_id,baseline_name,baseline_type,start_snap_id,end_snap_id
     FROM dba_hist_baseline;
 
BASELINE_ID BASELINE_NAME                                                    BASELINE_TYPE START_SNAP_ID END_SNAP_ID
----------- ---------------------------------------------------------------- ------------- ------------- -----------
          2 Good performance                                                 STATIC                13264       13265
          0 SYSTEM_MOVING_WINDOW                                             MOVING_WINDOW         13237       13265

The user created baselines are of type STATIC, versus the default SYSTEM_MOVING_WINDOW baselines that is of type MOVING. Windows moving size is in DBA_HIST_BASELINE.MOVING_WINDOW_SIZE column and alterable with DBMS_WORKLOAD_REPOSITORY.MODIFY_BASELINE_WINDOW_SIZE procedure. Even using baseline templates it is not possible to create a new moving window baseline.

The baseline template helps you, for example, to create a repeating baselines (every Monday between 8 AM and 8 PM for example), their type will be GENERATED. DBA_HIST_BASELINE_DETAILS, DBA_HIST_BASELINE_METADATA and DBA_HIST_BASELINE_TEMPLATE view display all details of baselines and templates.

The moving baseline aim is to compare how performance of your database evolve over time, in other words is your performance going in right or wrong direction ?

You can display baselines statistics and metric using the two below procedures of DBMS_WORKLOAD_REPOSITORY package:

SQL> SET lines 200 pages 1000
SQL> SELECT metric_name, metric_unit, average, minimum, maximum
     FROM TABLE(DBMS_WORKLOAD_REPOSITORY.SELECT_BASELINE_METRIC('Good performance'))
     ORDER BY metric_name;
 
METRIC_NAME                                                      METRIC_UNIT                                                         AVERAGE    MINIMUM    MAXIMUM
---------------------------------------------------------------- ---------------------------------------------------------------- ---------- ---------- ----------
Active Parallel Sessions                                         Sessions                                                                  0          0          0
Active Serial Sessions                                           Sessions                                                               1.35          0          3
Average Active Sessions                                          Active Sessions                                                  .185770022          0 .903106368
Average Synchronous Single-Block Read Latency                    Milliseconds                                                     2.65291323          0 8.22211738
Background CPU Usage Per Sec                                     CentiSeconds Per SECOND                                          .914520922          0 6.01504323
Background Checkpoints Per Sec                                   CHECK Points Per SECOND                                                   0          0          0
Background TIME Per Sec                                          Active Sessions                                                  .029696648          0 .217157599
Branch Node Splits Per Sec                                       Splits Per SECOND                                                         0          0          0
Branch Node Splits Per Txn                                       Splits Per Txn                                                            0          0          0
Buffer Cache Hit Ratio                                           % (LogRead - PhyRead)/LogRead                                    98.6432958          0        100
CPU Usage Per Sec                                                CentiSeconds Per SECOND                                          3.39302187          0 9.67427669
.
.
.
SQL> SELECT * FROM TABLE(DBMS_WORKLOAD_REPOSITORY.SELECT_BASELINE_DETAILS(2));
 
      DBID BASELINE_ID INSTANCE_NUMBER START_SNAP_ID START_SNAP_TIME                                                             END_SNAP_ID
---------- ----------- --------------- ------------- --------------------------------------------------------------------------- -----------
END_SNAP_TIME                                                               SHU ERROR_COUNT PCT_TOTAL_TIME
--------------------------------------------------------------------------- --- ----------- --------------
3673847771           2               1         13264 07-MAY-13 10.10.37.940 AM                                                         13265
07-MAY-13 10.20.38.541 AM                                                   NO            0            100

Bad performance

Then I now fill (using same method) the TEST1 table with 1,000,000 rows and I execute the exact same PL/SQL block that now elapses in 8 minutes and 40.70 seconds. I create a new baseline that I call ‘Bad performance’:

SQL> EXEC DBMS_WORKLOAD_REPOSITORY.CREATE_BASELINE(13266,13267,'Bad performance');
 
PL/SQL PROCEDURE successfully completed.
 
SQL> SELECT baseline_id,baseline_name,baseline_type,start_snap_id,end_snap_id FROM dba_hist_baseline;
 
BASELINE_ID BASELINE_NAME                                                    BASELINE_TYPE START_SNAP_ID END_SNAP_ID
----------- ---------------------------------------------------------------- ------------- ------------- -----------
          2 Good performance                                                 STATIC                13264       13265
          3 Bad performance                                                  STATIC                13266       13267
          0 SYSTEM_MOVING_WINDOW                                             MOVING_WINDOW         13237       13267

To compare the two baselines you have multiple option DBMS_WORKLOAD_REPOSITORY.AWR_DIFF_REPORT_HTML, DBMS_WORKLOAD_REPOSITORY.AWR_DIFF_REPORT_TEXT or awrddrpt.sql script located in $ORACLE_HOME/rdbms/admin directory. I prefer to use awrddrpt.sql script and display result in my web browser:

Good versus bad

So what do we see (non exhaustively) ?:

DB time moved from 2.2 (min) to 9.7 (min) over a 10 minutes period.
Top 5 wait events move from I/O waits (db file sequential read & direct path read) to CPU at more than 91%.
parse time elapsed, DB CPU and hard parse elapsed time have increased a lot.
sql_id 81uvdbt46vt5p (my PL/SQL block, thanks to /* Yannick */ comment to retrieve it) is the one that has most been impacted between the two periods (% of DB time, elapsed time, CPU time, …).

So clearly something has changed and you can peacefully forward to applicative team and move to something else… But as a good DBA you may, as well, have a look…

Performance improved

To try to optimize the PL/SQL code I create an index on id column:

CREATE INDEX test1_idx_id ON test1(id) TABLESPACE users;
 
EXEC DBMS_STATS.GATHER_TABLE_STATS(ownname=>'yjaquier', tabname=>'test1', estimate_percent=>NULL, method_opt=>'FOR ALL INDEXED COLUMNS SIZE AUTO', degree=>DBMS_STATS.DEFAULT_DEGREE, CASCADE=>TRUE);

And remove the hard coded value by using a bind variable:

DECLARE /* Yannick */
  i NUMBER;
  sql_stmt VARCHAR2(200);
	max_id NUMBER;
BEGIN
  SELECT MAX(id) INTO max_id FROM yjaquier.test1;
  DBMS_RANDOM.SEED(SYSDATE);
  sql_stmt:='SELECT /* Yannick */ descr FROM yjaquier.test1 WHERE id=:id';
  FOR i IN 1..max_id
  LOOP
    EXECUTE IMMEDIATE sql_stmt USING ROUND(DBMS_RANDOM.VALUE(1,max_id));
  END LOOP;
END;
/

The PL/SQL block is now executed in 24.50 seconds. I create a new baseline called ‘Performance improved’:

SQL> EXEC DBMS_WORKLOAD_REPOSITORY.CREATE_BASELINE(13268,13269,'Performance improved');
 
PL/SQL PROCEDURE successfully completed.
 
SQL> SELECT baseline_id,baseline_name,baseline_type,start_snap_id,end_snap_id FROM dba_hist_baseline;
 
BASELINE_ID BASELINE_NAME                                                    BASELINE_TYPE START_SNAP_ID END_SNAP_ID
----------- ---------------------------------------------------------------- ------------- ------------- -----------
          2 Good performance                                                 STATIC                13264       13265
          3 Bad performance                                                  STATIC                13266       13267
          4 Performance improved                                             STATIC                13268       13269
          0 SYSTEM_MOVING_WINDOW                                             MOVING_WINDOW         13238       13269

The comparison report is the following:

Bad versus improved

So what do we see (non exhaustively) ?:

DB time moved from 9.7 (min) to 2.9 (min) over a 10 minutes period.
Top 5 wait events move from CPU at more than 91% to I/O waits (db file sequential read & direct path read).
parse time elapsed, DB CPU and hard parse elapsed time have decreased a lot.
sql_id 81uvdbt46vt5p load (% of DB time, elapsed time, CPU time, …) has decreased because not present in third run (performance improved) you would say… But we can retrieve improved PL/SQL sql_id, again, using /* Yannick */ comment and now see it is 28ja91pnth38j. By comparing we can see that main performance indicator (% of DB time, elapsed time, CPU time, …) have decreased.

This sql_id change also demonstrate, was it needed to re-demonstrate it ?, that checking performance improvement, or not, is not always a piece of cake…

Conclusion

All can also be seen graphically in Cloud Control at a very high level but comparing period and keeping baselines is best way to be able to go in details:

AWR baseline 1

As a DBA I would always keep a baseline when users are happy with performance. This would provide invaluable piece of information when things start to go wrong and that you need to prove from where are coming problems…

For the fun I have generated the comparison report between good performance and performance improved baselines:

Good versus improved

Without entering too much in details we can see that even with a 10 times more figures the performance globally is almost flat, I would love achieve it for all our environments…

References

MySQL 5.6 and MySQL 5.5 performance comparison

Yannick — Fri, 12 Apr 2013 14:34:46 +0000

Preamble
Configuration
Results and conclusion
References

Preamble

I have seen many blog post over Internet questioning the performances of MySQL 5.6. There is even one on world know mysqlperformance blog from Peter Zaitsev called Is MySQL 5.6 slower than MySQL 5.5?. I had caution look to all comments and I have to say that results from various person that replied were diverging a lot…

On top of this, internally I have more and more pressure to make MySQL 5.6 standard release for MySQL. So I have finally decided to conduct very basic test using a tool I have already work on called Sysbench (0.4.12).

For this testing I have used Oracle Linux Server release 6.4 64 bits (updated from Oracle public yum at the time of writing of this post) under VirtualBox 4.2.10 r84104. I have allocated 2 cores of my physical processor (Intel Xeon E5507 at 2.27 GHz) and 4GB of RAM (I have 8 GB physical RAM). I downloaded MySQL Community Server generic Linux 64 bits edition (tar files) for MySQL 5.5.30 and MySQL 5.6.10 (latest releases available at the time of writing of this post).

Configuration

To have two MySQL installations on same server I had to refine a bit my way of installing MySQL, so chosen the below naming convention:

/mysql/backup/x.y.z for backup files if any
/mysql/data01/x.y.z for database files (letting capability to create data02 to spread the I/Os)
/mysql/dump/x.y.z for log files
/mysql/logs/x.y.z for mysql binary logs
/mysql/software/x.y.z for binaries

Where x.y.z if MySQL exact release, under /mysql/software/x.y.z I create a conf directory to store my.cnf, pid and lock files.

I add below aliases and PATH to mysql OS account:

PATH=$PATH:$HOME/bin:/mysql/sysbench/sysbench-0.4.12/sysbench
 
export PATH
 
# MySQL
alias start_mysql55='cd /mysql/software/5.5.30/; ./bin/mysqld_safe --defaults-file=/mysql/software/5.5.30/conf/my.cnf &'
alias stop_mysql55="kill \`ps -ef | grep /mysql/software/5.5.30/bin/mysqld | grep -v grep | awk '{print \$2}'\`"
 
alias start_mysql56='cd /mysql/software/5.6.10/; ./bin/mysqld_safe --defaults-file=/mysql/software/5.6.10/conf/my.cnf &'
alias stop_mysql56="kill \`ps -ef | grep /mysql/software/5.6.10/bin/mysqld | grep -v grep | awk '{print \$2}'\`"
 
alias mysql55='/mysql/software/5.5.30/bin/mysql --defaults-file=/mysql/software/5.5.30/conf/my.cnf --user=root'
alias mysql56='/mysql/software/5.6.10/bin/mysql --defaults-file=/mysql/software/5.6.10/conf/my.cnf --user=root'

Then configuring Sysbench for MySQL 5.5 or 5.6 is as simple as (please refer to my other post for issues). I have chosen to recompile it using libraries of exact release I’m currently testing:

./configure --with-mysql-includes=/mysql/software/5.5.30/include --with-mysql-libs=/mysql/software/5.5.30/lib/
make
export LD_LIBRARY_PATH=/mysql/software/5.5.30/lib

./configure --with-mysql-includes=/mysql/software/5.6.10/include --with-mysql-libs=/mysql/software/5.6.10/lib/
make
export LD_LIBRARY_PATH=/mysql/software/5.6.10/lib

On both MySQL instances I created a sysbench account (test default database will be reused):

GRANT ALL PRIVILEGES ON *.* TO 'sysbench'@'localhost' IDENTIFIED BY 'sysbench' WITH GRANT OPTION;
FLUSH PRIVILEGES;

To create my my.cnf file I started from ./support-files/my-small.cnf file adding few specific parameters, no memory tuning, just straightforward. The one for MySQL 5.5.30:

[client]
port = 3316
socket = /mysql/software/5.5.30/conf/mysql.sock
 
[mysqld]
port = 3316
socket = /mysql/software/5.5.30/conf/mysql.sock
basedir = /mysql/software/5.5.30
datadir = /mysql/data01/5.5.30
pid-file = /mysql/software/5.5.30/conf/5.5.30.pid
log-bin = /mysql/logs/5.5.30/mysql-bin.log
log_error = /mysql/dump/5.5.30/server1.err
slow_query_log_file = /mysql/dump/5.5.30/slow.log
skip-external-locking
key_buffer_size = 16K
max_allowed_packet = 1M
table_open_cache = 4
sort_buffer_size = 64K
read_buffer_size = 256K
read_rnd_buffer_size = 256K
net_buffer_length = 2K
thread_stack = 128K
server-id	= 1
performance_schema = off
 
[mysqldump]
quick
max_allowed_packet = 16M
 
[mysql]
no-auto-rehash
 
[myisamchk]
key_buffer_size = 8M
sort_buffer_size = 8M
 
[mysqlhotcopy]
interactive-timeout

And the one for MySQL 5.6.10:

[client]
port = 3326
socket = /mysql/software/5.6.10/conf/mysql.sock
 
[mysqld]
port = 3326
socket = /mysql/software/5.6.10/conf/mysql.sock
basedir = /mysql/software/5.6.10
datadir = /mysql/data01/5.6.10
pid-file = /mysql/software/5.6.10/conf/5.6.10.pid
log-bin = /mysql/logs/5.6.10/mysql-bin.log
log_error = /mysql/dump/5.6.10/server1.err
slow_query_log_file = /mysql/dump/5.6.10/slow.log
skip-external-locking
key_buffer_size = 16K
max_allowed_packet = 1M
table_open_cache = 4
sort_buffer_size = 64K
read_buffer_size = 256K
read_rnd_buffer_size = 256K
net_buffer_length = 2K
thread_stack = 128K
server-id = 1
innodb_file_per_table = off
performance_schema = off
 
[mysqldump]
quick
max_allowed_packet = 16M
 
[mysql]
no-auto-rehash
 
[myisamchk]
key_buffer_size = 8M
sort_buffer_size = 8M
 
[mysqlhotcopy]
interactive-timeout

Finally sysbench commands are:

sysbench --test=oltp --mysql-socket=/mysql/software/5.5.30/conf/mysql.sock --mysql-port=3316 --mysql-user=sysbench --mysql-password=sysbench
--mysql-db=test --oltp-table-size=10000000 --num-threads=1 --oltp-read-only=on --init-rng=on --max-requests=0 --max-time=300 [prepare | run]
 
sysbench --test=oltp --mysql-socket=/mysql/software/5.6.10/conf/mysql.sock --mysql-port=3326 --mysql-user=sysbench --mysql-password=sysbench
--mysql-db=test --oltp-table-size=10000000 --num-threads=1 --oltp-read-only=on --init-rng=on --max-requests=0 --max-time=300 [prepare | run]

Remark:
So creating a table with 10 million rows (2400 MB around), initializing random number before each test and running it for 5 minutes with different number of threads (from 1 to 64). Reading multiple posts around and having double checked in MySQL 5.6 official documentation it appears that performance schema is activated by default starting with MySQL 5.6.6 (off by default for previous releases and in all 5.5.x releases) so the performance_schema = off in my.cnf. I finally tested performance activating it as I personally flag this feature with high interest…
Even if it does not change too much as sysbench benchmark is made of one table I also set innodb_file_per_table to off to avoid having one InnoDB file per table and club all of them in big ibdata1 datafile.

Results and conclusion

Number of threads	MySQL 5.5.30 Number of TPS	MySQL 5.6.10 Performance Schema=off, number of TPS	MySQL 5.6.10 Performance Schema=on, number of TPS
1	143	132	124
8	459	463	388
16	463	463	388
32	432	453	383
64	417	441	370

With an Excel chart:

MySQL 5.6 vs MySQL 5.5 performance

From my small test there is not much difference from performance point of view between MySQL 5.5 and MySQL 5.6… if you deactivate Performance Schema. Even if it adds lots of value it may impact your workload up to 20%… With pretty high number of threads (>16) I even see MySQL 5.6 in front of MySQL 5.5.

References

Is MySQL 5.6 slower than MySQL 5.5?

Media recovery with NOLOGGING tables

Yannick — Fri, 15 Mar 2013 09:00:02 +0000

Preamble
Nologging and CTAS
Nologging and insert command
Tablespaces logging attributes
RMAN and nologging operations
References

Preamble

There’s been a long debate with teammates about usage of NOLOGGING in our Oracle databases. Some application manager have discovered (and experienced) that some objects may be unrecoverable when using this clause on the opposite few DBAs wanted to audit all their databases to simply remove it. The performance gain using it well is obvious but to really understand what’s really doing this NOLOGGING option and the (bad) consequences of its usage I had to come back to official documentation. The default logging mode is the one of the tablespace (and the one of the tablespace is LOGGING).

In this blog post I will focus on what I rate the standard DML/DDL commands we execute the most i.e. INSERT, Create Table As Select (CTAS) and CREATE INDEX.

Few evidences:

Standard INSERT and UPDATE are always logged whatever LOGGING/NOLOGGING option chosen.
Direct-path INSERT (APPEND or APPEND_VALUES hints) supported by NOLOGGING option.
Direct Loader (SQL*Loader) supported by NOLOGGING option.

Testing has been done on Oracle Linux Server release 6.3 using Oracle 11.2.0.3. My database is in archivelog mode as I’m doing datafile recovery…

As clearly written by Tom Kyte don’t be shy when creating your test table. In clear if your NOLOGGING test table is too small then Oracle will generate redo for it and bypass the NOLOGGING option… The post also contains a very interesting table comparing when redo are generated with database in archivelog/noarchivelog mode and insert method used.

To perform this test I have created an USERS tablespace with one datafile:

SQL> SELECT FILE_ID,FILE_NAME FROM dba_data_files WHERE  TABLESPACE_NAME='USERS';
 
   FILE_ID FILE_NAME
---------- --------------------------------------------------
         4 /u01/app/oracle/oradata/orcl/users01.dbf

When the tablespace is still empty I have taken a backup using RMAN and:

RMAN> backup datafile 4;
 
Starting backup at 15-FEB-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting compressed full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00004 name=/u01/app/oracle/oradata/orcl/users01.dbf
channel ORA_DISK_1: starting piece 1 at 15-FEB-13
channel ORA_DISK_1: finished piece 1 at 15-FEB-13
piece handle=/u01/app/oracle/fast_recovery_area/ORCL/backupset/2013_02_15/o1_mf_nnndf_TAG20130215T122853_8kw6zotn_.bkp tag=TAG20130215T122853 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 15-FEB-13
 
Starting Control File Autobackup at 15-FEB-13
piece handle=/u01/app/oracle/fast_recovery_area/ORCL/autobackup/2013_02_15/o1_mf_n_807452934_8kw6zqj6_.bkp comment=NONE
Finished Control File Autobackup at 15-FEB-13

To simulate the crash, shutdown the database, delete at OS level the datafile located at /u01/app/oracle/oradata/orcl/users01.dbf, startup the database that will complains for:

ORA-01157: cannot identify/LOCK data FILE 4 - see DBWR trace FILE
ORA-01110: data FILE 4: '/u01/app/oracle/oradata/orcl/users01.dbf'

You can anyway start it with:

SQL> ALTER DATABASE datafile 4 OFFLINE;
 
DATABASE altered.
 
SQL> ALTER DATABASE OPEN;
 
DATABASE altered.

Then recover using Data Recovery Advisor (DRA) is as simple as:

list failure [detail]
advise failure
repair failure

At the end, as you guess, the recover script generated by DRA is:

   # restore and recover datafile
   restore datafile 4;
   recover datafile 4;

You will then just need to issue under RMAN (or SQL by the way):

RMAN> sql 'alter database datafile 4 online';

Or else you get:

ORA-00376: FILE 4 cannot be read AT this TIME
ORA-01110: data FILE 4: '/u01/app/oracle/oradata/orcl/users01.dbf'

Nologging and CTAS

For the test case I create a table and an index both in nologging. I measure in parallel the redo size generated:

SQL> SET lines 200 pages 200
SQL> SELECT name, a.VALUE
FROM v$sesstat a, v$sysstat b
WHERE b.statistic# = a.statistic#
AND b.name = 'redo size'
AND a.sid = (SELECT sid FROM v$mystat WHERE ROWNUM = 1);
 
NAME                                                                  VALUE
---------------------------------------------------------------- ----------
redo SIZE                                                               664
 
SQL> CREATE TABLE test1 nologging TABLESPACE users AS SELECT * FROM dba_objects;
 
TABLE created.
 
SQL> CREATE INDEX test1_idx_OWNER ON test1(owner) nologging TABLESPACE users;
 
INDEX created.
 
SQL> EXEC DBMS_STATS.GATHER_TABLE_STATS(ownname =>'YJAQUIER', tabname => 'TEST1');
 
PL/SQL PROCEDURE successfully completed.
 
SQL> SELECT name, a.VALUE
FROM v$sesstat a, v$sysstat b
WHERE b.statistic# = a.statistic#
AND b.name = 'redo size'
AND a.sid = (SELECT sid FROM v$mystat WHERE ROWNUM = 1);
 
NAME                                                                  VALUE
---------------------------------------------------------------- ----------
redo SIZE                                                            287668

Doing the same SQL block in LOGGING mode would generate instead 10,808,868 bytes of redo (37 times more !!).

Let’s simulate the crash as we have seen in preamble section and select on our test table. One important thing to note here is that table and index will still be valid after the recover even if by selecting them you will generate an error:

SQL> SELECT * FROM yjaquier.test1 WHERE owner='YJAQUIER';
SELECT * FROM yjaquier.test1 WHERE owner='YJAQUIER'
                       *
ERROR AT line 1:
ORA-01578: ORACLE data block corrupted (FILE # 4, block # 1595)
ORA-01110: data FILE 4: '/u01/app/oracle/oradata/orcl/users01.dbf'
ORA-26040: Data block was loaded USING the NOLOGGING OPTION
 
SQL> SELECT SEGMENT_TYPE, OWNER || '.' || SEGMENT_NAME
FROM DBA_EXTENTS
WHERE FILE_ID = 4 AND 1595 BETWEEN BLOCK_ID AND BLOCK_ID+BLOCKS-1;
 
SEGMENT_TYPE       OWNER||'.'||SEGMENT_NAME
------------------ ----------------------------------------------------------------------------------------------------------------
INDEX              YJAQUIER.TEST1_IDX_OWNER
 
SQL> SELECT * FROM V$DATABASE_BLOCK_CORRUPTION;
 
     FILE#     BLOCK#     BLOCKS CORRUPTION_CHANGE# CORRUPTIO
---------- ---------- ---------- ------------------ ---------
         4        168          8            2382810 NOLOGGING
         4        177         15            2382837 NOLOGGING
 
SQL> SELECT table_name,status,logging FROM dba_tables WHERE table_name='TEST1';
 
TABLE_NAME                     STATUS   LOG
------------------------------ -------- ---
TEST1                          VALID    NO
 
SQL> SELECT INDEX_NAME,status,LOGGING FROM dba_indexes WHERE table_name='TEST1';
 
INDEX_NAME                     STATUS   LOG
------------------------------ -------- ---
TEST1_IDX_OWNER                VALID    NO

As only the index seems corrupted let’s drop it:

SQL> DROP INDEX TEST1_IDX_OWNER;
 
INDEX dropped.
 
SQL> SELECT * FROM test1 WHERE owner='YJAQUIER';
SELECT * FROM test1 WHERE owner='YJAQUIER'
              *
ERROR AT line 1:
ORA-01578: ORACLE data block corrupted (FILE # 4, block # 171)
ORA-01110: data FILE 4: '/u01/app/oracle/oradata/orcl/users01.dbf'
ORA-26040: Data block was loaded USING the NOLOGGING OPTION
 
SQL> SELECT SEGMENT_TYPE, OWNER || '.' || SEGMENT_NAME
FROM DBA_EXTENTS
WHERE FILE_ID = 4 AND 171 BETWEEN BLOCK_ID AND BLOCK_ID+BLOCKS-1;
 
SEGMENT_TYPE       OWNER||'.'||SEGMENT_NAME
------------------ ----------------------------------------------------------------------------------------------------------------
TABLE              YJAQUIER.TEST1

Worst situation here the index and the table are corrupted. If not a big deal for the index it is an issue for the table if you are not in a DataWareHouse environment.

Remark
If you create the table in LOGGING and index in NOLOGGING then the index is obviously still corrupted but you can still access to table figures that are not corrupted.

Nologging and insert command

For INSERT command the test table I’m gonna use is the following:

DROP TABLE test1;
 
CREATE TABLE test1(id NUMBER, descr VARCHAR2(50)) NOLOGGING TABLESPACE users;

To see that only direct-path INSERT are concerned by NOLOGGING mode, if you load this test table with below block I have already used in my Adaptive Cursor Sharing (ACS) post:

DECLARE
  i NUMBER;
  nbrows NUMBER;
BEGIN
  i:=1;
  nbrows:=50000;
  LOOP
    EXIT WHEN i>nbrows;
    IF (i=1) THEN
      INSERT INTO test1 VALUES(1,RPAD('A',49,'A'));
    ELSE
      INSERT INTO test1 VALUES(nbrows,RPAD('A',49,'A'));
    END IF;
    i:=i+1;
  END LOOP;
  COMMIT;
END;
/

If we apply same opus operandi:

SQL> CREATE INDEX test1_idx_id ON test1(id) NOLOGGING TABLESPACE users;
 
INDEX created.
 
SQL> EXEC DBMS_STATS.GATHER_TABLE_STATS(ownname =>'YJAQUIER', tabname => 'TEST1');
 
PL/SQL PROCEDURE successfully completed.
 
SQL> SELECT name, a.VALUE
FROM v$sesstat a, v$sysstat b
WHERE b.statistic# = a.statistic#
AND b.name = 'redo size'
AND a.sid = (SELECT sid FROM v$mystat WHERE ROWNUM = 1);
 
NAME                                                                  VALUE
---------------------------------------------------------------- ----------
redo SIZE                                                          15568652

This is proven by successful restore of the table (index still corrupted obviously)… To force index usage either use an hint (INDEX) or use id = 1 in your select statement on test1.

To do direct-path insert I replace my insert statement by:

INSERT /*+ append */ INTO test1 VALUES(nbrows,RPAD('A',49,'A'));

But it didn’t work and then found in official documentation:

The APPEND hint is only supported with the subquery syntax of the INSERT statement, not the VALUES clause. If you specify the APPEND hint with the VALUES clause, it is ignored and conventional insert will be used. To use direct-path INSERT with the VALUES clause, refer to “APPEND_VALUES Hint”.

When replacing append hint by append_values hint and committing at the end of the block I got:

ORA-12838: cannot read/modify an object after modifying it in parallel

Again it is in official documentation:

You cannot query or modify data inserted using direct-path INSERT immediately after the insert is complete. If you attempt to do so, an ORA-12838 error is generated. You must first issue a COMMIT statement before attempting to read or modify the newly-inserted data.

So had to use below non-optimal block:

DECLARE
  i NUMBER;
  nbrows NUMBER;
BEGIN
  i:=1;
  nbrows:=50000;
  LOOP
    EXIT WHEN i>nbrows;
    IF (i=1) THEN
      INSERT /*+ append_values */ INTO test1 VALUES(1,RPAD('A',49,'A'));
      COMMIT;
    ELSE
      INSERT /*+ append_values */ INTO test1 VALUES(nbrows,RPAD('A',49,'A'));
      COMMIT;
    END IF;
    i:=i+1;
  END LOOP;
END;
/

Execution time was much longer (19 minutes instead of 13 seconds) and number of extents moved from 19 (4 MB) to 121 (400 MB) in my local autoallocate tablespace.

If we perform same commands and measure redo size:

SQL> CREATE INDEX test1_idx_id ON test1(id) NOLOGGING TABLESPACE users;
 
INDEX created.
 
SQL> EXEC DBMS_STATS.GATHER_TABLE_STATS(ownname =>'YJAQUIER', tabname => 'TEST1');
 
PL/SQL PROCEDURE successfully completed.
 
SQL> SELECT name, a.VALUE
FROM v$sesstat a, v$sysstat b
WHERE b.statistic# = a.statistic#
AND b.name = 'redo size'
AND a.sid = (SELECT sid FROM v$mystat WHERE ROWNUM = 1);
 
NAME                                                                  VALUE
---------------------------------------------------------------- ----------
redo SIZE                                                          83720616

Redo size generated is 5 times greater !! Which is exactly the opposite of what we want. I would say it is worst of both world as we have more redo and table and index are not recoverable…

There is apparently a bug with this append_values 11gR2 new hint (see references) with no clear solution so far… Also few references on Internet on this but in Expert PL/SQL Practices: for Oracle Developers and DBAs book they advise to use FORALL PL/SQL command when working with append_values hint so created the alternative PL/SQL block:

DECLARE
  i NUMBER;
  nbrows NUMBER;
  TYPE type1 IS TABLE OF VARCHAR2(50);
  array1 type1 := type1();
BEGIN
  nbrows:=50000;
 
  FOR i IN 1..nbrows LOOP
    array1.extend;
    array1(i):=RPAD('A',49,'A');
  END LOOP;
 
  INSERT /*+ append_values */ INTO test1 VALUES(1,array1(1));
  COMMIT;
 
  FORALL i IN 2..array1.COUNT
    INSERT /*+ append_values */ INTO test1 VALUES (nbrows,array1(i));
 
  COMMIT;
END;
/

Execution time moved to 4 seconds and redo generated is equal to 384 KB so clearly as expected (smaller execution time and less redo generated). Table and index are still corrupted after media recovery but this was planned…

Tablespaces logging attributes

There are two logging attribute for a tablespace, the default logging mode for newly created object (if you change it, it does not affect already created objects) and the force logging mode (can also be specified at database level, that has preference over the one set at tablespace level) for generation of redo information even if users specify NOLOGGING in supported NOLOGGING operations:

SQL> SELECT TABLESPACE_NAME,LOGGING,FORCE_LOGGING FROM dba_tablespaces WHERE TABLESPACE_NAME='USERS';
 
TABLESPACE_NAME                LOGGING   FOR
------------------------------ --------- ---
USERS                          LOGGING   NO

This could be used if you want to be 100% sure that you media recovery will not let objects with corrupted blocks…

RMAN and nologging operations

RMAN has a very interesting reporting function for nologging operations called report unrecoverable. To test it I empty my users tablespace and backup it:

RMAN> backup datafile 4;
 
Starting backup at 15-FEB-13
using channel ORA_DISK_1
channel ORA_DISK_1: starting compressed full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00004 name=/u01/app/oracle/oradata/orcl/users01.dbf
channel ORA_DISK_1: starting piece 1 at 15-FEB-13
channel ORA_DISK_1: finished piece 1 at 15-FEB-13
piece handle=/u01/app/oracle/fast_recovery_area/ORCL/backupset/2013_02_15/o1_mf_nnndf_TAG20130215T175913_8kwtc1r9_.bkp tag=TAG20130215T175913 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 15-FEB-13
 
Starting Control File Autobackup at 15-FEB-13
piece handle=/u01/app/oracle/fast_recovery_area/ORCL/autobackup/2013_02_15/o1_mf_n_807472754_8kwtc3bo_.bkp comment=NONE
Finished Control File Autobackup at 15-FEB-13
 
RMAN> report unrecoverable;
 
Report of files that need backup due to unrecoverable operations
File Type of Backup Required Name
---- ----------------------- -----------------------------------

If I create a table with logging attribute:

SQL> CREATE TABLE test1 logging TABLESPACE users AS SELECT * FROM dba_objects;
 
TABLE created.

Still no issue:

RMAN> report unrecoverable;
 
Report of files that need backup due to unrecoverable operations
File Type of Backup Required Name
---- ----------------------- -----------------------------------

If I now create a table with nologging attribute:

SQL> CREATE TABLE test2 nologging TABLESPACE users AS SELECT * FROM dba_objects;
 
TABLE created.

Then RMAN is able to track it:

RMAN> report unrecoverable;
 
using target database control file instead of recovery catalog
Report of files that need backup due to unrecoverable operations
File Type of Backup Required Name
---- ----------------------- -----------------------------------
4    full or incremental     /u01/app/oracle/oradata/orcl/users01.dbf

At this stage you know that you must perform another backup of users01.dbf datafile to be 100% sure your media recovery will not let objects in an unrecoverable state…

References

logging_clause
Improving INSERT Performance with Direct-Path INSERT
APPEND Hint (Direct-Path) Insert with Values Causes Excessive Space Usage on 11G [ID 842374.1]

Tmpfs vs Ramfs vs (Transparent) Huge Pages

Yannick — Thu, 28 Feb 2013 07:00:01 +0000

Preamble
Tmpfs
Ramfs
Huge Pages
Transparent Huge Pages (THP)
References

Preamble

What are the possible configuration for Automatic Shared Memory Management (ASMM) and Automatic Memory Management (AMM) with ramfs, tmpfs and (Transparent) Huge Pages. Some configurations are simply not supported while for some you have to choose between flexibility and performance…

Testing has been done using Oracle Linux Server release 6.3 (Kernel 2.6.39-300.26.1.el6uek.x86_64) and Oracle 11.2.0.3.

Tmpfs

Previously known as shmfs, tmpfs is created by default when installing most Linux distribution:

[root@server1 ~]# grep shm /etc/fstab
tmpfs                           /dev/shm                                tmpfs   defaults        0 0

By default 50% of server physical memory will be *used*, in fact not really used because pages can be swap out. This is pseudo File System to configure if you want to use Automatic Memory Management (AMM) feature i.e. using memory_target and memory_max_target initialization parameters.

This implementation is quite common but not at all recommended, even by Oracle, when having a server with plenty of memory. In such scenario managing memory page table can be quite time consuming:

[root@server1 ~]# grep -i pagetables /proc/meminfo
PageTables:        12500 kB

Ramfs

The main differences, well sum up by The Geek Stuff (see references), are:

Experimentation	Tmpfs	Ramfs
Fill maximum space and continue writing	Will display error	Will continue writing
Fixed Size	Yes	No
Uses Swap	Yes	No
Volatile Storage	Yes	Yes

So on the paper it looks great:

Does not use swap !
Can fill beyond limit but not an issue if you are careful, in any case we are talking of Oracle memory allocation so you control it !

To configure it use the following:

[root@server1 ~]# grep shm /etc/fstab
ramfs                   /dev/shm                ramfs   defaults        0 0
[root@server1 ~]# df -P /dev/shm
Filesystem         1024-blocks      Used Available Capacity Mounted on
ramfs                   510352       112    510240       1% /dev/shm

The database is starting well and use /dev/shm pseudo-File System correctly:

[root@server1 orcl]# df -P /dev/shm
Filesystem         1024-blocks      Used Available Capacity Mounted on
ramfs                   510352    192540    317812      38% /dev/shm

Unfortunately as it is explained in MOS note 749851.1 it is not supported:

Please also note that ramfs (instead of tmpfs mount over /dev/shm) is not supported for AMM at all. With AMM the Oracle database needs to grow and reduce the size of SGA dynamically. This is not possible with ramfs where it possible and supported with tmpfs (which is the default for the OS installation).

Huge Pages

As we have seen in this blog post Huge Pages is an answer to swapping. But unfortunately as clearly mentioned in documentation available at (install kernel-doc package):

[root@server1 ~]# ll /usr/share/doc/kernel-doc-2.6.39/Documentation/vm/hugetlbpage.txt
-r--r--r-- 1 root root 15156 Jan  4 03:06 /usr/share/doc/kernel-doc-2.6.39/Documentation/vm/hugetlbpage.txt

It is only for shared memory (so SGA and not PGA):

Users can use the huge page support in Linux kernel by either using the mmap system call or standard SYSV shared memory system calls (shmget, shmat).

The documentation also mentioned its usage using mmap method and so allocating Huge Pages in a filesystem. Why not using it with AMM ? To test it I mounted a filesystem of hugetlbfs type giving privileges to my oracle Unix account.

Kernel settings:

# Huge Pages
vm.nr_hugepages = 101
vm.hugetlb_shm_group = 500

Oracle settings:

*.memory_target=200m
*.pre_page_sga=TRUE
*.use_large_pages='ONLY'

To mount the filesystem:

[oracle@server1 ~]$ id
uid=501(oracle) gid=500(dba) groups=500(dba)
[root@server1 ~]# mount -t hugetlbfs -o uid=501,gid=500,size=202M none /dev/shm
[root@server1 ~]# df -P /dev/shm
Filesystem         1024-blocks      Used Available Capacity Mounted on
none                    206848         0    206848       0% /dev/shm

But unfortunately Oracle is not starting:>/p>

SQL> startup
ORA-27102: out of memory
Linux-x86_64 Error: 12: Cannot allocate memory
Additional information: 1
Additional information: 1081350
Additional information: 14

I tried to manually create and fill a dummy file, but same error:

[oracle@server1 ~]$ cd /dev/shm
[oracle@server1 shm]$ touch test1
[oracle@server1 shm]$ ll
total 0
-rw-r--r-- 1 oracle dba 0 Feb  5 12:09 test1
[oracle@server1 shm]$ echo test1 > test1
-bash: echo: WRITE error: Invalid argument

And finally found in Kernel documentation (!!):

While read system calls are supported on files that reside on hugetlb file systems, write system calls are not.

Then I’m wondering the real usage of this mmap method using Huge Pages…

There are few posts about using _realfree_heap_pagesize_hint initialization parameter to make PGA use Huge Pages but it does not work on Linux…

Remark:
Huge Pages size can also be obtained using:

[root@server1 orcl]# hugeadm --page-sizes-all
2097152

Transparent Huge Pages (THP)

Official documentation is available at:

[root@server1 ~]# ll /usr/share/doc/kernel-doc-2.6.39/Documentation/vm/transhuge.txt
-r--r--r-- 1 root root 14310 Jan  4 03:06 /usr/share/doc/kernel-doc-2.6.39/Documentation/vm/transhuge.txt

As its name stand for it is Huge Pages benefit:

Bigger pages (2MB on Linux) so 512 times less ping-pong with kernel for page fault
Faster and reduced Translation Lookaside Buffer (TLB) misses and reduced kernel PageSize

And this transparently for every applications without the need to allocate them in advance (no server reboot). The only drawback I see versus standard Huge Pages is that they can be swap out (as 4KB pages) even if RedHat state it is an added value, at least not for Oracle databases.

You can also see in documentation some interesting insights for future:

Currently it only works for anonymous memory mappings but in the future it can expand over the pagecache layer starting with tmpfs.

To activate THP set below file to always or madvise:

[root@server1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never

This will start khugepaged daemon:

[root@server1 ~]# ps -ef | grep khugepaged | grep -v grep
root        26     2  0 Feb04 ?        00:00:02 [khugepaged]

Memory automatic defragmentation to allocate HugePages is controlled by:

[root@server1 ~]# cat /sys/kernel/mm/transparent_hugepage/defrag
[always] madvise never

You can, as well, find many khugepaged parameters in /sys/kernel/mm/transparent_hugepage/khugepaged directory.

RedHat claim 10% improvement using THP, but how to monitor THP usage ?

For testing I have used below settings, and measured memory allocation before and after starting an Oracle database:

*.sga_target=150m
*.pga_aggregate_target=100m
*.pre_page_sga=TRUE
*.use_large_pages='ONLY'

For Linux:

[root@server1 ~]# sysctl -w vm.nr_hugepages=77
vm.nr_hugepages = 77

Before:

[root@server1 ~]# grep Ano /proc/meminfo
AnonPages:         24140 kB
AnonHugePages:      6144 kB
[root@server1 ~]# egrep 'trans|thp' /proc/vmstat
nr_anon_transparent_hugepages 3
thp_fault_alloc 3765
thp_fault_fallback 201
thp_collapse_alloc 805
thp_collapse_alloc_failed 14
thp_split 167

After:

[root@server1 ~]# grep Ano /proc/meminfo
AnonPages:        426380 kB
AnonHugePages:    350208 kB
[root@server1 ~]# egrep 'trans|thp' /proc/vmstat
nr_anon_transparent_hugepages 173
thp_fault_alloc 3996
thp_fault_fallback 233
thp_collapse_alloc 805
thp_collapse_alloc_failed 14
thp_split 167

So yes it is used but it’s difficult to know what is really using it (transparent you said ?), here I know because the Oracle database is the only thing that has been started… On top of this mapping what has been set at Oracle level (PGA = 100MB) and what you see at OS level is not possible…

References

Overview of RAMFS and TMPFS on Linux
Use and Impact of tmpfs and ramfs for Applications Running on a Compute Node [ID 1394580.1]
HugePages and Oracle Database 11g Automatic Memory Management (AMM) on Linux [ID 749851.1]
_realfree_heap_pagesize_hint – Assessing the impact on Solaris and Linux
How to use and monitor transparent hugepages in Red Hat Enterprise Linux 6?

Have you recently taken a backup of your SQL Server databases ?

Yannick — Tue, 12 Feb 2013 10:44:41 +0000

Available databases backup
References

Available databases backup

Have you recently taken a backup of your SQL Server databases ? Where I work we do full database backup on disk with T-SQL BACKUP command. Those backup files are then put on tape using Tivoli Storage Manager (TSM).

The below T-SQL will list backups that have been taken, or not (backup_start_date column will be NULL), for all your databases (outer join) within the last 10 days:

SELECT
  e.name AS database_name,
  d.backup_start_date,
  d.backup_finish_date,
  DATEDIFF(mi,d.backup_start_date,d.backup_finish_date) AS "backup duration (mi)",
  d.name,
  d.physical_device_name,
  CAST(ROUND(d.backup_size/(1024*1024),0) AS INT) AS "backup size (MB)",
  d.is_compressed,
  d.recovery_model
FROM
  (
    SELECT
      a.backup_set_id,
      a.name,
      a.backup_start_date,
      a.backup_finish_date,
      a.backup_size,
      a.database_name,
      a.recovery_model,
      b.is_compressed,
      c.physical_device_name
    FROM
      msdb.dbo.backupset a,
      msdb.dbo.backupmediaset b,
      msdb.dbo.backupmediafamily c
    WHERE a.media_set_id = b.media_set_id
    AND b.media_set_id = c.media_set_id
    AND a.type = 'D'
    AND a.backup_start_date > GETDATE()-10
  )
  d
RIGHT OUTER JOIN sys.databases e
ON d.database_name = e.name
ORDER BY d.backup_set_id DESC;

Where type can take below value:

Backup type. Can be:
D = Database
I = Differential database
L = Log
F = File or filegroup
G =Differential file
P = Partial
Q = Differential partial
Can be NULL.

References

backupset (Transact-SQL)

IT WorldIT World

AWR baselines and performance comparison

Table Of Contents

Preamble

Good performance

Bad performance

Performance improved

Conclusion

References

MySQL 5.6 and MySQL 5.5 performance comparison

Table Of Contents

Preamble

Configuration

Results and conclusion

References

Media recovery with NOLOGGING tables

Table Of Contents

Preamble

Nologging and CTAS

Nologging and insert command

Tablespaces logging attributes

RMAN and nologging operations

References

Tmpfs vs Ramfs vs (Transparent) Huge Pages

Table Of Contents

Preamble

Tmpfs

Ramfs

Huge Pages

Transparent Huge Pages (THP)

References

Have you recently taken a backup of your SQL Server databases ?

Table Of Contents

Available databases backup

References