IT World https://blog.yannickjaquier.com RDBMS, Unix and many more... Thu, 11 Oct 2018 10:40:03 +0000 en-US hourly 1 https://wordpress.org/?v=4.9.8 Oracle REST Data Services (ORDS) installation and usage https://blog.yannickjaquier.com/oracle/oracle-rest-data-services-ords.html https://blog.yannickjaquier.com/oracle/oracle-rest-data-services-ords.html#respond Mon, 08 Oct 2018 13:20:36 +0000 https://blog.yannickjaquier.com/?p=4334 Preamble In the database world technology trend you have surely already heard buzz words like Hadoop and NoSQL. Around those new non-relational databases there is a common open-standard file format massively used to read and write on those new databases call JavaScript Object Notation (JSON). If you have followed a bit few new web technologies […]

The post Oracle REST Data Services (ORDS) installation and usage appeared first on IT World.

]]>

Table of contents

Preamble

In the database world technology trend you have surely already heard buzz words like Hadoop and NoSQL. Around those new non-relational databases there is a common open-standard file format massively used to read and write on those new databases call JavaScript Object Notation (JSON).

If you have followed a bit few new web technologies trainings (Vue.js, Angular, React, …) each time they use a back-end database to store information the exchanges are always done through an asynchronous request using promises (Axios for the one I have used in a Vue.js project). The information are, also, always transferred using JSON format. Those exposed services by newest databases flavors are using a REpresentational State Transfer (REST) architecture and expose this is a RESTful web service or RESTful API.

Oracle corporation with their legacy Oracle database (back in the 80’s for first release) have climb on the bandwagon and have created a product called Oracle REST Data Services (ORDS) to make the bridge between newest generation of developers and (becoming old) past generation DBAs (guess in which category I am ?). This product/tool that is a simple jar file to run through Java is creating a RESTful service to expose Oracle database figures through http(s) requests…

To have figures to display I have decided, this time, to use the default HR sample schema that you can create using script located at:

$ORACLE_HOME/demo/schema/human_resources/hr_main.sql

Idea is to display and interact with employees table using a RESTful API. We would be able to display all employees or a particular one specifying an id in provided url as well as inserting, deleting and updating additional ones.

Testing has been done using a VirtualBox virtual machine running Oracle Linux Server release 7.4 and an Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 – 64bit.

ORDS installation

I could have used the JDK of the Oracle home but I rated the release a bit too old:

[oracle@server1 ~]$ $ORACLE_HOME/jdk/bin/java -version
java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

As I’m rock’n’roll I have decided to install and use Java 10:

[oracle@server1 ~]$ java -version
java version "10" 2018-03-20
Java(TM) SE Runtime Environment 18.3 (build 10+46)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)

There is also an $ORACLE_HOME/ords but I have not been able to make it working so decided to replace it with the latest release available at the time of writing this post:

[oracle@server1 ords]$ java -jar ords.war version
Oracle REST Data Services 18.1.1.95.1251

I have also chosen the multitenant installation even if I have only on pluggable database, the installation is called Installation Enabling Multiple Releases (Recommended). In the advanced installation process you must supply the container (CDB) information and it will eb deployed in all pluggable databases including the seed one (and also in root one for common objects and accounts).

Note:
I have discovered that the configuration directory is relative to the directory where you have started the installation so simply config for me to be in $ORACLE_HOME/ords/config..

[oracle@server1 ords]$ cd $ORACLE_HOME/ords
[oracle@server1 ords]$ java -jar ords.war install advanced
This Oracle REST Data Services instance has not yet been configured.
Please complete the following prompts

Enter the location to store configuration data:config
Enter the name of the database server [localhost]:server1.domain.com
Enter the database listen port [1521]:1531
Enter 1 to specify the database service name, or 2 to specify the database SID [1]:
Enter the database service name:orcl
Enter 1 if you want to verify/install Oracle REST Data Services schema or 2 to skip this step [1]:
Enter the database password for ORDS_PUBLIC_USER:
Confirm password:
Requires SYS AS SYSDBA to verify Oracle REST Data Services schema.

Enter the database password for SYS AS SYSDBA:
Confirm password:

Retrieving information...
Your database connection is to a CDB.  ORDS common user ORDS_PUBLIC_USER will be created in the CDB.  ORDS schema will be installed in the PDBs.
Root CDB$ROOT - create ORDS common user
PDB PDB$SEED - install ORDS 18.1.1.95.1251 (mode is READ ONLY, open for READ/WRITE)
PDB PDB1 - install ORDS 18.1.1.95.1251

Enter 1 if you want to install ORDS or 2 to skip this step [1]:
Enter the default tablespace for ORDS_METADATA [SYSAUX]:
Enter the temporary tablespace for ORDS_METADATA [TEMP]:
Enter the default tablespace for ORDS_PUBLIC_USER [SYSAUX]:
Enter the temporary tablespace for ORDS_PUBLIC_USER [TEMP]:
Enter 1 if you want to use PL/SQL Gateway or 2 to skip this step.
If using Oracle Application Express or migrating from mod_plsql then you must enter 1 [1]:
Enter the PL/SQL Gateway database user name [APEX_PUBLIC_USER]:
Enter the database password for APEX_PUBLIC_USER:
Confirm password:
Enter 1 to specify passwords for Application Express RESTful Services database users (APEX_LISTENER, APEX_REST_PUBLIC_USER) or 2 to skip this step [1]:
Enter the database password for APEX_LISTENER:
Confirm password:
Enter the database password for APEX_REST_PUBLIC_USER:
Confirm password:
Apr 11, 2018 5:18:14 PM
INFO: Updated configurations: defaults, apex, apex_pu, apex_al, apex_rt


Installing Oracle REST Data Services version 18.1.1.95.1251 in CDB$ROOT
... Log file written to /u01/app/oracle/product/12.2.0/dbhome_1/ords/logs/ords_cdb_install_core_CDB_ROOT_2018-04-11_171814_00573.log
... Verified database prerequisites
... Created Oracle REST Data Services proxy user
Completed installation for Oracle REST Data Services version 18.1.1.95.1251. Elapsed time: 00:00:01.690

Installing Oracle REST Data Services version 18.1.1.95.1251 in PDB$SEED
... Log file written to /u01/app/oracle/product/12.2.0/dbhome_1/ords/logs/ords_cdb_install_core_PDB_SEED_2018-04-11_171818_00681.log
... Verified database prerequisites
... Created Oracle REST Data Services schema
... Created Oracle REST Data Services proxy user
... Granted privileges to Oracle REST Data Services
... Created Oracle REST Data Services database objects
... Log file written to /u01/app/oracle/product/12.2.0/dbhome_1/ords/logs/ords_cdb_install_datamodel_PDB_SEED_2018-04-11_171947_00378.log
... Log file written to /u01/app/oracle/product/12.2.0/dbhome_1/ords/logs/ords_cdb_install_apex_PDB_SEED_2018-04-11_171955_00184.log
Completed installation for Oracle REST Data Services version 18.1.1.95.1251. Elapsed time: 00:01:43.21

Installing Oracle REST Data Services version 18.1.1.95.1251 in PDB1
... Log file written to /u01/app/oracle/product/12.2.0/dbhome_1/ords/logs/ords_cdb_install_core_PDB1_2018-04-11_172036_00713.log
... Verified database prerequisites
... Created Oracle REST Data Services schema
... Created Oracle REST Data Services proxy user
... Granted privileges to Oracle REST Data Services
... Created Oracle REST Data Services database objects
... Log file written to /u01/app/oracle/product/12.2.0/dbhome_1/ords/logs/ords_cdb_install_datamodel_PDB1_2018-04-11_172204_00677.log
... Log file written to /u01/app/oracle/product/12.2.0/dbhome_1/ords/logs/ords_cdb_install_apex_PDB1_2018-04-11_172210_00596.log
Completed installation for Oracle REST Data Services version 18.1.1.95.1251. Elapsed time: 00:01:38.462

Completed CDB installation for Oracle REST Data Services version 18.1.1.95.1251.
Total elapsed time: 00:04:00.631

Enter 1 if you wish to start in standalone mode or 2 to exit [1]:2

You can have the configuration directory with:

[oracle@server1 ords]$ java -jar ords.war configdir
Apr 12, 2018 12:31:09 PM
INFO: The config.dir value is /u01/app/oracle/product/12.2.0/dbhome_1/ords/config

I have decided to not start it after installation to understand the real start procedure, the first time you start it a set of questions will be asked. As Internet is know almost full HTTPS this is the option I have chosen. In any case if you plan to use REST Enabled SQL HTTPS has to be chosen:

[oracle@server1 ords]$ java -jar ords.war standalone
Enter the APEX static resources location:
Enter 1 if using HTTP or 2 if using HTTPS [1]:2
Enter the HTTPS port [8443]:
Enter the SSL hostname:server1.domain.com
Enter 1 to use the self-signed certificate or 2 if you will provide the SSL certificate [1]:
2018-04-11 17:30:52.110:INFO::main: Logging initialized @26842ms to org.eclipse.jetty.util.log.StdErrLog
Apr 11, 2018 5:30:53 PM
INFO: HTTPS and HTTPS/2 listening on port: 8443
Apr 11, 2018 5:30:53 PM
INFO: Disabling document root because the specified folder does not exist: /u01/app/oracle/product/12.2.0/dbhome_1/ords/config/ords/standalone/doc_root
2018-04-11 17:30:53.408:INFO:oejs.Server:main: jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
2018-04-11 17:30:53.464:INFO:oejs.session:main: DefaultSessionIdManager workerName=node0
2018-04-11 17:30:53.464:INFO:oejs.session:main: No SessionScavenger set, using defaults
2018-04-11 17:30:53.465:INFO:oejs.session:main: Scavenging every 660000ms
Apr 11, 2018 5:30:55 PM
WARNING: The pool named: |apex|| is invalid and will be ignored: The username or password for the connection pool named apex, are invalid, expired, or the account is locked
Apr 11, 2018 5:30:56 PM
INFO: Creating Pool:|apex|pu|
Apr 11, 2018 5:30:56 PM
INFO: Configuration properties for: |apex|pu|
cache.caching=false
cache.directory=/tmp/apex/cache
cache.duration=days
cache.expiration=7
cache.maxEntries=500
cache.monitorInterval=60
cache.procedureNameList=
cache.type=lru
db.hostname=server1.domain.com
db.password=******
db.port=1531
db.servicename=orcl
db.username=ORDS_PUBLIC_USER
debug.debugger=false
debug.printDebugToScreen=false
error.keepErrorMessages=true
error.maxEntries=50
jdbc.DriverType=thin
jdbc.InactivityTimeout=1800
jdbc.InitialLimit=3
jdbc.MaxConnectionReuseCount=1000
jdbc.MaxLimit=10
jdbc.MaxStatementsLimit=10
jdbc.MinLimit=1
jdbc.statementTimeout=900
log.logging=false
log.maxEntries=50
misc.compress=
misc.defaultPage=apex
security.disableDefaultExclusionList=false
security.maxEntries=2000
security.requestValidationFunction=wwv_flow_epg_include_modules.authorize
security.validationFunctionType=plsql

Apr 11, 2018 5:30:56 PM
WARNING: *** jdbc.MaxLimit in configuration |apex|pu| is using a value of 10, this setting may not be sized adequately for a production environment ***
Apr 11, 2018 5:30:56 PM
WARNING: *** jdbc.InitialLimit in configuration |apex|pu| is using a value of 3, this setting may not be sized adequately for a production environment ***
Apr 11, 2018 5:30:57 PM
WARNING: The pool named: |apex|al| is invalid and will be ignored: The username or password for the connection pool named apex_al, are invalid, expired, or the account is locked
Apr 11, 2018 5:30:58 PM
WARNING: The pool named: |apex|rt| is invalid and will be ignored: The username or password for the connection pool named apex_rt, are invalid, expired, or the account is locked
Apr 11, 2018 5:30:58 PM
INFO: Oracle REST Data Services initialized
Oracle REST Data Services version : 18.1.1.95.1251
Oracle REST Data Services server info: jetty/9.4.z-SNAPSHOT

2018-04-11 17:30:58.804:INFO:oejsh.ContextHandler:main: Started o.e.j.s.ServletContextHandler@eadd4fb{/ords,null,AVAILABLE,@Secured}
2018-04-11 17:30:58.821:INFO:oejus.SslContextFactory:main: x509=X509@7ce97ee5(selfsigned,h=[server1.domain.com],w=[]) for SslContextFactory@32c8e539[provider=null,keyStore=oracle.dbtools.standalone.InMemoryResource@73dce0e6,trustStore=oracle.dbtools.standalone.InMemoryResource@73dce0e6]
2018-04-11 17:30:58.888:INFO:oejs.AbstractConnector:main: Started Secured@a5bd950{SSL,[ssl, alpn, h2, http/1.1]}{0.0.0.0:8443}
2018-04-11 17:30:58.888:INFO:oejs.Server:main: Started @33622ms

Obviously you would need to use nohup command because in interactive mode the process is stopped when you quit the terminal…

To make all PDBs addressable by Oracle REST Data Services (Pluggable Mapping) I have finally used below command. A bit different than Oracle official documentation and my DB_DOMIAN parameter is unset:

[oracle@server1 ords]$ java -jar ords.war set-property db.serviceNameSuffix ''
Apr 12, 2018 12:38:47 PM oracle.dbtools.rt.config.setup.SetProperty execute
INFO: Modified: /u01/app/oracle/product/12.2.0/dbhome_1/ords/config/ords/defaults.xml, setting: db.serviceNameSuffix =

Then I have added my single pluggable database with:

[oracle@server1 ords]$ java -jar ords.war setup --database pdb1
Enter the name of the database server [server1.domain.com]:
Enter the database listen port [1531]:
Enter 1 to specify the database service name, or 2 to specify the database SID [1]:
Enter the database service name [orcl]:pdb1
Enter 1 if you want to verify/install Oracle REST Data Services schema or 2 to skip this step [1]:
Enter the database password for ORDS_PUBLIC_USER:
Confirm password:

Retrieving information.
Enter 1 if you want to use PL/SQL Gateway or 2 to skip this step.
If using Oracle Application Express or migrating from mod_plsql then you must enter 1 [1]:2
Apr 12, 2018 3:13:42 PM
INFO: Updated configurations: pdb1_pu
Apr 12, 2018 3:13:42 PM oracle.dbtools.rt.config.setup.SchemaSetup install
INFO: Oracle REST Data Services schema version 18.1.1.95.1251 is installed.

It has created below configuration file:

[oracle@server1 conf]$ cat $ORACLE_HOME/ords/config/ords/conf/pdb1_pu.xml



Saved on Thu Apr 12 15:13:42 CEST 2018
@05695FBEB8J4C5B200HG56EB28FF0F27B61F9B56AFF75F8472
pdb1
ORDS_PUBLIC_USER

I define the routing based on the request path prefix with (nothing original as the url will have the pluggable database name):

[oracle@server1 ords]$ java -jar ords.war map-url --type base-path /pdb1 pdb1
Apr 12, 2018 3:22:25 PM
INFO: Creating new mapping from: [base-path,/pdb1] to map to: [pdb1,,]

ORDS setup

When trying with SYS account I have gotten a strange error:

SQL> exec ords.enable_schema(p_schema => 'hr', p_url_mapping_type => 'BASE_PATH', p_url_mapping_pattern => 'hr');
BEGIN ords.enable_schema(p_schema => 'hr', p_url_mapping_type => 'BASE_PATH', p_url_mapping_pattern => 'hr'); END;

*
ERROR at line 1:
ORA-06598: insufficient INHERIT PRIVILEGES privilege
ORA-06512: at "ORDS_METADATA.ORDS", line 1
ORA-06512: at line 1

So finally executed it with my nominative DBA account:

SQL> exec ords.enable_schema(p_schema => 'hr', p_url_mapping_type => 'BASE_PATH', p_url_mapping_pattern => 'hr');

PL/SQL procedure successfully completed.

Instead of using the example of the official documentation:

exec ords.define_service(p_module_name => 'examples.routes', p_base_path => '/examples/routes/', p_pattern => 'greeting/:name', -
p_source => 'select ''Hello '' || :name || '' from '' || nvl(:whom,sys_context(''USERENV'',''CURRENT_USER'')) "greeting" from dual');

I have decided to try something much simpler for my first test but it failed for a strange error:

SQL> show user
USER is "HR"
SQL> exec ords.define_service(p_module_name => 'examples', p_base_path => 'examples/', p_method => 'GET', p_pattern => 'greeting/', -
     p_source => 'select sysdate from dual');
BEGIN ords.define_service(p_module_name => 'examples', p_base_path => 'examples/', p_method => 'GET', p_pattern => 'greeting/',  p_source => 'select sysdate from dual'); END;

*
ERROR at line 1:
ORA-01403: no data found
ORA-06512: at "ORDS_METADATA.ORDS_INTERNAL", line 617
ORA-06512: at "ORDS_METADATA.ORDS_SECURITY", line 85
ORA-06512: at "ORDS_METADATA.ORDS_SERVICES", line 117
ORA-06512: at "ORDS_METADATA.ORDS_SERVICES", line 52
ORA-06512: at "ORDS_METADATA.ORDS", line 694
ORA-06512: at line 1

Then the magic idea came to my mind and while I was grumbling about the fact that Oracle could have used a UPPER command for account name I remembered that since 11g account are now case sensitive, and this by default:

SQL> show parameter SEC_CASE_SENSITIVE_LOGON

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
sec_case_sensitive_logon             boolean     TRUE

SQL> select parsing_schema, status, auto_rest_auth from ords_metadata.ords_schemas;

PARSING_SCHEMA                 STATUS                         AUTO_REST_AUTH
------------------------------ ------------------------------ ------------------------------
ORDS_METADATA                  DISABLED                       ENABLED
hr                             ENABLED                        ENABLED

So did a bit of cleaning with (no DISABLE_SCHEMA or using false with ENABLE_SCHEMA is not deleting the line in ORDS_METADATA.ORDS_SCHEMAS:

SQL> EXECUTE ORDS.DROP_REST_FOR_SCHEMA('hr');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

SQL> exec ords.enable_schema(p_schema => 'HR', p_url_mapping_type => 'BASE_PATH', p_url_mapping_pattern => 'hr');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

SQL> select parsing_schema, status, auto_rest_auth from user_ords_schemas;

PARSING_SCHEMA                 STATUS                         AUTO_REST_AUTH
------------------------------ ------------------------------ ------------------------------
HR                             ENABLED                        ENABLED

And finally the service definition went well (even if it looks stupid the COMMIT is strongly suggested), I have also decided to format a bit the date display:

SQL> show user
USER is "HR"
SQL> exec ords.define_service(p_module_name => 'examples', p_base_path => '/examples/', p_method => 'GET', p_pattern => '/greeting/', -
     p_source => 'select to_char(sysdate,''dd-mon-yyyy hh24:mi:ss'') as current_date from dual');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

Then a simple GET request on the url should provide current date. To be honest I have not expected to spend so much time on this !! Initial request with Curl failed::

[oracle@server1 ~]$ curl https://server1.domain.com:8443/ords/pdb1/hr/examples/greeting/
curl: (60) Issuer certificate is invalid.
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
[oracle@server1 ~]$ curl -k https://server1.domain.com:8443/ords/pdb1/hr/examples/greeting/
invalid_preface

So decided to activate debugging mode with ($ORACLE_HOME/ords/config/ords/defaults.xml file):

true
true

Either you edit the file or use:

[oracle@server1 ords]$ java -jar ords.war set-property debug.debugger true
Apr 13, 2018 11:50:21 AM oracle.dbtools.rt.config.setup.SetProperty execute
INFO: Modified: /u01/app/oracle/product/12.2.0/dbhome_1/ords/config/ords/defaults.xml, setting: debug.debugger = true
[oracle@server1 ords]$ java -jar ords.war set-property debug.printDebugToScreen true
Apr 13, 2018 11:50:36 AM oracle.dbtools.rt.config.setup.SetProperty execute
INFO: Modified: /u01/app/oracle/product/12.2.0/dbhome_1/ords/config/ords/defaults.xml, setting: debug.printDebugToScreen = true

But to be honest this has bring nothing to help me… Do not forget to remove it afterwards as it is quite verbose… Then I noticed the HTTPS/2 in ORDS startup output:

INFO: HTTPS and HTTPS/2 listening on port: 8443

And wanted to use the –http2 option to tell curl to use HTTP version 2 but the release available in my OEL 7.4 (curl 7.29.0) at the time of writing this post is too old.

I have tried to download a binary on my Windows desktop and it has worked but I have remembered a Web training video where the presenter has introduced Postman. But I have not been able to make it working… So finally downloaded Insomnia and yeepee got the expected result:

ords01
ords01

If I try with a parameter in the url to get information for only one employee:

SQL> show user
USER is "HR"
SQL> exec ords.define_service(p_module_name => 'employees', p_base_path => '/employees/', p_method => 'GET', p_pattern => '/:id', -
     p_source => 'select * from employees where employee_id=:id');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

I can now specify in the url the employee id I would like to display:

ords02
ords02

With dictionary views you can double check what has been done (no ALL_xx or DBA_xx views, account owner is ORDS_METADATA):

SQL> set lines 200
SQL> col pattern for a15
SQL> col name for a15
SQL> col uri_prefix for a15
SQL> col uri_template for a15
SQL> col source_type for a20
SQL> select id, parsing_schema, type, pattern, status, auto_rest_auth from user_ords_schemas;

        ID PARSING_SCHEMA                 TYPE       PATTERN         STATUS                         AUTO_REST_AUTH
---------- ------------------------------ ---------- --------------- ------------------------------ ------------------------------
     10062 HR                             BASE_PATH  hr              ENABLED                        ENABLED

SQL> select id, name, uri_prefix, items_per_page, status from user_ords_modules;

        ID NAME            URI_PREFIX      ITEMS_PER_PAGE STATUS
---------- --------------- --------------- -------------- ------------------------------
     10120 employees       /employees/                 25 PUBLISHED
     10067 examples        /examples/                  25 PUBLISHED

SQL> select id, module_id, uri_template from user_ords_templates;

        ID  MODULE_ID URI_TEMPLATE
---------- ---------- ---------------
     10133      10120 /:id
     10101      10067 /greeting/

SQL> select id, template_id, source_type, method, source from user_ords_handlers;

        ID TEMPLATE_ID SOURCE_TYPE          METHOD     SOURCE
---------- ----------- -------------------- ---------- --------------------------------------------------------------------------------
     10134       10133 json/collection      GET        select * from employees where employee_id=:id
     10102       10101 json/collection      GET        select to_char(sysdate,'dd-mon-yyyy hh24:mi:ss') as current_date from dual

Automatic Enabling of Schema Objects for REST Access (AutoREST)

So far we have seen on how to fetch data from tables but how we modify them ? When looking in official documentation on how to update, delete and insert rows in objects I end up in AutoREST chapter. This is in fact linked to two procedures of ORDS package that remain unused: ENABLE_SCHEMA and ENABLE_OBJECT.

The shortest and self explaining definition of AutoREST is:

AutoREST is a quick and easy way to expose database tables as REST resources.

By default AutoREST is enabled but when using ENABLE_SCHEMA you might want to deactivate AutoREST authentication for easier testing (needless to say you must not do this in production):

SQL> exec ords.enable_schema(p_schema => 'HR', p_url_mapping_type => 'BASE_PATH', p_url_mapping_pattern => 'hr', p_auto_rest_auth => FALSE);

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

SQL> select parsing_schema, type, pattern, status, auto_rest_auth from user_ords_schemas;

PARSING_SCHEMA                 TYPE       PATTERN         STATUS                         AUTO_REST_AUTH
------------------------------ ---------- --------------- ------------------------------ ------------------------------
HR                             BASE_PATH  hr              ENABLED                        DISABLED

Means we can display meta-data using special url:

ords03
ords03

Objects on the contrary are not enables by default:

SQL> set pages 1000
SQL> col parsing_object for a20
SQL> col object_alias for a20
SQL> select parsing_object, object_alias, type, status, auto_rest_auth from user_ords_objects;

PARSING_OBJECT       OBJECT_ALIAS         TYPE                           STATUS                         AUTO_REST_AUTH
-------------------- -------------------- ------------------------------ ------------------------------ ------------------------------
ADD_JOB_HISTORY      add_job_history      PROCEDURE                      DISABLED                       ENABLED
COUNTRIES            countries            TABLE                          DISABLED                       ENABLED
DEPARTMENTS          departments          TABLE                          DISABLED                       ENABLED
EMPLOYEES            employees            TABLE                          DISABLED                       ENABLED
EMP_DETAILS_VIEW     emp_details_view     VIEW                           DISABLED                       ENABLED
JOBS                 jobs                 TABLE                          DISABLED                       ENABLED
JOB_HISTORY          job_history          TABLE                          DISABLED                       ENABLED
LOCATIONS            locations            TABLE                          DISABLED                       ENABLED
REGIONS              regions              TABLE                          DISABLED                       ENABLED
SECURE_DML           secure_dml           PROCEDURE                      DISABLED                       ENABLED

10 rows selected.

Let enable EMPLOYEES table with (I have chosen another alias not to mess up with already existing one we have created above). I also deactivate authentication (needless to say you must not do this in production):

SQL> exec ords.enable_object(p_enabled => TRUE, p_schema => 'HR', p_object => 'EMPLOYEES', -
     p_object_type => 'TABLE',  p_object_alias => 'emp', p_auto_rest_auth => FALSE);

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

SQL> select parsing_object, object_alias, type, status, auto_rest_auth from user_ords_enabled_objects;

PARSING_OBJECT       OBJECT_ALIAS         TYPE                           STATUS                         AUTO_REST_AUTH
-------------------- -------------------- ------------------------------ ------------------------------ ------------------------------
EMPLOYEES            emp                  TABLE                          ENABLED                        DISABLED

You can get meta-data with:

ords04
ords04

And here a list of multiple possible queries (not displaying a picture each time):

  • https://server1.domain.com:8443/ords/pdb1/hr/emp/ to display all employees
  • https://server1.domain.com:8443/ords/pdb1/hr/emp/100 to display employee id 100
  • https://server1.domain.com:8443/ords/pdb1/hr/emp/?limit=5 to display first five employees

To insert a row, I get REST current date format with (ISO 8601):

SQL> select to_char(sysdate,'YYYY-MM-DD')||'T'||to_char(sysdate,'HH24:MI:SS')||'Z' from dual;

TO_CHAR(SYSDATE,'YYY
--------------------
2018-04-19T16:59:36Z
ords05
ords05

To delete the just inserted row:

ords06
ords06

PUT would be use to perform an upsert (insert or update). Have a look to ORDS official documentation it contains plenty of different cases…

REST-Enabled SQL Service

From official documentation:

The REST Enabled SQL service is a HTTPS web service that provides access to the Oracle Database SQL Engine. You can POST SQL statements to the service. The service then runs the SQL statements against Oracle database and returns the result to the client in a JSON format.

I have activated REST-Enabled SQL Service with:

[oracle@server1 ords]$ java -jar ords.war set-property restEnabledSql.active true
Apr 12, 2018 12:41:30 PM oracle.dbtools.rt.config.setup.SetProperty execute
INFO: Modified: /u01/app/oracle/product/12.2.0/dbhome_1/ords/config/ords/defaults.xml, setting: restEnabledSql.active = true

The documentation provide below command with curl:

curl -i -X POST --user ORDSTEST:ordstest --data-binary "select sysdate from dual" -H "Content-Type: application/sql" -k https://localhost:8088/ords/ordstest/_/sql

With Insomnia it gives:

ords07
ords07
ords08
ords08
ords09
ords09

On the right part of the above screen shots we can see the correct result: 107 rows.

Configuring Secure Access to RESTful Services

We have configured our RESTful service with an HTTPS access but what if someone has been able to copy past the url ? Then he would be able to display all our employees information, which is most probably not a good thing. How to restrict this ? Well fortunately it has been implemented and you can secure RESTful API access with two methods:

  • First Party Cookie-Based Authentication
  • Third Party OAuth 2.0-Based Authentication

First Party Cookie-Based Authentication

Start by creating a role:

SQL> show user
USER is "HR"
SQL> exec ords.create_role(p_role_name => 'employees_role');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

Create a privilege associated with the role:

SQL>
DECLARE
l_arr OWA.vc_arr;
BEGIN
l_arr(1) := 'employees_role';
ords.define_privilege(p_privilege_name => 'employees_priv', p_roles => l_arr, p_label => 'Employees data', -
                      p_description => 'Securing access to employees data');
commit;
END;
/

PL/SQL procedure successfully completed.

Protect the RESTful API with he newly created privilege with (procedure not documented at the time of writing this post in ORDS 18.1):

SQL> exec ords.create_privilege_mapping(p_privilege_name => 'employees_priv', p_pattern => '/employees/*');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

With ORDS dictionary views it gives:

SQL> set lines 200
SQL> col name for a15
SQL> col label for a20
SQL> col description for a40
SQL> col pattern for a15
SQL> col privilege_name for a15
SQL> col role_name for a15
SQL> select id, name, schema_id from user_ords_roles where name='employees_role';

        ID NAME             SCHEMA_ID
---------- --------------- ----------
     10135 employees_role       10062

SQL> select id, label, name, description from user_ords_privileges where name='employees_priv';

        ID LABEL                NAME            DESCRIPTION
---------- -------------------- --------------- ----------------------------------------
     10136 Employees data       employees_priv  Securing access to employees data

SQL> select privilege_id, privilege_name, role_id, role_name from user_ords_privilege_roles where privilege_name='employees_priv';

PRIVILEGE_ID PRIVILEGE_NAME     ROLE_ID ROLE_NAME
------------ --------------- ---------- ---------------
       10136 employees_priv       10135 employees_role

SQL> select privilege_id, name, pattern from user_ords_privilege_mappings where name='employees_priv';

PRIVILEGE_ID NAME            PATTERN
------------ --------------- ---------------
       10136 employees_priv  /employees/*

finally as expected the RESTful API is no more accessible (HTTP error 401 Unauthorized):

ords10
ords10

Create a user to access again the RESTful API with:

[oracle@server1 ords]$ java -jar ords.war user hr_user employees_role
Enter a password for user hr_user:
Confirm password for user hr_user:
Apr 17, 2018 4:40:27 PM oracle.dbtools.standalone.ModifyUser execute
INFO: Created user: hr_user in file: /u01/app/oracle/product/12.2.0/dbhome_1/ords/config/ords/credentials

And then either you click on the link if you use a browser or with Insommia you can fill in the authentication tab as follow and you can again access the RESTfull API in a secure manner:

ords11
ords11

Third Party OAuth 2.0-Based Authentication

SQL> exec oauth.create_client(p_name => 'My Employees Application', p_grant_type => 'client_credentials', p_owner => 'Yannick', -
     p_description => 'A Vue.JS client to access Employees data', p_support_email => 'yannick@domain.com', p_privilege_names => 'employees_priv');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.
SQL> exec oauth.grant_client_role(p_client_name => 'My Employees Application', p_role_name  => 'employees_role');

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

With my Insomnia tool the access is then a bit different as I am not required to get the access token first. Insomnia is able to do it in one call. I have chosen OAuth 2 in authentication method and I specify token url, client id and client secret the url I aim to fetch is still the same (the access token has been auto-filled by Insomnia):

ords12
ords12

In Timeline tab of response we can see that Insomnia as fetch the access token (Authorization: Bearer line) and used it to fetch my final url to get employee id 100 information:

ords13
ords13

With ORDS dictionary views it gives:

SQL> col name for a30
SQL> col client_name for a25
SQL> col response_type for a15
SQL> select name, description, response_type, client_id, client_secret from user_ords_clients;

NAME                           DESCRIPTION                              RESPONSE_TYPE   CLIENT_ID                        CLIENT_SECRET
------------------------------ ---------------------------------------- --------------- -------------------------------- --------------------------------
My Employees Application       A Vue.JS client to access Employees data TOKEN           6YoYpRwsaeH19coruhZAsw..         yM49CRbG8WkvAb5AUx07lA..

SQL> select * from user_ords_client_roles;

 CLIENT_ID CLIENT_NAME                  ROLE_ID ROLE_NAME
---------- ------------------------- ---------- ---------------
     10163 My Employees Application       10135 employees_role

     SQL> select name, label, description, client_name FROM user_ords_client_privileges;

NAME                           LABEL                DESCRIPTION                              CLIENT_NAME
------------------------------ -------------------- ---------------------------------------- -------------------------
employees_priv                 Employees data       Securing access to employees data        My Employees Application

ORDS uninstall

To remove ORDS from you database use:

[oracle@server1 ords]$ java -jar ords.war uninstall advanced

References

The post Oracle REST Data Services (ORDS) installation and usage appeared first on IT World.

]]>
https://blog.yannickjaquier.com/oracle/oracle-rest-data-services-ords.html/feed 0
Application Continuity (AC) for Java – JDBC HA – part 6 https://blog.yannickjaquier.com/oracle/application-continuity-ac-jdbc-ha-part-6.html https://blog.yannickjaquier.com/oracle/application-continuity-ac-jdbc-ha-part-6.html#comments Thu, 13 Sep 2018 13:23:53 +0000 https://blog.yannickjaquier.com/?p=4321 Preamble Application Continuity (AC) feature introduced Oracle Database 12c Release 1 (12.1) masks database outages to application and end users. You must use Transaction Guard 12.2 for using this feature. Application Continuity is a feature of the Oracle JDBC Thin driver and is not supported by JDBC OCI driver. I am using the exact same […]

The post Application Continuity (AC) for Java – JDBC HA – part 6 appeared first on IT World.

]]>

Table of contents

Preamble

Application Continuity (AC) feature introduced Oracle Database 12c Release 1 (12.1) masks database outages to application and end users.

You must use Transaction Guard 12.2 for using this feature.
Application Continuity is a feature of the Oracle JDBC Thin driver and is not supported by JDBC OCI driver.

I am using the exact same test table as with Transaction Guard (TG). I start in this initial state:

SQL> select * from test01;

        ID        VAL
---------- ----------
         1          1
         2          1
         3          1
         4          1
         5          1
         6          1
         7          1
         8          1
         9          1
        10          1

10 rows selected.

You must configure your connection string using RETRY_COUNT, RETRY_DELAY, CONNECT_TIMEOUT, and TRANSPORT_CONNECT_TIMEOUT. As you can find in Oracle documentation:

Starting from Oracle Database 12c Release 2 (12.2.0.1), you must specify the value of the TRANSPORT_CONNECT_TIMEOUT parameter in milliseconds, instead of seconds.

The definition of each parameter:

  • RETRY_COUNT: To specify the number of times an ADDRESS list is traversed before the connection attempt is terminated.
  • RETRY_DELAY: To specify the delay in seconds between subsequent retries for a connection. This parameter works in conjunction with RETRY_COUNT parameter.
  • CONNECT_TIMEOUT: To specify the timeout duration in seconds for a client to establish an Oracle Net connection to an Oracle database.
  • TRANSPORT_CONNECT_TIMEOUT: To specify the transportation timeout duration in milliseconds for a client to establish an Oracle Net connection to an Oracle database.

I have defined mine as:

(DESCRIPTION=
  (RETRY_COUNT=20)
  (RETRY_DELAY=3)
  (CONNECT_TIMEOUT=60)
  (TRANSPORT_CONNECT_TIMEOUT=3000)
  (ADDRESS=
    (PROTOCOL=TCP)
    (HOST=rac-cluster-scan.domain.com)
    (PORT=1531))
    (CONNECT_DATA=(SERVICE_NAME=pdb1srv)))

I have used the exact same service as for Transaction Guard (TG):

[oracle@server2 ~]$ srvctl config service -db orcl -service pdb1srv
Service name: pdb1srv
Server pool: server_pool01
Cardinality: UNIFORM
Service role: PRIMARY
Management policy: AUTOMATIC
DTP transaction: false
AQ HA notifications: true
Global: false
Commit Outcome: true
Failover type: TRANSACTION
Failover method:
TAF failover retries:
TAF failover delay:
Failover restore: NONE
Connection Load Balancing Goal: LONG
Runtime Load Balancing Goal: NONE
TAF policy specification: NONE
Edition:
Pluggable database name: pdb1
Maximum lag time: ANY
SQL Translation Profile:
Retention: 86400 seconds
Replay Initiation Time: 300 seconds
Drain timeout:
Stop option:
Session State Consistency: DYNAMIC
GSM Flags: 0
Service is enabled
Service is individually enabled on nodes:
Service is individually disabled on nodes:
CSS critical: no

Application Continuity (AC) testing

The idea is almost the same as for Transaction Guard (TG) I start a transaction and before the commit I kill h=the pmon of the instance where my Java program has started. And we expect to see… nothing, means that replay will re-submit the transaction and the Java program will exit successfully !

Java testing code (you need to add ojdbc8.jar, ons.jar and ucp.jar to your project):

package ac01;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Locale;
import oracle.jdbc.replay.ReplayStatistics;
import oracle.jdbc.replay.ReplayableConnection;
import oracle.ucp.jdbc.PoolDataSource;
import oracle.ucp.jdbc.PoolDataSourceFactory;

public class ac01 {
  private static void display_replay_statistics(ReplayStatistics rs) {
    System.out.println("FailedReplayCount="+rs.getFailedReplayCount());
    System.out.println("ReplayDisablingCount="+rs.getReplayDisablingCount());
    System.out.println("SuccessfulReplayCount="+rs.getSuccessfulReplayCount());
    System.out.println("TotalCalls="+rs.getTotalCalls());
    System.out.println("TotalCallsAffectedByOutages="+rs.getTotalCallsAffectedByOutages()); 
    System.out.println("TotalCallsAffectedByOutagesDuringReplay="+ rs.getTotalCallsAffectedByOutagesDuringReplay());
    System.out.println("TotalCallsTriggeringReplay="+rs.getTotalCallsTriggeringReplay());
    System.out.println("TotalCompletedRequests="+rs.getTotalCompletedRequests());
    System.out.println("TotalProtectedCalls="+rs.getTotalProtectedCalls());
    System.out.println("TotalReplayAttempts="+rs.getTotalReplayAttempts());
    System.out.println("TotalRequests="+rs.getTotalRequests());
  }

  public static void main(String[] args) throws Exception {
    PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
    Connection connection1 = null;
    Statement statement1 = null;
    ResultSet resultset1 = null;
    int i = 0;

    //To have date format in English, my Windows desktop being in French 🙂
    Locale.setDefault(new Locale("en"));
    pds.setConnectionFactoryClassName("oracle.jdbc.replay.OracleDataSourceImpl");
    pds.setUser("yjaquier");
    pds.setPassword("secure_password");
    // The RAC connection using SCAN name and HA service
    pds.setURL("jdbc:oracle:thin:@(DESCRIPTION=(RETRY_COUNT=20)(RETRY_DELAY=3)(CONNECT_TIMEOUT=60)(TRANSPORT_CONNECT_TIMEOUT=3000)" + 
        "(ADDRESS=(PROTOCOL=TCP)(HOST=rac-cluster-scan.domain.com)(PORT=1531))(CONNECT_DATA=(SERVICE_NAME=pdb1srv)))");
    pds.setConnectionPoolName("ACPool");

    pds.setMinPoolSize(10);
    pds.setMaxPoolSize(20);
    pds.setInitialPoolSize(10);

    System.out.println("Trying to obtain a new connection from pool ...");
    connection1 = pds.getConnection();
    statement1 = connection1.createStatement();
    resultset1 = statement1.executeQuery("select sys_context('USERENV','SERVER_HOST') from dual");
    while (resultset1.next()) {
      System.out.println("Working on server " + resultset1.getString(1));
    }
    // To start fresh and to avoid
    // ORA-41412: results changed during replay; failover cannot continue
    ((oracle.jdbc.replay.ReplayableConnection) connection1).endRequest(); // Explicit request end
    try {
      ((oracle.jdbc.replay.ReplayableConnection) connection1).beginRequest(); // Explicit request begin
      connection1.setAutoCommit(false);
      statement1 = connection1.createStatement();
      for (i = 1; i<=10; i++) {
        System.out.println("Update "+i+" at "+LocalDateTime.now().format(DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss")));          
        resultset1 = statement1.executeQuery("update test01 set val=val+1 where id = " + i);
      }
      System.out.println("\nJust before the kill...");
      // Sleeping 30 seconds to let me kill the instance where the connection has been borrowed from the pool
      Thread.sleep(30000);
      connection1.commit();
      ((oracle.jdbc.replay.ReplayableConnection) connection1).endRequest(); // Explicit request end
    }
    catch (Exception e)
    {
      //Transaction is not recoverable ?
      System.out.println("Exception detected:");
      e.printStackTrace();
      display_replay_statistics(((oracle.jdbc.replay.ReplayableConnection) connection1).getReplayStatistics(ReplayableConnection.StatisticsReportType.FOR_CURRENT_CONNECTION));
      e = ((SQLException) e).getNextException();
      e.printStackTrace();
      resultset1.close();
      statement1.close();
      connection1.close();
      System.exit(1);
    }
    // Transaction has been recovered
    statement1 = connection1.createStatement();
    resultset1 = statement1.executeQuery("select sys_context('USERENV','SERVER_HOST') from dual");
    while (resultset1.next()) {
      System.out.println("Working on server " + resultset1.getString(1));
    }
    display_replay_statistics(((oracle.jdbc.replay.ReplayableConnection) connection1).getReplayStatistics(ReplayableConnection.StatisticsReportType.FOR_CURRENT_CONNECTION));
    connection1.close();
    System.out.println("Normal exit");
    resultset1.close();
    statement1.close();
    connection1.close();
  }
}

Output display:

ac01
ac01

So we see that the connection I have borrowed from the pool is on server2.domain.com. The kill of this instance is done just before the explicit commit, thanks to the 30 seconds delay I have added. The magic is because the catch block has NOT been executed and the update has been replayed on surviving node i.e. server3.domain.com.

This is double confirmed by a simple database query:

SQL> select * from test01;

        ID        VAL
---------- ----------
         1          2
         2          2
         3          2
         4          2
         5          2
         6          2
         7          2
         8          2
         9          2
        10          2

10 rows selected.

Issues encountered

No more data to read from socket

The complete error message is the following:

java.sql.SQLRecoverableException: No more data to read from socket
at oracle.jdbc.driver.T4CMAREngineNIO.prepareForReading(T4CMAREngineNIO.java:119)
at oracle.jdbc.driver.T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:534)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:485)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
at oracle.jdbc.driver.T4C7Ocommoncall.doOCOMMIT(T4C7Ocommoncall.java:72)
at oracle.jdbc.driver.T4CConnection.doCommit(T4CConnection.java:961)
at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:1937)
at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:1942)
at oracle.jdbc.proxy.oracle$1jdbc$1replay$1driver$1TxnReplayableConnection$2oracle$1jdbc$1internal$1OracleConnection$$$Proxy.commit(Unknown Source)
at ac01.ac01.main(ac01.java:100)

You will find plenty of Oracle notes on MOS about this issue. In my case this was the connection1.setAutoCommit(false); that was set outside of the begin/end request block.

Closed Connection

The complete trace is:

java.sql.SQLException: Closed Statement
	at oracle.jdbc.driver.OracleClosedStatement.executeQuery(OracleClosedStatement.java:2431)
	at oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:366)
	at oracle.jdbc.proxy.oracle$1jdbc$1replay$1driver$1TxnReplayableStatement$2oracle$1jdbc$1internal$1OracleStatement$$$Proxy.executeQuery(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at oracle.jdbc.replay.driver.TxnReplayableBase.replayOneCall(TxnReplayableBase.java:520)
	at oracle.jdbc.replay.driver.TxnFailoverManagerImpl.replayAllBeforeLastCall(TxnFailoverManagerImpl.java:1773)
	at oracle.jdbc.replay.driver.TxnFailoverManagerImpl.handleOutageInternal(TxnFailoverManagerImpl.java:1425)
	at oracle.jdbc.replay.driver.TxnFailoverManagerImpl.handleOutage(TxnFailoverManagerImpl.java:989)
	at oracle.jdbc.replay.driver.TxnReplayableBase.onErrorForAll(TxnReplayableBase.java:339)
	at oracle.jdbc.replay.driver.TxnReplayableConnection.onErrorForAll(TxnReplayableConnection.java:395)
	at oracle.jdbc.replay.driver.TxnReplayableBase.onErrorVoidForAll(TxnReplayableBase.java:262)
	at oracle.jdbc.replay.driver.TxnReplayableConnection.onErrorVoidForAll(TxnReplayableConnection.java:388)
	at oracle.jdbc.proxy.oracle$1jdbc$1replay$1driver$1TxnReplayableConnection$2oracle$1jdbc$1internal$1OracleConnection$$$Proxy.commit(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at oracle.ucp.jdbc.proxy.JDBCConnectionProxyFactory.invoke(JDBCConnectionProxyFactory.java:329)
	at oracle.ucp.jdbc.proxy.ConnectionProxyFactory.invoke(ConnectionProxyFactory.java:50)
	at com.sun.proxy.$Proxy17.commit(Unknown Source)
	at ac01.ac01.main(ac01.java:112)

I had it on top of the "No more data to read from socket" one. I have the feeling that the Java block between beginRequest() and endRequest() must be re-runnable and so I have been obliged to add in this block (even if set just above):

statement1 = connection1.createStatement();

ORA-41412: results changed during replay; failover cannot continue

This one was the trickiest I had ! To see it I have even been obliged to update my Java code by adding in catch part:

e = ((SQLException) e).getNextException();
e.printStackTrace();

the full trace is:

java.sql.SQLException: ORA-41412: results changed during replay; failover cannot continue
	at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
	at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:446)
	at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1054)
	at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
	at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
	at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
	at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:213)
	at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:37)
	at oracle.jdbc.driver.T4CStatement.executeForDescribe(T4CStatement.java:733)
	at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:904)
	at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1082)
	at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1276)
	at oracle.jdbc.driver.OracleStatementWrapper.executeQuery(OracleStatementWrapper.java:366)
	at oracle.jdbc.proxy.oracle$1jdbc$1replay$1driver$1TxnReplayableStatement$2oracle$1jdbc$1internal$1OracleStatement$$$Proxy.executeQuery(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at oracle.jdbc.replay.driver.TxnReplayableBase.replayOneCall(TxnReplayableBase.java:520)
	at oracle.jdbc.replay.driver.TxnFailoverManagerImpl.replayAllBeforeLastCall(TxnFailoverManagerImpl.java:1773)
	at oracle.jdbc.replay.driver.TxnFailoverManagerImpl.handleOutageInternal(TxnFailoverManagerImpl.java:1425)
	at oracle.jdbc.replay.driver.TxnFailoverManagerImpl.handleOutage(TxnFailoverManagerImpl.java:989)
	at oracle.jdbc.replay.driver.TxnReplayableBase.onErrorForAll(TxnReplayableBase.java:339)
	at oracle.jdbc.replay.driver.TxnReplayableConnection.onErrorForAll(TxnReplayableConnection.java:395)
	at oracle.jdbc.replay.driver.TxnReplayableBase.onErrorVoidForAll(TxnReplayableBase.java:262)
	at oracle.jdbc.replay.driver.TxnReplayableConnection.onErrorVoidForAll(TxnReplayableConnection.java:388)
	at oracle.jdbc.proxy.oracle$1jdbc$1replay$1driver$1TxnReplayableConnection$2oracle$1jdbc$1internal$1OracleConnection$$$Proxy.commit(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at oracle.ucp.jdbc.proxy.JDBCConnectionProxyFactory.invoke(JDBCConnectionProxyFactory.java:329)
	at oracle.ucp.jdbc.proxy.ConnectionProxyFactory.invoke(ConnectionProxyFactory.java:50)
	at com.sun.proxy.$Proxy17.commit(Unknown Source)
	at ac01.ac01.main(ac01.java:127)
Caused by: Error : 41412, Position : 49, Sql = select sys_context('USERENV','SERVER_HOST') from dual, OriginalSql = 
select sys_context('USERENV','SERVER_HOST') from dual, Error Msg = ORA-41412: results changed during replay; failover cannot continue

	at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:498)
	... 34 more

Of course the query result returning the server name on which I am connected has changed ! I have killed the initial one ! Even the Oracle example you can found on MOS are displaying the working instance so really strange to me to hit this issue. To solve it I have added the endRequest() call just before the block I am expecting to be replayable:

((oracle.jdbc.replay.ReplayableConnection) connection1).endRequest();

Diagnostics and Tracing

I have not been able to activate tracing of replay events either on console or in a log file. The documentation says you can use:

oracle.jdbc.internal.replay.level = FINER|FINEST

And use for example, to log on console:

oracle.jdbc.internal.replay.level = FINER
handlers = java.util.logging.ConsoleHandler
java.util.logging.ConsoleHandler.level = ALL
java.util.logging.ConsoleHandler.formatter = java.util.logging.XMLFormatter

But it does not display anything for me...

References

  • Application Continuity for Java
  • "java.sql.SQLRecoverableException: No more data to read from socket" Using Application Continuity with 12c JDBC Driver (Doc ID 2140406.1)
  • How To Test Application Continuity Using A Standalone Java Program (Doc ID 1602233.1)
  • Application Continuity Throws Exception No more data to read from socket For Commits After Failover (Doc ID 2197029.1)
  • Java Client Failover With Application Continuity Fails With java.sql.SQLRecoverableException: No more data to read from socket (Doc ID 2294143.1)
  • Configuring Oracle Universal Connection Pool (UCP) for Planned Outages (Doc ID 1593786.1)
  • oracle.jdbc.replay Interface ReplayStatistics
  • Package oracle.jdbc.replay
  • Application Continuity

The post Application Continuity (AC) for Java – JDBC HA – part 6 appeared first on IT World.

]]>
https://blog.yannickjaquier.com/oracle/application-continuity-ac-jdbc-ha-part-6.html/feed 1
Transaction Guard (TG) for Java – JDBC HA – part 5 https://blog.yannickjaquier.com/oracle/transaction-guard-tg-jdbc-ha-part-5.html https://blog.yannickjaquier.com/oracle/transaction-guard-tg-jdbc-ha-part-5.html#comments Mon, 27 Aug 2018 10:09:58 +0000 https://blog.yannickjaquier.com/?p=4315 Preamble Transaction Guard (TG) is a database feature for applications to ensure that every transaction has been executed at most once in case of planned or unplanned outage. In the background each transaction is associated with a logical transaction ID that let you determine, afterward, if the transaction has committed and has been completed. Without […]

The post Transaction Guard (TG) for Java – JDBC HA – part 5 appeared first on IT World.

]]>

Table of contents

Preamble

Transaction Guard (TG) is a database feature for applications to ensure that every transaction has been executed at most once in case of planned or unplanned outage. In the background each transaction is associated with a logical transaction ID that let you determine, afterward, if the transaction has committed and has been completed. Without Transaction Guard (TG) the typical example that is given is a transaction that is increasing by 5% the salary of all company employees. If the application fail, outside of the fact that you will make happy employees by giving them 10.25% you cannot rerun blindly the transaction from scratch. With Transaction Guard you can know from application layout how it went and present a warning or gray the submit button to avoid a second dramatic rerun.

As we have seen in part 1 you must use JDBC Thin driver to use this feature !

For testing I have created below test table:

SQL> create table test01 as select level as id, 1 as val from dual connect by level <= 10;

Table created.

SQL> desc test01;
 Name                                      Null?    Type
 ----------------------------------------- -------- ----------------------------
 ID                                                 NUMBER
 VAL                                                NUMBER

SQL> select * from test01;

        ID        VAL
---------- ----------
         1          1
         2          1
         3          1
         4          1
         5          1
         6          1
         7          1
         8          1
         9          1
        10          1

10 rows selected.

The idea is to update each row using id column as a key and pause between the updates to let me kill the session and simulate an crash scenario. By default the Java connection has autocommit property set to TRUE

From Oracle Technology Network the requirements for TG are:

  • Use Oracle Database Release 12.1 or later.
  • Use an application service for all database work. Create the service using srvctl if using RAC or DBMS_SERVICE if not using RAC. You may also use GDSCTL.
  • Set the following properties on the service – COMMIT_OUTCOME = TRUE for Transaction Guard.
  • Grant execute permission on DBMS_APP_CONT package to the application user.
  • Increase DDL_LOCK_TIMEOUT if using Transaction Guard with DDL statements (for example, 10 seconds).

In part 1 of this series I have chosen only few options for my service and so I do not satisfy the minimum requirements for TG. I had to drop and recreate the service using the mandatory option. See issues encountered section for this.

The Oracle PL/SQL supplied package to know if a transaction has committed and has completed is DBMS_APP_CONT.GET_LTXID_OUTCOME:

SQL> desc DBMS_APP_CONT
PROCEDURE GET_LTXID_OUTCOME
 Argument Name                  Type                    In/Out Default?
 ------------------------------ ----------------------- ------ --------
 CLIENT_LTXID                   RAW                     IN
 COMMITTED                      BOOLEAN                 OUT
 USER_CALL_COMPLETED            BOOLEAN                 OUT

This procedure has two Boolean out parameters and until recently it was not possible to map a PL/SQL Boolean in Java this is why all blog post (as well as official 12cR2 documentation) I have found, at the time of writing this post, are wrapping this procedure in a PL/SQL block to return numbers instead of Boolean. The limitation has gone with 12cR2:

JDBC Support for Binding PL/SQL BOOLEAN type

Starting from Oracle Database 12c Release 2 (12.2.0.1), Oracle JDBC drivers support binding PL/SQL BOOLEAN type, which is a true BOOLEAN type. PLSQL_BOOLEAN binds BOOLEAN type for input or output parameters when executing a PL/SQL function or procedure. With this feature, now JDBC supports the ability to bind PLSQL_BOOLEAN type into any PL/SQL block from Java.

Transaction Guard (TG) testing

Java testing code:

package tg01;

import java.sql.CallableStatement;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.SQLRecoverableException;
import java.sql.Statement;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;

import oracle.jdbc.LogicalTransactionId;
import oracle.jdbc.OracleConnection;
import oracle.jdbc.OracleTypes;
import oracle.jdbc.pool.OracleDataSource;

public class tg01 {
  private static Boolean[] dbms_app_cont(Connection connection, LogicalTransactionId ltxid) throws SQLException {
    Boolean[] result;
    String query = "begin dbms_app_cont.get_ltxid_outcome(?, ?, ?); end;";
    CallableStatement callStmt1 = null;

    result = new Boolean[2];
    callStmt1 = connection.prepareCall(query);
    callStmt1.setObject(1, ltxid);
    callStmt1.registerOutParameter(2, OracleTypes.PLSQL_BOOLEAN);
    callStmt1.registerOutParameter(3, OracleTypes.PLSQL_BOOLEAN);
    callStmt1.execute();

    result[0] = callStmt1.getBoolean(2);
    result[1] = callStmt1.getBoolean(3);
    return result;
  }

  public static void main(String[] args) throws Exception {
    Connection connection1=null, connection2 = null;
    Statement statement1 = null;
    ResultSet resultset1 = null;
    OracleDataSource ods1 = new OracleDataSource();
    int i = 1;
    LogicalTransactionId ltxid = null;
    Boolean[] ltxid_result;

    ltxid_result = new Boolean[2];

    try {
      Class.forName("oracle.jdbc.driver.OracleDriver");
    }
    catch (ClassNotFoundException e) {
      System.out.println("Where is your Oracle JDBC Thin driver ?");
      e.printStackTrace();
      System.exit(1);
    }

    System.out.println("Oracle JDBC Driver Registered!");

    try {
      ods1.setUser("yjaquier");
      ods1.setPassword("secure_password");
      ods1.setURL("jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=rac-cluster-scan.domain.com)(PORT=1531))(CONNECT_DATA=(SERVICE_NAME=pdb1srv)))");
      connection1 = ods1.getConnection();
    }
    catch (SQLException e) {
      System.out.println("Connection Failed! Check output console");
      e.printStackTrace();
    }
    System.out.println("Connected to Oracle database...");
    statement1 = connection1.createStatement();
    resultset1 = statement1.executeQuery("select sys_context('USERENV','SERVER_HOST') from dual");
    while (resultset1.next()) {
      System.out.println("Working on server " + resultset1.getString(1));
    }

    try {
      // lxtid must be taken "at right time"
      // If you take it after last update you might have committed returned value
      // equal to false. Means that all previous updates are not taken into account
      ltxid = ((OracleConnection)connection1).getLogicalTransactionId();
      for (i=1; i<=10; i++) {
        System.out.println("\nUpdate "+i+" at "+LocalDateTime.now().format(DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss")));          
        resultset1 = statement1.executeQuery("update test01 set val=val + 1 where id = " + i);
        // Sleeping 10 seconds to let me kill the session
        Thread.sleep(10000);
      }
    }
    catch (SQLRecoverableException recoverableException) {
      resultset1.close();
      connection1.close();
      try {
        connection2 = ods1.getConnection();
      }
      catch (SQLException e) {
        System.out.println("Connection Failed! Check output console");
        e.printStackTrace();
      }
      ltxid_result = dbms_app_cont(connection2, ltxid);
      System.out.println("Committed: " +  ltxid_result[0] + ", user_call_completed: " + ltxid_result[1]);
      connection1 = connection2;
    }
  }
}

I start the execution of my Java program (only ojdbc8.jar must be added to your project) and after 4th update I kill the Oracle session:

tg01
tg01

My Java program tells us that the transaction has committed and that the user call has not completed.

If we check at database level:

SQL> select * from test01;

        ID        VAL
---------- ----------
         1          2
         2          2
         3          2
         4          1
         5          1
         6          1
         7          1
         8          1
         9          1
        10          1

10 rows selected.

As we see on above picture DBMS_APP_CONT.GET_LTXID_OUTCOME procedure tells us that committed has been done and user call has not completed. This is obviously confirmed by the state of my test01 table. We clearly understand that the program cannot be re-submitted blindly and a bit of work is mandatory to recover the situation...

As a side note the transaction has committed because the auto commit feature of the connection is TRUE by default, you can deactivate this with:

connection1.setAutoCommit(false);

And use an explicit commit at the end of the ten updates with:

connection1.commit();

If you kill the session after the ten updates and just before the commit (slight modification of above Java code) the Java program will display:>/p>

Committed: false, user_call_completed: false

In this situation you know the table is unchanged and that you can safely resubmit he operation...

Issues encountered

In my initial testing I received an error message when calling DBMS_APP_CONT.GET_LTXID_OUTCOME procedure:

ORA-14903: Corrupt logical transaction detected.
ORA-06512: at "SYS.DBMS_APP_CONT", line 20
ORA-06512: at "SYS.DBMS_APP_CONT", line 71
ORA-06512: at line 1

This was simply coming, even if clearly written in documentation, from my service that was not satisfying the minimum requirement. So I had to drop and recreate my pdb1srv service that we created in part 1 of this series using the mandatory commit_outcome parameter (retention value I specify is in fact default value, 24 hours):

[oracle@server2 ~]$ srvctl stop service -d orcl -service pdb1srv
[oracle@server2 ~]$ srvctl remove service -d orcl -service pdb1srv
[oracle@server2 ~]$ srvctl add service -db orcl -pdb pdb1 -service pdb1srv -notification TRUE -serverpool server_pool01 \
                    -failovertype TRANSACTION -commit_outcome TRUE -retention 86400
[oracle@server2 ~]$ srvctl start service -db orcl -service pdb1srv
[oracle@server2 ~]$ crsctl stat res ora.orcl.pdb1srv.svc -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.orcl.pdb1srv.svc
      1        ONLINE  ONLINE       server2                  STABLE
      2        ONLINE  ONLINE       server3                  STABLE
--------------------------------------------------------------------------------

If later on you do not recall what you have configured for your service:

[oracle@server2 ~]$ srvctl config service -db orcl -service pdb1srv
Service name: pdb1srv
Server pool: server_pool01
Cardinality: UNIFORM
Service role: PRIMARY
Management policy: AUTOMATIC
DTP transaction: false
AQ HA notifications: true
Global: false
Commit Outcome: true
Failover type: TRANSACTION
Failover method:
TAF failover retries:
TAF failover delay:
Failover restore: NONE
Connection Load Balancing Goal: LONG
Runtime Load Balancing Goal: NONE
TAF policy specification: NONE
Edition:
Pluggable database name: pdb1
Maximum lag time: ANY
SQL Translation Profile:
Retention: 86400 seconds
Replay Initiation Time: 300 seconds
Drain timeout:
Stop option:
Session State Consistency: DYNAMIC
GSM Flags: 0
Service is enabled
Service is individually enabled on nodes:
Service is individually disabled on nodes:
CSS critical: no

References

The post Transaction Guard (TG) for Java – JDBC HA – part 5 appeared first on IT World.

]]>
https://blog.yannickjaquier.com/oracle/transaction-guard-tg-jdbc-ha-part-5.html/feed 1
Fast Connection Failover (FCF) – JDBC HA – part 4 https://blog.yannickjaquier.com/oracle/fast-connection-failover-fcf-jdbc-ha-part-4.html https://blog.yannickjaquier.com/oracle/fast-connection-failover-fcf-jdbc-ha-part-4.html#comments Tue, 07 Aug 2018 14:29:41 +0000 https://blog.yannickjaquier.com/?p=4293 Preamble Fast Connection Failover (FCF) is almost the same as previous testing except that here the FAN feature is provided by the connection pool directly and so you do not require the simplefan.jar class anymore. Instead we are going to use the Universal Connection Pool (UCP) class implemented in ucp.jar file. Fast Connection Failover (FCF) […]

The post Fast Connection Failover (FCF) – JDBC HA – part 4 appeared first on IT World.

]]>

Table of contents

Preamble

Fast Connection Failover (FCF) is almost the same as previous testing except that here the FAN feature is provided by the connection pool directly and so you do not require the simplefan.jar class anymore. Instead we are going to use the Universal Connection Pool (UCP) class implemented in ucp.jar file.

Fast Connection Failover (FCF) testing

Universal Connection Pool (UCP) for JDBC

If you dig a bit in Oracle documentation you can find below statement:

Starting from Oracle Database 11g Release 2 (11.2), implicit connection pool has been deprecated, and replaced with Universal Connection Pool (UCP) for JDBC. Oracle recommends that you take advantage of the new architecture, which is more powerful and offers better performance.

So obviously I will use UCP in my Fast Connection Failover (FCF) testing !

The complete reference of Java class is available at Oracle® Universal Connection Pool for JDBC Java API Reference.

You have also an Oracle UCP FAQ but at the time of writing this blog post the page is full of bugs with missing links…

As we have seen with FAN you need a service and ONS up and running on your RAC cluster !

Java testing code

In below example code FCF is activated using this command:

pds.setFastConnectionFailoverEnabled(true);

I am also logging in a separate log file all the UCP events like initialization, connections closed, connections borrowed and so on. This is done in below part of the script:

// Logging UCP events in a log file
Handler fh = new FileHandler("fcf01.log");
// Finally the UCPFormatter is not providing a so nice display
//fh.setFormatter(new UCPFormatter());
fh.setFormatter(new SimpleFormatter());
UniversalConnectionPoolManager mgr = UniversalConnectionPoolManagerImpl.getUniversalConnectionPoolManager();
mgr.setLogLevel(Level.FINE);
Logger log = Logger.getLogger("oracle.ucp");
log.setLevel(Level.FINE);
log.addHandler(fh);
// To avoid display on screen
log.setUseParentHandlers(false);

The Oracle Notification Service (ONS) part is a bit cumbersome in Oracle documentation. The easiest way to implement it is to rely on what they call automatic ONS and register to the service managed by Grid infrastructure. Investigating further I have seen that in my case (newest releases of product) it’s not even more mandatory to register with setONSConfiguration method:

For standalone Java applications, you must configure ONS using the setONSConfiguration method. However, if your application meets the following requirements, then you no longer need to call the setONSConfiguration method for enabling FCF:

  • Your application is using Oracle Database 12c Release 1 (12.1) UCP and Oracle RAC Database 12c Release 1 (12.1)
  • Your application does not require ONS wallet or keystore

Java testing code

Finally the complete code with as much comment as I can to explain each part. You need to add to your project (classpath) ojdbc8.jar, ons.jar and ucp.jar.

package fcf01;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Locale;
import java.util.logging.FileHandler;
import java.util.logging.Handler;
import java.util.logging.Level;
import java.util.logging.Logger;
import java.util.logging.SimpleFormatter;
import oracle.ucp.admin.UniversalConnectionPoolManager;
import oracle.ucp.admin.UniversalConnectionPoolManagerImpl;
import oracle.ucp.jdbc.JDBCConnectionPoolStatistics;
import oracle.ucp.jdbc.PoolDataSource;
import oracle.ucp.jdbc.PoolDataSourceFactory;
import oracle.ucp.jdbc.ValidConnection;

public class fcf01 {

  // To restrict display versus System.out.println(stats);
  // String fcfInfo = (oracleJDBCConnectionPool)stats.getFCFProcessingInfo(); NOT WORKING at all !!
  private static void display_statistics(JDBCConnectionPoolStatistics stats) {
    System.out.println("AbandonedConnectionsCount: " + stats.getAbandonedConnectionsCount());
    System.out.println("AvailableConnectionsCount: " + stats.getAvailableConnectionsCount());
    System.out.println("AverageBorrowedConnectionsCount: " + stats.getAverageBorrowedConnectionsCount());
    System.out.println("AverageConnectionWaitTime: " + stats.getAverageConnectionWaitTime());
    System.out.println("BorrowedConnectionsCount: " + stats.getBorrowedConnectionsCount());
    System.out.println("ConnectionsClosedCount: " + stats.getConnectionsClosedCount());
    System.out.println("ConnectionsCreatedCount: " + stats.getConnectionsCreatedCount());
    System.out.println("RemainingPoolCapacityCount: " + stats.getRemainingPoolCapacityCount());
    System.out.println("TotalConnectionsCount: " + stats.getTotalConnectionsCount());
    System.out.println("getPeakConnectionsCount: " + stats.getPeakConnectionsCount());
    System.out.println("getPendingRequestsCount: " + stats.getPendingRequestsCount());
    System.out.println("getRemainingPoolCapacityCount: " + stats.getRemainingPoolCapacityCount());
  }

  public static void main(String[] args) throws Exception {
    PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
    Connection connection1 = null;
    Statement statement1 = null;
    ResultSet resultset1 = null;

    // To have date format in English, my Windows desktop being in French 🙂
    Locale.setDefault(new Locale("en"));
    pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
    pds.setUser("yjaquier");
    pds.setPassword("secure_password");
    // The RAC connection using SCAN name and HA service
    pds.setURL("jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=rac-cluster-scan.domain.com)(PORT=1531))(CONNECT_DATA=(SERVICE_NAME=pdb1srv)))");
    pds.setConnectionPoolName("FCFPool");

    // Automatic ONS
    // pds.setONSConfiguration("nodes=server2.domain.com:6200,server3.domain.com:6200");
    // Pool sizing
    pds.setMinPoolSize(10);
    pds.setMaxPoolSize(20);
    pds.setInitialPoolSize(10);
    // Enable Fast Connection Failover
    pds.setFastConnectionFailoverEnabled(true);

    // Simple check to demonstrate one cool method
    System.out.println("FCF activated ?: " + pds.getFastConnectionFailoverEnabled());

    // Logging UCP events in a log file
    Handler fh = new FileHandler("fcf01.log");
    // Finally the UCPFormatter is not providing a so nice display
    //fh.setFormatter(new UCPFormatter());
    fh.setFormatter(new SimpleFormatter());
    UniversalConnectionPoolManager mgr = UniversalConnectionPoolManagerImpl.getUniversalConnectionPoolManager();
    mgr.setLogLevel(Level.FINE);
    Logger log = Logger.getLogger("oracle.ucp");
    log.setLevel(Level.FINE);
    log.addHandler(fh);
    // To avoid display on screen
    log.setUseParentHandlers(false);

    while (true)
    {
      try
      {
        System.out.println("Trying to obtain a new connection from pool ...");
        connection1 = pds.getConnection();
        System.out.println("Number of borrowed connections from the pool: " + pds.getBorrowedConnectionsCount());
        statement1 = connection1.createStatement();
        // The infinite loop awaiting RAC node down events
        while (true)
        {
          System.out.println("\nPool status at "+LocalDateTime.now().format(DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss"))+": ");
          JDBCConnectionPoolStatistics stats = pds.getStatistics();
          display_statistics(stats);
          resultset1 = statement1.executeQuery("select sys_context('USERENV','SERVER_HOST') from dual");
          while (resultset1.next()) {
            System.out.println("\nWorking on " + resultset1.getString(1));
          }
          resultset1 = statement1.executeQuery("select gvi.host_name,count(*) from gv$session gvs,  gv$instance gvi where gvs.inst_id=gvi.inst_id "
              + "and gvs.username='YJAQUIER' and program='JDBC Thin Client' group by gvi.host_name order by gvi.host_name");
          while (resultset1.next()) {
            System.out.println(resultset1.getString(1) + ": " + resultset1.getString(2));
          }
          Thread.sleep(2000);
          resultset1.close();
        }
      }
      catch (SQLException sqlexc)
      {
        System.out.println("SQLException detected ...");
        // Recommended method to check if a borrowed connection is still usable after an SQL exception
        if (connection1 == null || !((ValidConnection) connection1).isValid())
        {
          System.out.println("Connection retry necessary ...");
          try
          {
            connection1.close();
          }
          catch (Exception closeExc)
          {
            System.out.println("Exception detected when closing connection:");
            closeExc.printStackTrace();
          }
        }
      }
    }
  }
}

While the Java application is running I have directly killed (kill command under Linux) the instance on where the connection has been borrowed.

When the application start I have an almost equally distributed pool:

fcf01
fcf01

After I have killed the instance where the connection was borrowed we need to borrow a new one:

fcf02
fcf02

Once a new connection from pool has been borrowed we can continue working, with no application failure by the way:

fcf03
fcf03

In meanwhile the pool is growing to reach requested minimum on surviving instance:

fcf04
fcf04

Once the killed instance as restarted (automatically managed by Grid infrastructure) the pool is rebalanced across all the instances:

fcf05
fcf05

The Java application log file I have generated is confirming what we have graphically seen:

Feb 28, 2018 12:28:46 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.admin.UniversalConnectionPoolManagerMBeanImpl:getUniversalConnectionPoolManagerMBean::Universal Connection Pool Manager MBean created
Feb 28, 2018 12:28:47 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.jdbc.PoolDataSourceImpl:createPoolWithDefaultProperties:oracle.ucp.jdbc.PoolDataSourceImpl@591f989e:Connection pool instance is created with default properties
Feb 28, 2018 12:28:47 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.jdbc.PoolDataSourceImpl:createPool:oracle.ucp.jdbc.PoolDataSourceImpl@591f989e:Connection pool instance is created
Feb 28, 2018 12:28:48 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Topology:enableFANHeuristically:oracle.ucp.common.UniversalConnectionPoolBase$4@5c671d7f:Heuristically determine whether to enable FAN
Feb 28, 2018 12:28:48 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Topology:enableFANHeuristically:oracle.ucp.common.UniversalConnectionPoolBase$4@5c671d7f:RAC/GDS 12.x, FAN is heuristically enabled
Feb 28, 2018 12:28:51 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.UniversalConnectionPoolBase:start:oracle.ucp.jdbc.oracle.OracleConnectionConnectionPool@5f2050f6:pool started
Feb 28, 2018 12:28:51 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.jdbc.PoolDataSourceImpl:startPool:oracle.ucp.jdbc.PoolDataSourceImpl@591f989e:connection pool is started
Feb 28, 2018 12:29:25 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:growing...
Feb 28, 2018 12:29:29 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.FailoverDriver$1:handleNotifications:oracle.ucp.common.FailoverDriver$1@5cada3dd:event processed, snapshot:[orcl_1,db=orcl,service=pdb1srv,host=server2:(activeCount:4,borrowedCount:1,active:true,aff=true,violating=false,id=1), orcl_2,db=orcl,service=pdb1srv,host=server3:(activeCount:0,borrowedCount:0,active:false,aff=true,violating=false,id=0)]
Feb 28, 2018 12:29:29 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.FailoverDriver$1:handleNotifications:oracle.ucp.common.FailoverDriver$1@5cada3dd:event processed, snapshot:[orcl_1,db=orcl,service=pdb1srv,host=server2:(activeCount:4,borrowedCount:1,active:true,aff=true,violating=false,id=1), orcl_2,db=orcl,service=pdb1srv,host=server3:(activeCount:0,borrowedCount:0,active:false,aff=true,violating=false,id=0)]
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:grew up 1 connection to reach the minimum
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:growing...
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:grew up 1 connection to reach the minimum
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:growing...
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:grew up 1 connection to reach the minimum
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:growing...
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:grew up 1 connection to reach the minimum
Feb 28, 2018 12:30:07 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:growing...
Feb 28, 2018 12:30:09 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:grew up 1 connection to reach the minimum
Feb 28, 2018 12:30:09 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:growing...
Feb 28, 2018 12:30:09 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:grew up 1 connection to reach the minimum
Feb 28, 2018 12:31:12 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:growing...
Feb 28, 2018 12:31:15 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.Core:adjustMinLimit:oracle.ucp.common.Core@34f7cfd9:grew up 1 connection to reach the minimum
Feb 28, 2018 12:31:23 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.FailoverDriver$1:handleNotifications:oracle.ucp.common.FailoverDriver$1@5cada3dd:event processed, snapshot:[orcl_1,db=orcl,service=pdb1srv,host=server2:(activeCount:5,borrowedCount:1,active:true,aff=true,violating=false,id=1), orcl_2,db=orcl,service=pdb1srv,host=server3:(activeCount:6,borrowedCount:0,active:true,aff=true,violating=false,id=0)]
Feb 28, 2018 12:31:23 PM oracle.ucp.logging.ClioSupport _log
FINE: oracle.ucp.common.FailoverDriver$1:handleNotifications:oracle.ucp.common.FailoverDriver$1@5cada3dd:event processed, snapshot:[orcl_1,db=orcl,service=pdb1srv,host=server2:(activeCount:5,borrowedCount:1,active:true,aff=true,violating=false,id=1), orcl_2,db=orcl,service=pdb1srv,host=server3:(activeCount:6,borrowedCount:0,active:true,aff=true,violating=false,id=0)]

References

The post Fast Connection Failover (FCF) – JDBC HA – part 4 appeared first on IT World.

]]>
https://blog.yannickjaquier.com/oracle/fast-connection-failover-fcf-jdbc-ha-part-4.html/feed 1
Fast Application Notification (FAN) – JDBC HA – part 3 https://blog.yannickjaquier.com/oracle/fast-application-notification-fan-jdbc-ha-part-3.html https://blog.yannickjaquier.com/oracle/fast-application-notification-fan-jdbc-ha-part-3.html#comments Fri, 27 Jul 2018 13:37:33 +0000 https://blog.yannickjaquier.com/?p=4288 Preamble Fast Application Notification (FAN) require you build a Real Application Cluster (RAC) environment. You cannot use the classical Operating System cluster active/passive. So a bit more work… Fast Application Notification (FAN) testing Even if I have found this in Oracle official documentation: Although the Oracle JDBC drivers now support the FAN events, Oracle UCP […]

The post Fast Application Notification (FAN) – JDBC HA – part 3 appeared first on IT World.

]]>

Table of contents

Preamble

Fast Application Notification (FAN) require you build a Real Application Cluster (RAC) environment. You cannot use the classical Operating System cluster active/passive. So a bit more work…

Fast Application Notification (FAN) testing

Even if I have found this in Oracle official documentation:

Although the Oracle JDBC drivers now support the FAN events, Oracle UCP provides more comprehensive support for all FAN events.

I have always wondered the usage of this simplefan.jar file available for download on Java download page. So testing JDBC with this JAR file should be the most basic FAN testing.

The complete reference of the API is available at Oracle® Database RAC FAN Events Java API Reference 12c Release 2 (12.2).

Service setup

I have struggled a lot trying to use the internal service that is created with every pluggable database (even if you do not have the multitenant paid option and so a single pluggable database). This is simply not working and you must create a new one with the Grid infrastructure part of your RAC installation.

This has triggered a new problem when I have stupidly created this service with the same name as the internal one (pdb1), unbelievable that there is no control on this please see issues encountered section to overcome the situation if you face it.

I have chosen to create my RAC cluster with now to be used server pool policy so creating a FAN aware service with (-pdb to specify the pluggable database to be used or you end up in container one and obviously name MUSt NOT be pdb1 so using pdb1srv):

[oracle@server3 ~]$ srvctl add service -db orcl -pdb pdb1 -service pdb1srv -notification TRUE -serverpool server_pool01 -failovertype SELECT
[oracle@server2 ~]$ crsctl stat res ora.orcl.pdb1srv.svc -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.orcl.pdb1srv.svc
      1        OFFLINE OFFLINE                               STABLE
      2        OFFLINE OFFLINE                               STABLE
--------------------------------------------------------------------------------
[oracle@server2 ~]$ srvctl start service -db orcl -service pdb1srv
[oracle@server2 ~]$ crsctl stat res ora.orcl.pdb1srv.svc -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.orcl.pdb1srv.svc
      1        ONLINE  ONLINE       server2                  STABLE
      2        ONLINE  ONLINE       server3                  STABLE
--------------------------------------------------------------------------------

You can also check to any SCAN listener that it is taken into account:

[oracle@server2 ~]$ lsnrctl status listener_scan1

LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 22-FEB-2018 11:29:44

Copyright (c) 1991, 2016, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER_SCAN1)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER_SCAN1
Version                   TNSLSNR for Linux: Version 12.2.0.1.0 - Production
Start Date                20-FEB-2018 12:57:51
Uptime                    1 days 22 hr. 31 min. 54 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/12.2.0/grid/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/server2/listener_scan1/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.56.121)(PORT=1531)))
Services Summary...
Service "-MGMTDBXDB" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "65188d13fadb4a7be0536638a8c0aa34" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "651a2f85336128cde0536638a8c0ff54" has 2 instance(s).
  Instance "orcl_1", status READY, has 1 handler(s) for this service...
  Instance "orcl_2", status READY, has 1 handler(s) for this service...
Service "_mgmtdb" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "gimr_dscrep_10" has 1 instance(s).
  Instance "-MGMTDB", status READY, has 1 handler(s) for this service...
Service "orcl" has 2 instance(s).
  Instance "orcl_1", status READY, has 1 handler(s) for this service...
  Instance "orcl_2", status READY, has 1 handler(s) for this service...
Service "orclXDB" has 2 instance(s).
  Instance "orcl_1", status READY, has 1 handler(s) for this service...
  Instance "orcl_2", status READY, has 1 handler(s) for this service...
Service "pdb1" has 2 instance(s).
  Instance "orcl_1", status READY, has 1 handler(s) for this service...
  Instance "orcl_2", status READY, has 1 handler(s) for this service...
Service "pdb1srv" has 2 instance(s).
  Instance "orcl_1", status READY, has 1 handler(s) for this service...
  Instance "orcl_2", status READY, has 1 handler(s) for this service...
The command completed successfully

If you like to drop and recreate a new one with different option use:

[oracle@server2 ~]$ srvctl stop service -db orcl -service pdb1srv
[oracle@server3 ~]$ srvctl remove service -db orcl -service pdb1srv

Oracle Notification Service (ONS) setup

One of the prerequisite of FAN is Oracle Notification Service (ONS), the documentation says you must have it on node where JDBC is running but you can have it on each node of your RAC cluster using a feature called auto-ONS. Luckily with my fresh 12cR2 installation ONS is already configured and running (I just had to correct nodes value on second node where first server was missing. Don’t mess up with all config files located on $GRID_HOME/opmn/conf directory the one to use is the one displayed in ONS log file ($GRID_HOME/opmn/logs/ons.log.server2 for me). Means the file to modify for me IS NOT ons.config but ons.config.server2:

[2018-02-23T17:49:42+01:00] [ons] [TRACE:32] [] [ons-local] Reloading by request
[2018-02-23T17:49:45+01:00] [ons] [TRACE:32] [] [ons-local] Config file: /u01/app/12.2.0/grid/opmn/conf/ons.config.server2

After correction I end up with a file like:

[oracle@server2 ~]$ cat $ORACLE_HOME/opmn/conf/ons.config.server2
usesharedinstall=true
allowgroup=true
localport=6100          # line added by Agent
remoteport=6200         # line added by Agent
nodes=server2.domain.com:6200,server3.domain.com:6200
walletfile=/u01/app/grid/crsdata/server2/onswallet/             # line added by Agent
allowunsecuresubscriber=true            # line added by Agent

Check all is running fine with:

[oracle@server2 ~]$ onsctl ping
ons is running ...
[oracle@server2 ~]$ onsctl debug
[2018-02-28T12:31:12+01:00] [ons] [ERROR:1] [] [ons-local] /u01/app/12.2.0/grid/opmn/conf/ons.config.server2: 5: (warning) unkown key: ocrnodename
[2018-02-28T12:31:12+01:00] [ons] [ERROR:1] [] [ons-local] /u01/app/12.2.0/grid/opmn/conf/ons.config.server2: 5: (warning) unkown key: ocrnodename
HTTP/1.1 200 OK
Connection: close
Content-Type: text/html
Response:

== server2.domain.com:6200 9006 18/02/28 12:31:11 ==
Build: ONS_12.2.0.1.0_LINUX.X64_161121.1010 2016/11/21 19:53:59 UTC
Home: /u01/app/12.2.0/grid

======== ONS ========

           IP ADDRESS                   PORT    TIME   SEQUENCE  FLAGS
--------------------------------------- ----- -------- -------- --------
                         192.168.56.102  6200 5a9530ac 0000000d 00000008

Listener:

  TYPE                BIND ADDRESS               PORT  SOCKET
-------- --------------------------------------- ----- ------
Local                                        ::1  6100      6
Local                                  127.0.0.1  6100      7
Remote                                       any  6200      8
Remote                                       any  6200      -

Servers: (1)

            INSTANCE NAME                  TIME   SEQUENCE  FLAGS     DEFER
---------------------------------------- -------- -------- -------- ----------
dbInstance_server3.domain.com_6200       5a953161 00000003 00000002          0
                          192.168.56.103 6200

Connection Topology: (2)

                IP                      PORT   VERS  TIME
--------------------------------------- ----- ----- --------
                         192.168.56.103  6200     4 5a953161
                           **                          192.168.56.102 6200
                         192.168.56.102  6200     4 5a9530ac=
                           **                          192.168.56.103 6200

Server connections: (1)

   ID            CONNECTION ADDRESS              PORT  FLAGS  SNDQ REF PHA ACK
-------- --------------------------------------- ----- ------ ---- --- --- ---
       d                          192.168.56.103  6200 2004a6    0   1  IO   0

Client connections: (10)

   ID            CONNECTION ADDRESS              PORT  FLAGS  SNDQ REF PHA SUB
-------- --------------------------------------- ----- ------ ---- --- --- ---
       0                                internal     0 00044a    0   1  IO   1
       2                                     ::1 56365 20041a    0   1  IO   1
       3                                     ::1 56621 20041a    0   1  IO   1
       4                                     ::1 56877 20041a    0   1  IO   1
       5                                     ::1 57133 20041a    0   1  IO   1
       9                                     ::1 59949 20041a    0   1  IO   0
       a                                     ::1 60717 20041a    0   1  IO   1
     5f2                     ::ffff:192.168.56.1  1237 26042a    0   1  IO   2
     5f1                     ::ffff:192.168.56.1   725 26042a    0   1  IO   2
 request                                     ::1 29026 200e1a    0   1  IO   0

Events:

  Flags 00000000 Processed 17842
  Threads:
    Total 3 Idle 3
    Last started: 18/02/27 11:19:24

AIO:

  Sockets 9 Events 0 Waiters 3 Timers 2 Flags 00000000
  Threads:
    Total 3 Idle 3
    Last started: 18/02/27 11:19:24

Resources:

  Notifications:
    Received: Total 17 (local 13 internal 1)
              Queued 0 (threads 0, flags 00000000)

  Blocks:
    mLink   : 5000/5000 blocks 1
    subMatch: 10000/10000 blocks 1
    event   : 5000/5000 blocks 1

Java testing code

The Oracle official documentation is not super clear on this part and it is not possible to find a clear working example. The idea behind simplefan.jar library and classes is to register a listener that will listen on particular event. Currently only three are supported:

  • LoadAdvisoryEvent
  • NodeDownEvent
  • ServiceDownEvent

As I am not using a pool the LoadAdvisoryEvent has not raised for me. Then to be honest I did not know in advance which one would raise from the NodeDownEvent or ServiceDownEvent events in case I kill the used instance by my java program. At the end this is the ServiceDownEvent event that has raised…

You need to add to your project (classpath) ojdbc8.jar, ons.jar and simplefan.jar.

package fan01;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.Properties;
import oracle.jdbc.pool.OracleDataSource;
import oracle.simplefan.FanEventListener;
import oracle.simplefan.FanManager;
import oracle.simplefan.FanSubscription;
import oracle.simplefan.LoadAdvisoryEvent;
import oracle.simplefan.NodeDownEvent;
import oracle.simplefan.ServiceDownEvent;

public class fan01 {
  static Integer connection_status = 0;
  // Make Oracle connection and return a connection object
  private static Connection oracle_connection() throws Exception {
    Connection connection1 = null;
    OracleDataSource ods1 = new OracleDataSource();

    try {
      ods1.setUser("yjaquier");
      ods1.setPassword("secure_password");
      ods1.setURL("jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=rac-cluster-scan.domain.com)(PORT=1531))(CONNECT_DATA=(SERVICE_NAME=pdb1srv)))");
      connection1 = ods1.getConnection();
    }
    catch (SQLException e) {
      System.out.println("Connection Failed! Check output console");
      e.printStackTrace();
      return null;
    }
    System.out.println("Connected to Oracle database...");
    return connection1;
  }

  public static void main(String[] args) throws Exception {
    Connection connection1 = null;
    String query1 = null;
    ResultSet resultset1 = null;
    Properties props = new Properties();
    Properties onsProps = new Properties();
    FanManager fanMngr = FanManager.getInstance();


    try {
      Class.forName("oracle.jdbc.driver.OracleDriver");
    }
    catch (ClassNotFoundException e) {
      System.out.println("Where is your Oracle JDBC driver ?");
      e.printStackTrace();
      System.exit(1);
    }
    System.out.println("Oracle JDBC Driver Registered!");

    connection1=oracle_connection();
    if (connection1==null) {
      System.exit(1);
    }

    props.put("serviceName", "pdb1srv");
    onsProps.setProperty("onsNodes", "server2.domain.com:6200,server3.domain.com:6200");
    fanMngr.configure(onsProps);
    FanSubscription sub = fanMngr.subscribe(props);
    sub.addListener(new FanEventListener() {
      public void handleEvent(ServiceDownEvent event) {
        System.out.println("Service down event registered !");
        connection_status=1;
      }
      public void handleEvent(NodeDownEvent event) {
        System.out.println("Node down event registered !");
        connection_status=2;
      }
      public void handleEvent(LoadAdvisoryEvent event) {
        System.out.println("Load advisory event registered !");
        connection_status=3;
      }
    });

    query1="select HOST_NAME from v$instance";
    while (true) {
      System.out.println("\nStatus at "+LocalDateTime.now().format(DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss"))+": ");
      if (connection1!=null) {
        try {
          resultset1 = connection1.createStatement().executeQuery(query1);
          while (resultset1.next()) {
            System.out.println("Server used: "+resultset1.getString(1));
          }
        }
        catch (SQLException e1) {
          System.out.println("Query has failed...");
          if (connection_status != 0) {
            connection1.close();
            connection1 = null;
            while (connection1 == null) {
              try {
                connection1=oracle_connection();
              } catch (Exception e2) {
                e2.printStackTrace();
              }
            }
            connection_status = 0;
          }
        }
      }
      Thread.sleep(2000);
    }
    //resultset1.close();
    //connection1.close();
  }
}

Once the program is connected I have killed the pmon of the instance where the program connected. We clearly the ServiceDownEvent event raised and the re-connection handle by my Java program with no applicative failure:

fan01
fan01

Issues encountered

Default pluggable database service destroyed

If by mistake or lack of knowledge (my case) you try to play with default pluggable database service (DBMS_SERVICE package) you end up with a very particular situation where you cannot start your pluggable database when it is down:

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO
SQL> alter pluggable database pdb1 close immediate;

Pluggable database altered.

SQL> alter pluggable database pdb1 open;
 alter pluggable database pdb1 open
*
ERROR at line 1:
ORA-44304: service  does not exist
ORA-44777: Pluggable database service cannot be started.

SQL> select con_id, name from v$services;

    CON_ID NAME
---------- ----------------------------------------------------------------
         1 orclXDB
         1 orcl
         1 SYS$BACKGROUND
         1 SYS$USERS

SQL> select name from dba_services;

NAME
----------------------------------------------------------------
SYS$BACKGROUND
SYS$USERS
orclXDB
orcl

And if the pluggable database is up and running you cannot connect to it:

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO
SQL> alter session set container=pdb1;
ERROR:
ORA-44787: Service cannot be switched into.

SQL> alter session set container=pdb1;
alter session set container=pdb1
*
ERROR at line 1:
ORA-03113: end-of-file on communication channel
Process ID: 11524
Session ID: 54 Serial number: 53333

Situation and way to recover it as been well explained in Mike Dietrich and William Sescu blogs:

References

The post Fast Application Notification (FAN) – JDBC HA – part 3 appeared first on IT World.

]]>
https://blog.yannickjaquier.com/oracle/fast-application-notification-fan-jdbc-ha-part-3.html/feed 1
Transparent Application Failover (TAF) – JDBC HA – part 2 https://blog.yannickjaquier.com/oracle/jdbc-failover-highly-available-part-2.html https://blog.yannickjaquier.com/oracle/jdbc-failover-highly-available-part-2.html#comments Mon, 16 Jul 2018 09:18:29 +0000 https://blog.yannickjaquier.com/?p=4273 Preamble Transparent Application Failover (TAF) can be tested with a simple active/passive OS cluster (Pacemaker in my case) and you must use the JDBC OCI driver not the Thin one ! The principle is a primary node switchover to secondary node. In initial state my database is running on server2.domain.com: [root@server2 ~]# pcs status Cluster […]

The post Transparent Application Failover (TAF) – JDBC HA – part 2 appeared first on IT World.

]]>

Table of contents

Preamble

Transparent Application Failover (TAF) can be tested with a simple active/passive OS cluster (Pacemaker in my case) and you must use the JDBC OCI driver not the Thin one !

The principle is a primary node switchover to secondary node. In initial state my database is running on server2.domain.com:

[root@server2 ~]# pcs status
Cluster name: cluster01
Stack: corosync
Current DC: server2.domain.com (version 1.1.16-12.el7-94ff4df) - partition with quorum
Last updated: Wed Dec 20 15:56:27 2017
Last change: Wed Dec 20 15:55:54 2017 by root via cibadmin on server2.domain.com

2 nodes configured
5 resources configured

Online: [ server2.domain.com server3.domain.com ]

Full list of resources:

 Resource Group: oracle
     virtualip  (ocf::heartbeat:IPaddr2):       Started server2.domain.com
     vg01       (ocf::heartbeat:LVM):   Started server2.domain.com
     u01        (ocf::heartbeat:Filesystem):    Started server2.domain.com
     orcl       (ocf::heartbeat:oracle):        Started server2.domain.com
     listener_orcl      (ocf::heartbeat:oralsnr):       Started server2.domain.com

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

And I simply set the primary node to standby node with following command (that can be executed from any node of the cluster):

[root@server2 ~]# pcs node standby server2.domain.com

To finally reach below situation where all resources have been restarted on server3.domain.com:

[root@server2 ~]# pcs status
Cluster name: cluster01
Stack: corosync
Current DC: server2.domain.com (version 1.1.16-12.el7-94ff4df) - partition with quorum
Last updated: Wed Dec 20 17:42:49 2017
Last change: Wed Dec 20 17:35:52 2017 by root via cibadmin on server2.domain.com

2 nodes configured
5 resources configured

Node server2.domain.com: standby
Online: [ server3.domain.com ]

Full list of resources:

 Resource Group: oracle
     virtualip  (ocf::heartbeat:IPaddr2):       Started server3.domain.com
     vg01       (ocf::heartbeat:LVM):   Started server3.domain.com
     u01        (ocf::heartbeat:Filesystem):    Started server3.domain.com
     orcl       (ocf::heartbeat:oracle):        Started server3.domain.com
     listener_orcl      (ocf::heartbeat:oralsnr):       Started server3.domain.com

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled

You can unstandby server2.domain.com with:

[root@server2 ~]# pcs node unstandby server2.domain.com

I have defined below tnsnames entry:

PDB1 =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.99)(PORT = 1531))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = pdb1)
			(FAILOVER_MODE=(TYPE=SELECT)(METHOD=BASIC))
    )
  )

Transparent Application Failover (TAF) testing

The test code I am using is (running under Eclipse). You need to add ojdbc8.jar file to your project (classpath):

package taf01;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;

import oracle.jdbc.pool.OracleDataSource;
import oracle.jdbc.OracleConnection;

public class taf01 {
  // Make Oracle connection and return a connection object
  private static Connection oracle_connection() throws Exception {
    Connection connection1 = null;
    CallBack function_callback = new CallBack();
    String callback_message = null;
    OracleDataSource ods1 = new OracleDataSource();

    try {
      Class.forName("oracle.jdbc.driver.OracleDriver");
    }
    catch (ClassNotFoundException e) {
      System.out.println("Where is your Oracle JDBC driver ?");
      e.printStackTrace();
      return null;
    }

    System.out.println("Oracle JDBC Driver Registered!");

    try {
      ods1.setUser("yjaquier");
      ods1.setPassword("secure_password");
      ods1.setURL("jdbc:oracle:oci:@//192.168.56.99:1531/pdb1");
      connection1 = ods1.getConnection();
    }
    catch (SQLException e) {
      System.out.println("Connection Failed! Check output console");
      e.printStackTrace();
      return null;
    }
    System.out.println("Connected to Oracle database...");
    return connection1;
  }

  public static void main(String[] args) throws Exception {
    Connection connection1=null;
    String query1 = null;
    ResultSet resultset1 = null;

    // To set TNS_ADMIN variable in Java
    System.setProperty("oracle.net.tns_admin","C:/oracle/product/12.2.0/client_1/network/admin");
    connection1=oracle_connection();
    if (connection1==null) {
      System.exit(1);
    }
    query1="select HOST_NAME from v$instance";
    for(int i=1; i <= 10000; i++) {
      System.out.println("\nQuery "+i+" at "+LocalDateTime.now().format(DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss"))+": ");
      if (connection1!=null) {
        try {
          resultset1 = connection1.createStatement().executeQuery(query1);
          while (resultset1.next()) {
            System.out.println("Server used: "+resultset1.getString(1));
          }
        }
        catch (SQLException e) {
          System.out.println("Query has failed...");
        }
      }
      Thread.sleep(2000);
    }
    resultset1.close();
    connection1.close();
  }
}

With JDBC Thin driver my sample code does not fail but never ever reconnected to second node where the database is restarted (using jdbc:oracle:thin:@//192.168.56.99:1531/pdb1 as connect string):

taf01
taf01

Remark:
By the way we knew by looking at above table TAF is NOT available with JDBC Thin driver.

With JDBC OCI driver it is just simply failing (using jdbc:oracle:oci:@//192.168.56.99:1531/pdb1 as connect string):

taf02
taf02

If I try to use below TNS entry, either directly is Java program with:

ods1.setURL("jdbc:oracle:oci:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.56.99)(PORT=1531))(CONNECT_DATA=(SERVICE_NAME=pdb1)(FAILOVER_MODE=(TYPE=SELECT)(METHOD=BASIC))))");

Or simply with a direct TNS entry:

PDB1 =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.56.99)(PORT = 1531))
    )
    (CONNECT_DATA =
      (SERVICE_NAME = pdb1)
			(FAILOVER_MODE=(TYPE=SELECT)(METHOD=BASIC))
    )
  )

And the TNS_ADMIN property set with:

System.setProperty("oracle.net.tns_admin","C:/oracle/product/12.2.0/client_1/network/admin");

I have the same exact behavior even if from database standpoint it looks better:

SQL> set lines 200 pages 1000
SQL> select module, failover_type, failover_method, failed_over, service_name
     from v$session
     where username='YJAQUIER';

MODULE                                                         FAILOVER_TYPE FAILOVER_M FAI SERVICE_NAME
-------------------------------------------------------------- ------------- ---------- --- ----------------------------------------------------------------
javaw.exe                                                      SELECT        BASIC      NO  pdb1
SQL Developer                                                  NONE          NONE       NO  pdb1
SQL> set lines 200 pages 1000
SQL> col name for a10
SQL> select name,failover_delay, failover_method,failover_restore,failover_retries,failover_type
     from dba_services;

NAME       FAILOVER_DELAY FAILOVER_METHOD                              FAILOV FAILOVER_RETRIES FAILOVER_TYPE
---------- -------------- -------------------------------------------- ------ ---------------- --------------------------------------------
pdb1

Clearly implementing TAF and having a professional looking application requires a bit more of effort, let see how we can achieve this... If you dig in JDBC Developer's Guide you have a complete chapter on JDBC OCI driver TAF implementation.

Roughly this is achieved by creating a callback function that will handle the re-connection in case the connection has failed. You have multiple My Oracle Support (MOS) notes on the subject. It is also mandatory to use a TNS entry that is TAF enable. Refer to Net Services Administrator's Guide to know how to use it. The code I am using is:

package taf01;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;

import oracle.jdbc.OracleOCIFailover;
import oracle.jdbc.pool.OracleDataSource;
import oracle.jdbc.OracleConnection;

public class taf01 {
  // Make Oracle connection and return a connection object
  private static Connection oracle_connection() throws Exception {
    Connection connection1 = null;
    CallBack function_callback = new CallBack();
    String callback_message = null;
    OracleDataSource ods1 = new OracleDataSource();

    try {
      Class.forName("oracle.jdbc.driver.OracleDriver");
    }
    catch (ClassNotFoundException e) {
      System.out.println("Where is your Oracle JDBC Thin driver ?");
      e.printStackTrace();
      return null;
    }

    System.out.println("Oracle JDBC Driver Registered!");

    try {
      ods1.setUser("yjaquier");
      ods1.setPassword("secure_password");
      ods1.setURL("jdbc:oracle:oci:@pdb1");
      connection1 = ods1.getConnection();
      ((OracleConnection) connection1).registerTAFCallback(function_callback, callback_message);
    }
    catch (SQLException e) {
      System.out.println("Connection Failed! Check output console");
      e.printStackTrace();
      return null;
    }
    System.out.println("Connected to Oracle database...");
    return connection1;
  }

  public static void main(String[] args) throws Exception {
    Connection connection1=null;
    String query1 = null;
    ResultSet resultset1 = null;

    // To set TNS_ADMIN variable in Java
    System.setProperty("oracle.net.tns_admin","C:/oracle/product/12.2.0/client_1/network/admin");
    connection1=oracle_connection();
    if (connection1==null) {
      System.exit(1);
    }
    query1="select HOST_NAME from v$instance";
    for(int i=1; i <= 10000; i++) {
      System.out.println("\nQuery "+i+" at "+LocalDateTime.now().format(DateTimeFormatter.ofPattern("dd-MM-yyyy HH:mm:ss"))+": ");
      if (connection1!=null) {
        try {
          resultset1 = connection1.createStatement().executeQuery(query1);
          while (resultset1.next()) {
            System.out.println("Server used: "+resultset1.getString(1));
          }
        }
        catch (SQLException e) {
          System.out.println("Query has failed...");
        }
      }
      Thread.sleep(2000);
    }
    resultset1.close();
    connection1.close();
  }
}

// Define class CallBack
class CallBack implements OracleOCIFailover {
  // TAF callback function 
  public int callbackFn (Connection conn, Object ctxt, int type, int event) {
    String failover_type = null;

    switch (type) {
    case FO_SESSION: 
      failover_type = "SESSION";
      break;
    case FO_SELECT:
      failover_type = "SELECT";
      break;
    default:
      failover_type = "NONE";
    }

    switch (event) {
    case FO_BEGIN:
      System.out.println(ctxt + ": "+ failover_type + " failing over...");
      break;
    case FO_END:
      System.out.println(ctxt + ": failover ended");
      break;
    case FO_ABORT:
      System.out.println(ctxt + ": failover aborted.");
      break;
    case FO_REAUTH:
      System.out.println(ctxt + ": failover.");
      break;
    case FO_ERROR:
      System.out.println(ctxt + ": failover error gotten. Sleeping...");
      // Sleep for a while 
      try {
        Thread.sleep(1000);
      }
      catch (InterruptedException e) {
        System.out.println("Thread.sleep has problem: " + e.toString());
      }
      return FO_RETRY;
    default:
      System.out.println(ctxt + ": bad failover event.");
      break;
    }  
    return 0;
  }
}

I start the Java program when the database is running on server2.domain.com:

taf03
taf03

When the database switch to server3.domain.com the Java does not fail and wait until it is possible to re-initiate the connection and continue on server3.domain.com

taf04
taf04

Remark:
All what we have seen here above would be more than true with a RAC cluster. I have chosen to use a basic primary/secondary active/passive implementation because it is much more common in real life and also because it requires much less effort to build and maintain.

References

The post Transparent Application Failover (TAF) – JDBC HA – part 2 appeared first on IT World.

]]>
https://blog.yannickjaquier.com/oracle/jdbc-failover-highly-available-part-2.html/feed 1