Shell script to export all APEX applications as SQL scripts

Environment: Oracle 11gR2 EE, Application Express 4.2.2, Oracle Linux 6.4 64-bit, Java JDK 7u21

The APEX software contains a Java class which can be used to script or schedule the export of one or more APEX applications. I will show you in this post how to do this.

First, I unzipped the APEX software to /u01/app/apex. The Java class file is called “APEXExport.class” and can be found in the folder /u01/app/apex/utilities/oracle/apex.

You also need a Java JDK. I installed Java 7 JDK (update 21) under /u01/app/java/java7.

You might need to unlock (and set a password) for the APEX schema owner (in my case, APEX_040200). I’m not sure if it’s possible to export the applications with another user.

This is how my script looks like:

#!/bin/sh
# script to export all ApEx applications as sql scripts

CLASSPATH=$CLASSPATH:/u01/app/oracle/product/11.2.0/db_1/oui/jlib/classes12.jar; export CLASSPATH
BACKUPDIR=/u05/apex_backup

cd /u01/app/apex/utilities

/u01/app/java/java7/bin/java oracle.apex.APEXExport -db localhost:1521:ora11g01 -user apex_040200 -password <password> -instance

mv f*.sql $BACKUPDIR

if [ -f $BACKUPDIR/apex_export.log ]
then
    cat $BACKUPDIR/apex_export.log|mail -s "APEX export - `hostname -s`" matthiash@mydomain.com
fi

You need to include the file classes12.jar in your CLASSPATH or the JDBC connection won’t work.

The -instance setting will export all your applications. To limit the number of applications, you can also use -workspaceid or -applicationid. Check the readme.txt file under the utilities folder for more information.

To schedule the script with cron, I added the following line to my crontab file:

00 22 * * * /home/oracle/scripts/apex_export.sh 1>/u05/apex_backup/apex_export.log 2>&1

This will execute the script each night at 22:00 and mail the log file to the mail address inside the script.

Happy scripting! :-)

Matthias

Advertisements

Configuration of HugePages for Oracle database 11.2.0.3 on Oracle Linux 6.4

Environment: Oracle database 11.2.0.3 Enterprise Edition, Oracle Linux 6.4 64-bit

HugePages can give a performance boost on Linux systems with large SGA sizes. However, this is not set up by default, and the configuration can sometimes be a bit tricky.

This is how I did it on a system with a 4GB SGA size:

1) Disable Automatic Memory Management (AMM)

AMM is not compatible with HugePages. I disabled it by setting the following memory-related initialization parameters:

ALTER SYSTEM SET memory_max_target=0 SCOPE=SPFILE;

ALTER SYSTEM SET memory_target=0 SCOPE=BOTH;

ALTER SYSTEM SET sga_max_size=4016M SCOPE=SPFILE;

ALTER SYSTEM SET sga_target=4016M SCOPE=BOTH;

+ restart the instance

2) Calculate the number of HugePages needed

The size of one HugePage can be found as follows:

$ cat /proc/meminfo|grep Hugepagesize
Hugepagesize: 2048 kB

The amount of HugePages that you need can be found with the following script:

#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
do
   MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
   if [ $MIN_PG -gt 0 ]; then
      NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
   fi
done
# Finish with results
case $KERN in
   '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
          echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
   '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    *) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End

$ ./hugepages_settings.sh
Recommended setting: vm.nr_hugepages = 2013

3) Set the number of HugePages in /etc/sysctl.conf

Add the following to /etc/sysctl.conf:

vm.nr_hugepages = 2013

4) Set the memory limits for user “oracle”

The memory limits for user “oracle” can be calculated by multiplying the number of HugePages with the Hugepagesize (so, in my case, 2013*2048 = 4122624).

Add this number to /etc/security/limits.conf:

oracle soft memlock 4122624
oracle hard memlock 4122624

5) Set USE_LARGE_PAGES

By setting the initialization parameter USE_LARGE_PAGES, you can force the instance to only start up when enough HugePages are available.

ALTER SYSTEM SET USE_LARGE_PAGES=only SCOPE=SPFILE;

6) Reboot the system

Now reboot the server. You can check if large pages are being used in 2 ways:

$ cat /proc/meminfo|grep HugePages

AnonHugePages: 237568 kB
HugePages_Total: 2013
HugePages_Free: 1532
HugePages_Rsvd: 1528
HugePages_Surp: 0

Or check the alert.log file for the database instance:

Starting ORACLE instance (normal)
****************** Large Pages Information *****************
Parameter use_large_pages = ONLY

Total Shared Global Region in Large Pages = 4018 MB (100%)

Large Pages used by this instance: 2009 (4018 MB)
Large Pages unused system wide = 4 (8192 KB) (alloc incr 16 MB)
Large Pages configured system wide = 2013 (4026 MB)
Large Page size = 2048 KB
***********************************************************

Good luck! :-)

 

Matthias

Measuring Oracle I/O performance using DBMS_RESOURCE_MANAGER.CALIBRATE_IO

Environment: Oracle database 11.2.0.3.0, Oracle Linux 6.4

Oracle 11g has a procedure named “CALIBRATE_IO” which you can use to measure the I/O performance of your Oracle system. You can use this procedure to evaluate disk subsystem layout or configuration changes.

The procedure is part of the DBMS_RESOURCE_MANAGER PL/SQL package. Only users with the SYSDBA privilege can execute this procedure. You also need to enable ASYNCH_IO for your datafiles.

Note: do not run this on a live production system. For optimal results, use this on a quiet system with no other users connected.

This is an example with user SYS as SYSDBA (you have to enable DBMS_OUTPUT to see the results):

DECLARE
  lat  INTEGER;
  iops INTEGER;
  mbps INTEGER;
BEGIN
  DBMS_RESOURCE_MANAGER.CALIBRATE_IO(
      1 /* # of disks */
      , 10 /* maximum tolerable latency in milliseconds */
      , iops /* I/O rate per second */
      , mbps /* throughput, MB per second */
      , lat  /* actual latency in milliseconds */
  );
  DBMS_OUTPUT.PUT_LINE('max_iops = ' || iops);
  DBMS_OUTPUT.PUT_LINE('latency  = ' || lat);
  DBMS_OUTPUT.PUT_LINE('max_mbps = ' || mbps);
END;
/

On my system, the total execution time was about 8 minutes.
The output is as follows:

max_iops = 2588
latency = 8
max_mbps = 241

The results can also be found in the system table DBA_RSRC_IO_CALIBRATE.

HTH,
Matthias

OPatch warnings after installation of PSU 11.2.0.3.6 on Oracle 11.2.0.3.0

Environment: Oracle Database 11.2.0.3.0, Oracle Linux 6.4, OPatch v. 11.2.0.3.4

I was installing Patch Set Update 11.2.0.3.6 (as part of the CPU for April 2013) on a freshly installed Oracle 11.2.0.3.0 database, when I noticed the following warnings in the OPatch log file:

OPatch found the word “warning” in the stderr of the make command.
Please look at this stderr. You can re-run this make command.
Stderr output:
ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg’
/u01/app/oracle/product/11.2.0/db_1/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg’
/u01/app/oracle/product/11.2.0/db_1/precomp/lib/ins_precomp.mk:19: warning: overriding commands for target `pcscfg.cfg’
/u01/app/oracle/product/11.2.0/db_1/precomp/lib/env_precomp.mk:2160: warning: ignoring old commands for target `pcscfg.cfg’

The database seemed fine, though. I contacted Oracle support just in case, and it was confirmed as a known issue. The document with ID 1448337.1 has more information on it. So basically, these warnings can be safely ignored…

 

Matthias

 

Upgrade APEX 4.1.1 to 4.2.1 on Oracle Linux with GlassFish

Environment: Oracle Linux 6.3, Oracle database 11.2.0.3.0 EE, APEX 4.1.1.00.23, GlassFish 3.1.2.2

This blog post explains how you can upgrade your Application Express installation from version 4.1.1 to version 4.2.1, with the APEX Listener deployed under GlassFish.

Steps:

1) download the software

You can download APEX 4.2.1 here. I downloaded the “all languages” zip file. Please note that the documentation is no longer included with the software. You can download the documentation separately here.

2) copy and extract the software

Now copy and extract the apex_4.2.1.zip file to your server. I use “unzip” to unzip the zip file. This will create a folder named “apex”. I usually keep this folder on my server in a permanent location, for example /u01/app/apex.

3) check the tablespace requirements

You need about 275MB of free space in the APEX tablespace (mine is called “APEX”) and 100MB of free space in the SYSTEM tablespace.

4) stop the APEX Listener

Before you start with the upgrade, you need to stop the APEX Listener to make sure no one is using the software during the upgrade.

To stop the APEX Listener under GlassFish, you can do it as follows:

– connect to the GlassFish administration console (by default this is port 4848 using SSL).

– log in with the “admin” user.

– click on Standalone Instances => Applications.

– select the “apex” application and select “Disable” from the “More Actions” drop down list.

The APEX Listener should now be stopped on your GlassFish instance.

5) start the upgrade

To upgrade the APEX software, you need to run the “apexins.sql” sql script from within the “apex” folder that you extracted above. You need to connect as SYS as SYSDBA with the oracle software owner (mine is called “oracle”). Make sure the “apex” folder is writeable!

apexins.sql needs 4 parameters: the name of the tablespace for the APEX user (in my case: APEX), the name of the tablespace for the FLOWS_FILES user (in my case: APEX), the name of the TEMP tablespace (in my case: TEMP) and the name of the virtual image directory (which is “/i/”).

$ su - oracle
$ cd /u01/app/apex
$ sqlplus /nolog
SQL*Plus: Release 11.2.0.3.0 Production on Thu Jan 3 11:21:39 2013

Copyright (c) 1982, 2011, Oracle. All rights reserved.

SQL> connect / as sysdba
Connected.
SQL> @apexins.sql APEX APEX TEMP /i/

The upgrade will now start. This will create a new schema called “APEX_040200”. The upgrade can take some time, depending on the speed of your server. On my server (with fairly recent hardware), it took about 17 minutes. After the upgrade, check the installation log file (in the same folder where you exectuted the apexins.sql file) to make sure no errors were encountered during the process.

6) verify the installation

I always verify my APEX installation after a upgrade by querying dba_registry:

$ sqlplus /nolog

SQL*Plus: Release 11.2.0.3.0 Production on Thu Jan 3 12:06:21 2013

Copyright (c) 1982, 2011, Oracle.  All rights reserved.

SQL> conn / as sysdba
Connected.
SQL> select status from dba_registry where comp_id='APEX';

STATUS
-------------------------------------------------------
VALID

7) copy the images

Next, copy the new and updated images from your installation folder to the GlassFish docroot (you might take a backup folder of the original docroot/i folder just in case):

$ cp -rf /u01/app/apex/images/* /u01/app/glassfish/glassfish/nodes/localhost-domain1/node01/docroot/i/

8) start the APEX Listener

Finally, start the APEX Listener again (see under step 4) and check if all your applications work fine :-) You should now have a nicely upgraded APEX 4.2.1.00.08 installation!

HTH,

Matthias

 

Update:

If you are accessing network services from withing APEX, like LDAP authentication, print servers and such, you will need to grant the new APEX_040200 user network access or it won’t work. This is documented in the installation guide and is only needed for Oracle 11g. You can grant network access by running the following PL/SQL script with user SYS as SYSDBA:

DECLARE
ACL_PATH VARCHAR2(4000);
BEGIN
-- Look for the ACL currently assigned to '*' and give APEX_040200
-- the "connect" privilege if APEX_040200 does not have the privilege yet.
SELECT ACL INTO ACL_PATH FROM DBA_NETWORK_ACLS
WHERE HOST = '*' AND LOWER_PORT IS NULL AND UPPER_PORT IS NULL;
IF DBMS_NETWORK_ACL_ADMIN.CHECK_PRIVILEGE(ACL_PATH, 'APEX_040200',
'connect') IS NULL THEN
DBMS_NETWORK_ACL_ADMIN.ADD_PRIVILEGE(ACL_PATH,
'APEX_040200', TRUE, 'connect');
END IF;
EXCEPTION
-- When no ACL has been assigned to '*'.
WHEN NO_DATA_FOUND THEN
DBMS_NETWORK_ACL_ADMIN.CREATE_ACL('power_users.xml',
'ACL that lets power users to connect to everywhere',
'APEX_040200', TRUE, 'connect');
DBMS_NETWORK_ACL_ADMIN.ASSIGN_ACL('power_users.xml','*');
END;
/
COMMIT;

Script to monitor process on Linux

Environment: Oracle Linux 6.2

Sometimes you want to monitor a process on your Linux server and do some alerting when the process is not running anymore.

The following shell script can be used as an example. It will monitor the “dbconsole” process (database control) and send a e-mail message when the process is not running anymore.

$ vi check_db_control.sh

#!/bin/sh

PROCESSFILE='dbconsole'

if [ `ps ax|grep -v grep|grep -ic $PROCESSFILE` -gt 0 ]
then
        echo "$PROCESSFILE is running, everything is fine"
else
        echo "$PROCESSFILE is not running on `hostname -s`!" | mail -s "$PROCESSFILE down!" matthiash@mydomain.com
fi

exit

$ chmod u+x check_db_control.sh

To schedule the script, you can add it to cron. For example, this script will run every day at 10:00AM:

$ crontab -e

# Monitoring dbconsole process
00 10 * * * /home/oracle/scripts/check_db_control.sh 1>/dev/null 2>&1

 

HTH,
Matthias

How to modify the version of Java used for GlassFish 3.1.2.2

Environment: GlassFish 3.1.2.2 OSE Web Profile, Oracle Linux 6.3, Java 7 Update 7.

Whenever you upgrade your Java installation or install a new version, you have to make sure that your GlassFish installation is using the new version.

1) Download and install the new Java JDK 

New versions of Java can be found here. For GlassFish, you need to download the JDK (so not JRE), and you can use both Java 6 or Java 7.

For example, I downloaded and installed JDK 7 Update 7 to /u01/app/jdk1.7.0_07 and I created a symbolic link “java7” to it:

$ /u01/app/java>tar -zxf jdk-7u7-linux-x64.tar.gz
$ /u01/app/java>ln -sf jdk1.7.0_07 java7
$ /u01/app/java>ls -la
total 93904
drwxr-xr-x   4 oracle oinstall     4096 Sep 10 15:16 .
drwxr-xr-x   7 oracle oinstall     4096 Feb 17  2012 ..
lrwxrwxrwx   1 oracle oinstall       11 Feb  7  2012 java6 -> jdk1.6.0_30
lrwxrwxrwx   1 oracle oinstall       11 Sep 10 15:16 java7 -> jdk1.7.0_07
drwxr-xr-x  10 oracle oinstall     4096 Feb  3  2012 jdk1.6.0_30
drwxr-xr-x   8 oracle oinstall     4096 Aug 29 03:12 jdk1.7.0_07

2) Modify the GlassFish domain config file and JAVA_HOME environment variable

Next, you need to modify the AS_JAVA  variable in the GlassFish config file “asenv.conf”. In my case, this file was stored in the folder /u01/app/glassfish/glassfish/config. I changed it as follows:

AS_JAVA=/u01/app/java/java7

You also need to modify the environment variable JAVA_HOME. This variable can be found in the .bash_profile file of the user which is used to start GlassFish. Run “echo $JAVA_HOME” to see its current value. After the changes, log out and log in again (or run .bash_profile) to reset it.

3) Restart the GlassFish domain server

Restart the GlassFish domain server and any other server instances you might have.

/u01/app/glassfish/bin/asadmin stop-local-instance
/u01/app/glassfish/bin/asadmin stop-domain domain1

/u01/app/glassfish/bin/asadmin start-domain domain1
/u01/app/glassfish/bin/asadmin start-local-instance

 

HTH,

Matthias