Oracle 11gR2, RMAN, NetBackup, and the Media Layer (MML).

Oracle 11gR2 (11.2.0.1). This is a step-by-step approach to configuring Oracle 11gR2 with RMAN and a NetBackup Media Layer. This is a basic example of my configuration and not a in-depth discussion of RMAN and NetBackup MML. But it is enough to get started.

Verify your NetBackup installation. For 11gR2 you need a minimum of NBU 6.5.4 Master and Client.
If you are using version 6.5.4 you need to apply the Symantec EEB fix.
Details:
Symantec Bug ID: 1862252
Version: NetBackup_6.5.4
Installation Location: Client
Fixes Included:
(ET1940073) RMAN-10038 error when attempting to backup Oracle version 11.2 using NetBackup 6.x.
There is a different EEB if you are on NBU 6.5.5 and I believe NBU 7.0 has resolved the issue with Oracle 11gR2.

Be sure to shutdown the database instance and then run oracle_link.

Run oracle_link (as oracle owner) in found in the netbackup/bin directory.
This will link $ORACLE_HOME/lib/libobk.so to NBU libobk.so64
libobk.so -> /usr/openv/netbackup/bin/libobk.so64

Update the bp.conf file on the client to reflect your client, master, and media servers
SERVER =
EMMSERVER=
MEDIA_SERVER=
CLIENT_NAME=

Look in the following netbackup directory for a sample script. Copy this script to the location you want to keep your NBU scripts.
netbackup/ext/db_ext/oracle/samples/rman

I have modified this script greatly over time and for many different backup types. This is just a example of the minimum updates required to use it. Even if you choose to not use a script a NetBackup Policy is mandatory. Create an Oracle Policy even if you do not use this script as a template.

Using an RMAN catalog? I highly recommend it.
I am starting with the script hot_database_backup.sh and will copy it to $ORACLE_HOME/scripts/rman and update it for my environment.

Update the permissions on the script so you can edit it.
scripts/rman% chmod 775 hot_database_backup.sh

I will modify the following lines.

ORACLE_HOME=your Oracle home
ORACLE_SID=your SID
ORACLE_USER=oracle
TARGET_CONNECT_STR=/

Add the following for the catalog connection
CATALOG_CONNECT_STR=xxx/xxx@xxx (catalog login password and SID)

Now update the RMAN variable prior to the run command
$RMAN target $TARGET_CONNECT_STR catalog $CATALOG_CONNECT_STR msgno append <<>

Finally I add a line to email the results to me.
The command is all one line. Your host must already be configured to send mail.
cat $RMAN_LOG_FILE | mailx -s "${ORACLE_SID} on `uname -n` $BACKUP_TYPE $LOGMSG
- EXIT Status = $RSTAT" youremail@company.com

Before configuring RMAN, create a backup policy in NetBackup. This is a specific Oracle Policy for this particular client. The existence of a Policy is mandatory even if you run a command for the command line or call an RMAN script.

These are the minimum RMAN parameter configuration I recommend.

Start RMAN and connect to a target database and a recovery catalog
rman target / catalog catlogin/password@catalog_sid

RMAN> list incarnation;
RMAN> Register database;
database registered in recovery catalog
starting full resync of recovery catalog
full resync complete

RMAN> list incarnation;

Run the RMAN SHOW command.
RMAN> SHOW ALL;

RMAN> SHOW RETENTION POLICY;
RMAN> SHOW RETENTION POLICY;
RMAN configuration parameters for database with db_unique_name PROD are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default

RMAN> CONFIGURE RETENTION POLICY TO REDUNDANCY 3;
RMAN> SHOW DEFAULT DEVICE TYPE;
RMAN configuration parameters for database with db_unique_name PROD are:
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default

RMAN> CONFIGURE DEFAULT DEVICE TYPE TO sbt;
RMAN> CONFIGURE DEVICE TYPE sbt PARALLELISM 2;
RMAN> CONFIGURE DEVICE TYPE sbt BACKUP TYPE TO BACKUPSET;
RMAN> CONFIGURE CONTROLFILE AUTOBACKUP ON;

Now test a backup to your MML
RMAN> backup spfile;

You can view the Activity Monitor on your NetBackup Administrative Console GUI for activity of the newly created policy.

Example of the output you should see on the server.

RMAN> backup spfile;

Starting backup at 09-JUL-10
allocated channel: ORA_SBT_TAPE_1
channel ORA_SBT_TAPE_1: SID=228 device type=SBT_TAPE
channel ORA_SBT_TAPE_1: Veritas NetBackup for Oracle - Release 6.5 (2010011113)
allocated channel: ORA_SBT_TAPE_2
channel ORA_SBT_TAPE_2: SID=254 device type=SBT_TAPE
channel ORA_SBT_TAPE_2: Veritas NetBackup for Oracle - Release 6.5 (2010011113)
channel ORA_SBT_TAPE_1: starting full datafile backup set
channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set
including current SPFILE in backup set
channel ORA_SBT_TAPE_1: starting piece 1 at 09-JUL-10
channel ORA_SBT_TAPE_1: finished piece 1 at 09-JUL-10
piece handle=01lic0o5_1_1 tag=TAG20100709T143028 comment=API Version 2.0,MMS Version 5.0.0.0
channel ORA_SBT_TAPE_1: backup set complete, elapsed time: 00:01:35
Finished backup at 09-JUL-10

Starting Control File and SPFILE Autobackup at 09-JUL-10
piece handle=c-134461749-20100709-00 comment=API Version 2.0,MMS Version 5.0.0.0
Finished Control File and SPFILE Autobackup at 09-JUL-10

Veritas NetBackup - querying your media

To see what tapes you currently have in your storage unit, which pool they are assigned to, which ones are full, and how much is used of the others.

From the Master Server, change directories to goodies and run the available media command:

# cd /usr/openv/netbackup/bin/goodies
#./available_media

media media robot robot robot side/ ret size status ID type type # slot face level KBytes
-----------------------------------------------------------------------------------------------

CatalogBackup pool

xxx151 HCART TLD 0 47 - 3 69689341 ACTIVE
xxx152 HCART TLD 0 46 - 3 177652257 ACTIVE

NetBackup pool

xxx030 HCART TLD 0 28 - 6 477197502 ACTIVE/MPX
xxx139 HCART TLD 0 126 - 4 286852847 ACTIVE/MPX
xxx161 HCART TLD 0 59 - 4 58710487 ACTIVE/MPX
xxx195 HCART NONE - - - - - AVAILABLE
xxx007 HCART TLD 0 29 - 4 1439675239 FULL/MPX
xxx072 HCART TLD 0 128 - 4 1188893711 FULL/MPX
xxx134 HCART TLD 0 40 - 4 1287883499 FULL/MPX
xxx136 HCART TLD 0 100 - 4 1119822897 FULL/MPX
xxx137 HCART TLD 0 139 - 4 1365587550 FULL/MPX
xxx138 HCART TLD 0 140 - 4 1244368634 FULL/MPX
xxx144 HCART TLD 0 57 - 4 1101532193 FULL/MPX

Scratch pool

xxx162 HCART TLD 0 60 - - - AVAILABLE
xxx163 HCART TLD 0 109 - - - AVAILABLE
xxx164 HCART TLD 0 102 - - - AVAILABLE
xxx165 HCART TLD 0 111 - - - AVAILABLE
xxx166 HCART TLD 0 4 - - - AVAILABLE

Veritas NetBackup - Insert media failed: EMM database error (196)

Update failed: could not add new media ID 'XYZ166' into slot 4
Insert media failed:
EMM database error (196)


I am on NetBackup 6.5.4
Attempting to add new media to my tape silo I encounter the above error. Fortunately the problem was easily resolved, but not before spending most of the day researching and on the phone with technical support.
What causes this error? If you have tapes that have been Inventoried by a previous version of NetBackup and have since upgraded or re-installed the master server application. This was a new installation of NetBackup 6.5.4 and I am using unused tapes previously inventoried under the former install of 6.5.2 which had been upgraded from version 5.2. The error only occurs because I am adding new tapes to the inventory and there are existing tapes (used or not) already in the silo that were inventoried once upon a time by a previous version. Why? I believe this is unique to the circumstances, and obviously a bug, but fortunately there is a solution.
On the Master Server type the following (exactly as shown and all on the same line):

# cd /opt/openv/netbackup/bin
# ./nbpushdata -add_flag NBPUSHDATA_G_GLOBDB_HOST dummyhostname:dummyhostname

This command only needs to be executed once. You do not need to restart the Netbackup services. Inventory the robot again and you should have success.