DB2 Extract begin now positioning delay issues - GoldenGate

We are seeing delays starting up a (DB2 ( extract process of up to 6 hours that appear to be related to troubles with positioning itself in the DB2 transaction log when using the 'begin now' option. The extract report file will show this line:
2010-10-13 12:28:28 INFO OGG-01515 Positioning to begin time Oct 13, 2010 12:28:24 PM.
but then will only show the subsequent positioned message minutes or even hours later:
2010-10-13 12:47:30 INFO OGG-01560 Positioned to 3496288233245.
Until this positioning has finished, the extract process will not respond to ggsci commands such as attempts to stop it.
When investigating on the DB2 side, the db2diag showed these errors:
2010-10-13- I203065E576 LEVEL: Warning
PID : 1707 TID : 47882228984128PROC : db2sysc 0
INSTANCE: phxinst1 NODE : 000 DB : VAP
APPHDL : 0-43653 APPID: *LOCAL.phxinst1.101013175112
EDUID : 28 EDUNAME: db2agent (VAP) 0
FUNCTION: DB2 UDB, recovery manager, sqlpMoveBackExtents, probe:100
MESSAGE : Log file size has changed. Guessing at log containing LSN
00000197058252CB, based on log file size 5000, calculated extent
number is 0,
2010-10-13- E203642E482 LEVEL: Warning
PID : 1707 TID : 47882228984128PROC : db2sysc 0
INSTANCE: phxinst1 NODE : 000 DB : VAP
APPHDL : 0-43653 APPID: *LOCAL.phxinst1.101013175112
EDUID : 28 EDUNAME: db2agent (VAP) 0
FUNCTION: DB2 UDB, data protection services, sqlpALR_OpenExtent, probe:0
MESSAGE : ADM1500W DB2 is unable to locate log file "S4967295.LOG".
S4967295.LOG request appeared to be out of range compared to the LSNs showed by 'db2pd -db <dbname> -logs' (0x032F9C630000 - 0x032FA75F8000).
Any ideas?

Wish I did, Marcell. Sounds like a candidate for an SR. I suspect a permissions issue.
Good luck,


[UNEXPECTED]Duplicate rows found for host XXXX in FND_OAM_CONTEXT_FILES table

Hi, After a adop session completed successfully , i wanted to run an fs_clone phase but i had this error :  Checking for existing adop sessions.    [UNEXPECTED]Duplicate rows found for host vmwqsr-pgi2-q01 in FND_OAM_CONTEXT_FILES table*******FATAL ERROR*******PROGRAM : (/pgi/pgiqual/fs1/EBSapps/appl/ad/12.0.0/bin/adzdoptl.pl)TIME    : Mon Jun  6 15:28:11 2016FUNCTION: ADOP::CommonUtil::getAppltopId [ Level 1 ]ERRORMSG: Duplicate rows found for host vmwqsr-pgi2-q01 in FND_OAM_CONTEXT_FILES table  I saved the table "FND_OAM_CONTEXT_FILES" and drop it , and run an autoconfig in dbtier and run fs but still have the same issue.  when i check the adopscanlog    Lines #(77-81):    [PROCEDURE] Calling: /pgi/pgiqual/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl    [EVENT]     Log: /pgi/pgiqual/fs_ne/EBSapps/log/adop/17/prepare_20160606_150203/PGIQUAL_vmwqsr-pgi2-q01    [UNEXPECTED]Error occurred running "perl /pgi/pgiqual/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl  -contextfile=/pgi/pgiqual/fs2/inst/apps/PGIQUAL_vmwqsr-pgi2-q01/appl/admin/PGIQUAL_vmwqsr-pgi2-q01.xml -patchctxfile=/pgi/pgiqual/fs1/inst/apps/PGIQUAL_vmwqsr-pgi2-q01/appl/admin/PGIQUAL_vmwqsr-pgi2-q01.xml -phase=prepare -logloc=/pgi/pgiqual/fs_ne/EBSapps/log/adop/17/prepare_20160606_150203/PGIQUAL_vmwqsr-pgi2-q01 -promptmsg=hide"    [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on vmwqsr-pgi2-q01[PROCEDURE] [START 2016/06/06 15:03:02] Unlocking sessions table  Lines #(78-82):    [EVENT]     Log: /pgi/pgiqual/fs_ne/EBSapps/log/adop/17/prepare_20160606_150203/PGIQUAL_vmwqsr-pgi2-q01    [UNEXPECTED]Error occurred running "perl /pgi/pgiqual/fs2/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPValidations.pl  -contextfile=/pgi/pgiqual/fs2/inst/apps/PGIQUAL_vmwqsr-pgi2-q01/appl/admin/PGIQUAL_vmwqsr-pgi2-q01.xml -patchctxfile=/pgi/pgiqual/fs1/inst/apps/PGIQUAL_vmwqsr-pgi2-q01/appl/admin/PGIQUAL_vmwqsr-pgi2-q01.xml -phase=prepare -logloc=/pgi/pgiqual/fs_ne/EBSapps/log/adop/17/prepare_20160606_150203/PGIQUAL_vmwqsr-pgi2-q01 -promptmsg=hide"    [UNEXPECTED]Error 1 occurred while Executing txkADOPValidation script on vmwqsr-pgi2-q01[PROCEDURE] [START 2016/06/06 15:03:02] Unlocking sessions table[STATEMENT] Unlocking ad_adop_sessions table for vmwqsr-pgi2-q01 with wait interval of 60 seconds and number of tries 2 Best regardsMarouane

Essbase application abnormal termination

Hi All, I am facing a strange issue. When we are running business rule, essbase application is shutting down abnormally with the following error. [Thu Jan 22 05:01:24 2016]Local/xxx/xxxx/admin#Native Directory/1221593408/Info(1020082)Spreadsheet Extractor Big Block Allocs -- Dyn.Calc.Cache : [5196] non-Dyn.Calc.Cache : [0][Thu Jan 22 11:34:15 2016]Local/xxxx/xxxx/admin#Native Directory/1372125504/Error(1070051)Duplicate key [0.1452574662] found in AVL-Tree.[Thu Jan 22 11:34:16 2016]Local/xxxx///1372125504/Info(1008108)Essbase Internal Logic Error [7101][Thu Jan 22 11:34:16 2016]Local/xxxx///1372125504/Info(1008106)Exception error log [/ap01/cap/hyperion/Oracle/Middleware/user_projects/epmsystem1_FND/diagnostics/logs/essbase/essbase/app/xxxx/log00048.xcp] is being created I do see the following entries in the last lines of expection file.Thread Id 554646:   Request Name:        SpreadsheetOperation   Database Name:       xxxx   User Name:           50211456#Native Directory   Start Time:          Thu Jan 22 11:34:16 2016   End Time:            Thu Jan 22 11:34:16 2016Thread Id 45465:   Request Name:        Calculate   Database Name:       xxx   User Name:           50211456#Native Directory   Start Time:          Thu Jan 22 11:34:16 2016   End Time:            Pending Could someone please give some insight it this. Message was edited by: 1060285
Corrupt database - Error: "Duplicate Key Found in AVL-Tree" When Running a Calculation. (Doc ID 780783.1) Cheers John
Thanks John for the response, Yes, The database got corrupted but I would like to know why this error occured as I dont see any errors in the logs expect the below entries before this error occured. Current selection of task dimensions [1]  will generate insufficient number of tasks [1] for parallel calculation. See whether calculation time can be improved by increasing the number of task dimensions by one (see SET CALCTASKDIMS topic in the Technical Reference). Also, consider using FIXPARALLEL to make custom task selections that are different from CALCPARALLEL[Thu xxxx/xxxx]Local/xxxx/xxxx/admin#Native Directory/1373178176/Info(1012678)Calculating in parallel with [4] threads[Thu xxxx/xxxx] Local/xxxx/xxxx/admin#Native Directory/1373178176/Info(1012679)Calculation task schedule [1]xxxx/xxxxLocal/xxxx/xxxx/admin#Native Directory/1373178176/Info(1012680)Parallelizing using [1] task dimensions.[]Local/xxxx/xxxx/admin#Native Directory/1378441536/Info(1012568)Commit Blocks Interval was adjusted to be [100000] blocksxxxx/xxxxLocal/xxxx/xxxx/admin#Native Directory/1373178176/Info(1012675)Commit Blocks Interval for the calculation is [3000]xxxx/xxxxLocal/xxxx/xxxx/admin#Native Directory/1373178176/Info(1012717)Remote bitmap cache is [Disabled]xxxx/xxxxLocal/xxxx/xxxx/admin#Native Directory/1373178176/Info(1012678)Calculating in parallel with [4] threads
You probably won't see anything in the logs, I can't tell you why you have corruption, it could be a problem writing to the disk, it could be a bug, the problem could have been there for a while, it could be a corrupt block on the disk and when it was accessed Essbase will crash.Have a read of Troubleshooting Oracle Hyperion Essbase Crashes and Hangs (Doc ID 2028052.2) and log it with Oracle and see what they have to say.
Thanks John for the inputs, I will check with oracle on the same.

Configure the downstream database to receive and write shipped redo data locally doesn't work

I'm following the note Oracle GoldenGate Best Practices: Configuring Downstream Integrated Extract Document ID: 1485620.1 The configuration in mining database spfile *.log_archive_config='DG_CONFIG=(DB11G01A,DOWNGGDB)'*.log_archive_dest_1='LOCATION=/u03/archivers/DOWNGGDB/local'*.log_archive_dest_2='LOCATION=/u03/archivers/DOWNGGDB/foreign VALID_FOR=(STANDBY_LOGFILE,PRIMARY_ROLE)'*.log_archive_dest_state_1='ENABLE'*.log_archive_dest_state_2='ENABLE'*.log_archive_format='%t_%s_%r.arc' I receiving ths standby redologs in the same directory as the local archivers ( /u03/archivers/DOWNGGDB/local ) rather than /u03/archivers/DOWNGGDB/foreignI see the next message in the alert.log of DWONGGDB ( the mining database ) Wed Feb 21 10:23:23 2018Using STANDBY_ARCHIVE_DEST parameter default value as /u03/archivers/DOWNGGDB/local I doesn't define a value for this parameter in my configuration but it is initialized with this valueSQL> show parameter STANDBY_ARCHIVE_DEST  NAME     TYPE VALUE------------------------------------ ----------- ------------------------------standby_archive_dest     string ?/dbs/arch The system is working fine ( I mean, I'm receiving the standby redo logs in real-time ) but I don't know why the foreign archivers are not created in the specified directory. Any ideas ?Thank you very much for your help.
Sorry folksI have found the problem,,,Unfortunately I have defined the local destination uncompletly as you can see *.log_archive_dest_1='LOCATION=/u03/archivers/DOWNGGDB/local' rather than the complete and correct way *.log_archive_dest_1='LOCATION=/u03/archivers/DOWNGGDB/local VALID_FOR=(ONLINE_LOGFILE,PRIMARY_ROLE)' The local archivers were created in the correct directory but the log_archive_dest_2 was not working as spected. Actions executed : ALTER SYSTEM SET log_archive_dest_1='LOCATION=/u03/archivers/DOWNGGDB/local VALID_FOR=(ONLINE_LOGFILE,PRIMARY_ROLE)' SCOPE=BOTH; Wed Feb 21 11:10:42 2018ALTER SYSTEM SET log_archive_dest_1='LOCATION=/u03/archivers/DOWNGGDB/local VALID_FOR=(ONLINE_LOGFILE,PRIMARY_ROLE)' SCOPE=BOTH;Wed Feb 21 11:12:26 2018RFS[1]: Opened log for thread 1 sequence 270 dbid -910718994 branch 953917363Wed Feb 21 11:13:21 2018Using STANDBY_ARCHIVE_DEST parameter default value as USE_DB_RECOVERY_FILE_DEST ALTER SYSTEM SET log_archive_dest_state_2='DEFER' SCOPE=MEMORY;I switch logfile in origin DB and the new standby archive log was created in db_recovery_file_dest   directory. ALTER SYSTEM SET log_archive_dest_state_2='ENABLE' SCOPE=MEMORY;I switch logfile in origin DB and the new standby archive log was created correctly in log_archive_dest_2 directory. Sorry for the inconvenience.

Metric Collection Error oracle 11.2g RAC

Dear experts, At the time of loging Oracle Enterprise Manager (RAC configuration with 2 nodes), I am getting Metric Collection Error inGeneral - Staus - Metric Collection ErrorThis database is not configured.After clicking on Metric Collection Error its showingCurrent Status: After clicking onIn Erro Since Apr 11, 2015 1:06:50 PMAvailabiltiy (%) Not ApplicableDown Time (Minutes) 0Blackout Time (Minutes) 0Agent Down (Minutes) 0System Error (Minutes) 1440Status Pending Time (Minutes) 0Acknowledged YesAcknowledged by SYSTEM Overall availability graph is showing System Error Time (100%) when selecting View Data (24hrs)   I checked alert file its showing Fatal NI connect error 12170and IP is for Node2Tracing not truned on.Tns error struct:ns main err code: 12535TNS-12535 : TNS: operation time outns secondary err code : 12606nt main err code : 0nt os err code warning: inbound connection timed out (ORA-3136) Suggest what to do ?

Oracle Instance Crashes when allocating channel using Oracle Database Backup Service

I am running Oracle Version on a Windows Sever 2008 SP 2 64bit OS.  Today when I tried to allocate a channel for 'SBT_TAPE' my instance crashed.  Has anyone else experienced this?  set ORACLE_HOME=D:\app\oracle\product\11.2.0\dbhome_1\ set ORACLE_SID=CNFCRAC1 set SBT_LIBRARY=D:\app\oracle\product\11.2.0\dbhome_1\BIN\oraopc.dll set OPC_PFILE=D:\app\oracle\product\11.2.0\dbhome_1\database\opccnfcrac1.ora  rman> connect target rman>  run { allocate channel t1 device type 'SBT_TAPE' PARMS 'SBT_LIBRARY=D:\app\oracle\product\11.2.0\dbhome_1\BIN\oraopc.dll, ENV=OPC_PFILE=D:\app\oracle\product\11.2.0\dbhome_1\database\opccnfcrac1.ora)'; } Windows Event Viewer:Faulting application name:  ORACLE.EXE, version module Name: oraopc.dll, version code: 0xc0000005Fault offset:  0x00000000000249afFaulting process id: 0x1510Faulting application start tiem: 0x01d0eb098ecd2d2fFaulting application path: d:\app\oracle\product\11.2.0\dbhome_1\bin\ORACLE.EXEFaulting Moduel Path: D:\app\oracle\product\11.2.0\dbhome_1\BIN\oraopc.dll I have made sure that the PATH environment variable has the following entry.  D:\app\oracle\product\11.2.0\dbhome_1\BIN  RAC1 Alert Log Contents:Fri Sep 11 09:04:23 2015Thread 1 advanced to log sequence 11277 (LGWR switch)  Current log# 3 seq# 11277 mem# 0: +REDO/cnfcrac/onlinelog/group_3.258.824214591Fri Sep 11 09:04:31 2015Archived Log entry 23067 added for thread 1 sequence 11276 ID 0x82a6e611 dest 1:Fri Sep 11 09:06:20 2015Reconfiguration started (old inc 16, new inc 18)List of instances: 1 (myinst: 1) Fri Sep 11 09:06:44 2015 Global Resource Directory frozen * dead instance detected - domain 0 invalid = TRUE Communication channels reestablished Master broadcasted resource hash value bitmaps Non-local Process blocks cleaned outFri Sep 11 09:06:44 2015 LMS 0: 1 GCS shadows cancelled, 0 closed, 0 Xw survivedFri Sep 11 09:06:44 2015 LMS 1: 1 GCS shadows cancelled, 1 closed, 0 Xw survived Set master node info Submitted all remote-enqueue requests Dwn-cvts replayed, VALBLKs dubious All grantable enqueues granted Post SMON to start 1st pass IRFri Sep 11 09:06:46 2015Instance recovery: looking for dead threadsBeginning instance recovery of 1 threads Submitted all GCS remote-cache requests Post SMON to start 1st pass IR Fix write in gcs resources RAC 2 Alert Log Contents Fri Sep 11 08:57:47 2015Archived Log entry 23064 added for thread 2 sequence 11793 ID 0x82a6e611 dest 1:Fri Sep 11 09:06:42 2015Starting ORACLE instance (normal)LICENSE_MAX_SESSION = 0LICENSE_SESSIONS_WARNING = 0 Followed by normal start up parameters... I have looked in CRS, ORAAGENT, OCTSSD, and CRSD logs but I don't find anything that stands out as to what may have caused the abnomal termination, other than communciation between the two instances didn't respond in the specified amount of time.    -Jen 
Hi Jen, There is a known issue with Windows module. We are working on fixing it and the new module should be available soon. I will update you as soon as we release the fixed module. Sridhar
Sridhar, Thank you for the reply.  I have an open SR with OS and they explained the same thing to me.  I just tested a beta version yesterday and I am still experiencing issues. -Jenn
Sridhar.R-OracleCan I ask you one question about Cloud backup Modulo on windows env. that matches with this topic?You said there is a know issue on Windows plataform. Is that the reason I cant complete thie set up on Oracle Cloud to use Oracle Cloud Backup? java -jar opc_install.jar -serviceName name -identityDomain name -opcId user -opcPass pass -walletDir C:\app\Administrator\product\11.2.0\dbhome_1\opc\wallet -libDir C:\app\Administrator\product\11.2.0\dbhome_1\opc\lib Oracle Database Cloud Backup Module Install Tool, build 2015-09-05Oracle Database Cloud Backup Module credentials are valid.Oracle Database Cloud Backup Module wallet created in directory C:\app\Administrator\product\11.2.0\dbhome_1\opc\wallet.Oracle Database Cloud Backup Module initialization file C:\app\Administrator\product\11.2.0\dbhome_1\database\opcORCL.ora created. Downloading Oracle Database Cloud Backup Module Software Library from file opc_windows64.zip. Exception in thread "main" java.io.FileNotFoundException: https://storage.us2.oraclecloud.com/v1/odbsdev-usoracleinter76305/oracle-opc-downloads/opc_windows64.zip         at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1835)        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)        at oracle.backup.util.FileDownload.download(FileDownload.java:159)        at oracle.backup.opc.install.OpcConfig.downloadLibrary(OpcConfig.java:362)        at oracle.backup.opc.install.OpcConfig.doOpcConfig(OpcConfig.java:207)        at oracle.backup.opc.install.OpcConfig.main(OpcConfig.java:196)  Tks a lot if this also affects me on this download
I am sorry I worked through this issue and I have not checked back on this post in a bit.  Are you still having issues with getting the module to download?  In our Enterprise environment the trusted sites were set as wildcards *storage.oraclecloud.com and *.oracle.com.  It was determined that he full entry https:// had to be in the trusted sites.  Once this was corrected and my java cacert were updated with the correct information I was able to download just fine. -Jen