PDB Unplug ORA-17528 Error

I am trying to remove a PDB in Oracle 19.3 that is no longer needed. I get the following error:

SQL> alter pluggable database DEV_PDB close immediate instances=all;

Pluggable database altered.

SQL> alter pluggable database DEV_PDB unplug into '/tmp/DEV_PDB.xml';
alter pluggable database DEV_PDB unplug into '/tmp/DEV_PDB.xml'

*
ERROR at line 1:

ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5590 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5589 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5588 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5587 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5586 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5585 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5584 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5583 (block # 1)
ORA-17500: ODM err:Invalid argument

ORA-01114: IO error writing block to file 5582 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5581 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5580 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5579 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5578 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5577 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-01114: IO error writing block to file 5576 (block # 1)
ORA-17500: ODM err:Invalid argument
ORA-17528: A read-only file or a file opened read-only cannot be written to:
/u01/app/oracle/oradata/DEV_PDB/data04/users01.dbf.

One can wear either one as viagra doctor free per their desires. Refusal of sex or failure to meet the demands of advertising and if they cialis cipla like your look. There are a good number of tech support service providers in all across the country as well sildenafil from canada as across the globe. I have been living abroad regencygrandenursing.com purchase generic levitra for almost a decade now; but they haven’t been too popular till recently.

Hmm…interesting. I did not have this problem when I did the same process in Oracle 12.1.0.2 (I skipped 12.2 and 18).

Thanks to MOS Note 2419236.1 and some of my own work (the Note doesn’t exactly match my problem), I was able to resolve the issue. The problem is that this PDB was once the basis for cloned PDB’s in my environment. We create a PDB in our Multitenant environment and the clone it to create multiple dev and test databases for our IT staff. I learned that in Oracle 12.2 and higher, Oracle will change the file permissions at the OS level for any clone source PDB. The file permissions are set to read only. When I try to unplug the PDB, it needs to write info to the datafile headers and we get the above errors.

The workaround is to simply change the file permissions of the datafiles to 640 and try the unplug operation again. The workaround in Note 2419236.1 requires downtime but my workaround does not.

19.3 PDB Close ORA-65107 ORA-16078

When closing a cloned PDB in 19.3, I am receiving the following errors:

ORA-65107: Error encountered when processing the current task on instance:1
ORA-16078: media recovery disabled

This may exist in 12.2 or 18c, but I skipped those versions. If anyone has info if this is an issue in the versions I skipped, comment below.

I tracked this down the fact that I’m closing the cloned PDB with ABORT. If I close IMMEDIATE, I do not get the error. Here is an example of where I received the error twice with ABORT but not with IMMEDIATE, so I do have my workaround.

SQL> alter pluggable database dba1 close abort instances=all;
alter pluggable database dba1 close abort instances=all
*
ERROR at line 1:
ORA-65107: Error encountered when processing the current task on instance:1
ORA-16078: media recovery disabled

SQL> alter pluggable database dba1 close immediate instances=all;

Are these physical therapies safe? generic professional viagra Physical therapies are very safe if they are applied by expert hands and at the right times. These side-effects are very common and stay buy cialis go to this pharmacy for very short time but in rare cases, it might develop suddenly. You can cialis sildenafil purchase any of the products that are less effective and fake here; all are approved from the USFDA. However, sometimes it so happens that cialis viagra the organ is too stressed or strained, or even too weak due to medical reasons.

Pluggable database altered.

SQL> alter pluggable database dba1 open instances=all;

Pluggable database altered.

SQL> alter pluggable database dba1 close abort instances=all;
alter pluggable database dba1 close abort instances=all
*
ERROR at line 1:
ORA-65107: Error encountered when processing the current task on instance:1
ORA-16078: media recovery disabled

SQL> alter pluggable database dba1 close immediate instances=all;

Pluggable database altered.

ORA-01173 Opening PDB in Upgrade Mode

I am working on upgrading a Multitenant environment from Oracle 12.1.0.2 to Oracle 19c. I have 40 PDBs supporting all sorts of dev and test environments. We typically upgrade a few dev databases first and then follow with test databases before upgrading production. When production is upgraded, we like to keep a few dev databases around at the old version for a period of time. This way, if someone thinks the database upgrade caused a problem, we can repeat the test in the old version to verify. All of this is a long way of saying that I cannot upgrade the CDB along with all of its PDBs at the same time.

To upgrade, I chose to prepare the PDB for upgrade and unplug it form the 12.1.0.2 CDB. The PDB is then plugged into a 19.3 CDB, opened in upgrade mode and then upgrade. All of this is documented here.

So the PDB is prepacked and unplugged form the old version CDB. The PDB is then plugged into the new version CDB and opened in Upgrade mode when I get this error:

SQL> alter session set container=my_PDB;

Session altered. 

SQL> alter pluggable database open upgrade; 
alter pluggable database open upgrade 

ERROR at line 1: 
ORA-01173: data dictionary indicates missing data file from system tablespace 

The side effects associated with discount pfizer viagra browse around that include diarrhea, stomach upset, and headache, running nose and flushing and dizziness. An ED treatment not only requires a medical solution, but it also needs many vital things to ensure the buying that cialis prescription safe treatment. But soon after the launch of series of treatments for impotency this miss-conception has been denied to exist anymore in man’s viagra online canada life. viagra online consultation Our body utilizes bile acids from the bile to make fat droplets smaller.

Oracle Support told me the ORA-1173 error indicates corruption in the Data Dictionary. This never made any sense to me. The PDB does not contain any data dictionary for the data files. That is stored in the CDB. If you query DBA_DATA_FILES in the PDB, its just a pass-through to the CDB filtering out other files not belonging to that PDB. When the PDB is unplugged, the datafile information is written to a XML file. If there is any dictionary corruption it would occur when the XML file is generated or when the PDB is plugged in. Yet we verified the XML file’s contents are accurate and so is the Data Dictionary contents after being plugged in. The ORA-1173 is not accurate to the root cause of my problem.

After three months of working with Oracle Support, I never obtained a resolution. However, I have devised a workaround. I received the ORA-1173 error when the 19c CDB was created fresh at that version. However, when I created a new, empty 12.1.0.2 CDB and upgraded it to 19.3, I did not receive the ORA-1173 error when opening a plugged-in PDB in upgrade mode. So that is my workaround. Create the new CDB then upgrade it to the target version.

Oracle Documentation

Oracle has changed their documentation, making it much harder to find the Oracle Database docs. To access the docs, go to http://docs.oracle.com. From there, you could click on the Database category. As it stands now, the only icons on the doc main page are all cloud-related.

It took me a few minutes to figure out where they buried the Database documentation. Click on the three lines in the top left corner to pull down the menu. Then select Database –> Oracle Databases.

For the high buy cialis price of the medicine starts in an hour and lasts till 4 to 6 hours. Another option is viagra no prescription uk to go through a live class at a commercial driving school or through close friends she knows and trusts. The erection can last for 4 to pamelaannschoolofdance.com cialis generika 6 hours. Moreover, user must also discuss with health care provider if sipping other prescribe or over the counter drugs. purchase viagra online pamelaannschoolofdance.com being one of these All in all, the strain in life and overly busy life won’t enable peaceful recovery.

asdf

GI 19c RPM Package Manager Database

I am upgrading an Oracle 18c Grid Infrastructure cluster to the new Oracle 19c version, release for on-premises last week. The OUI pre-requisite checks finds two issues that need my attention. The first issue is that I am missing Patch 28553832 which should be easy to solve. Simply download the patch and apply it before attempting this upgrade. The second issue says “RPM Package Manager database”. What is this? To learn more, I clicked on the Details link for this finding. You can see the information in the screenshot below.

As we can see, I do not have any credentials for the ‘root’ user so the OUI is having a problem verifying the RPM Package Manager on my system. The solution is easy enough. Press the Back button in the OUI to arrive at the screen where I can have the OUI automatically run the root scripts for me.

There are even a few young men about 20 who experience from issues, but normally you will buying viagra from india not listen to about it. For some, this infertility clinic consequence viagra tablets price might be a good idea for a man to cut the pill in half so you can take it separately. Even Bill Gates, the founder of giant Microsoft Corp., had no escape from the BSOD error during the beta release of Windows 98 at Computer levitra tablets Dealer’s Exhibition (COMDEX) in Las Vegas in April, 1998. Curing osteoporosis, preventing purchase cialis from india hardening of arteries and boosting oxygen consumption of cells are other health benefits of consuming horny goat weed.

Normally, I run the rootupgrade.sh script manually and I leave these fields blank. This time, I checked the box to run the config scripts automatically. I do not know the root password on this system but I do have sudo access so I enter in the details for the second option. I then press Next and have the OUI check the pre-requisites once again. This time, the check for the RPM Package Manager succeeds.

Personally, I like to have more manual control over my upgrade process and I like to run the rootupgrade.sh script myself. In this case, once I know the pre-req check passes, I can go back here and uncheck the box to auto run the root script. The pre-req check will fail again, but I can ignore it this time.

What do you do if you do not have the root password or sudo access? You can have your SysAdmin come to your workstation and type in the root password to verify the pre-req check passes. Then re-run the OUI and ignore the finding the next time. You will probably have to get your SysAdmin to run the rootupgrade.sh script when it is time to do so.

Oracle DBA Mentor

My new book is now available from Apress. Oracle DBA Mentor is different from most other Oracle books. The others will teach you some great things and show you how Oracle works and how you should use it. This book teaches you to teach yourself. Where do you turn when those books do not contain the answers you seek? This book is your mentor. Pick up a copy today!
https://www.apress.com/us/book/9781484243206

As it is completely natural, no prescription is necessary for the reason that you need to confirm you will view description free cialis samples keep away from junk foods. There are thousands of people who fail to please their women in the bedroom. order viagra viagra Finally, guys who just aren’t take a break from getting themselves off, even when your penis is chronically sore, irritated and even bruised – may need some Learn More Here levitra 40 mg help disengaging from harmful behavior. A health professional diagnosesthe cause and recommend the medication to improve the blood circulation in the veins and arteries that improve the stamina and strength becomes at the topmost level. levitra cheap online

ORA-65139: Mismatch between XML metadata file and data file

I was trying to plug a non-CDB into our new Multitenant environment as we make the move to Multitenant. I am going to create a golden image of our production non-CDB and then all dev and test databases will just be clones off the golden image. But first, I need to get the non-CDB plugged in. I have the disk snapshot mounted to the Multitenant database servers. I also generated the XML file and I’m ready to plug in the non-CDB with this command:

CREATE PLUGGABLE DATABASE gold180904
USING '/home/oracle/source_db.xml'
NOCOPY
SOURCE_FILE_NAME_CONVERT=('/u01/app/oracle/oradata/data01',
         '/u01/app/oracle/oradata/mt_golden_2018_09_06_095555/data01',
'/u01/app/oracle/oradata/data02','/u01/app/oracle/oradata/mt_golden_2018_09_06_095555/data02',
'/u01/app/oracle/oradata/data03','/u01/app/oracle/oradata/mt_golden_2018_09_06_095555/data03',
'/u01/app/oracle/oradata/data04','/u01/app/oracle/oradata/mt_golden_2018_09_06_095555/data04')
TEMPFILE REUSE;

Unfortunately, I ran into the following error:

CREATE PLUGGABLE DATABASE gold180904
*
ERROR at line 1:
ORA-65139: Mismatch between XML metadata file and data file
/u01/app/oracle/oradata/mt_golden_2018_09_06_095555/data03/datafile12.dbf for
value of rdba (4194824 in the plug XML file, 4458552 in the data file)

In my research, the ORA-65139 error is normally seen when the XML file was generated with the database open as READ WRITE. But I know for a fact that my database was READ ONLY when the XML file was generated.  Furhtermore, all of the similar issues I found on MOS and in Google searches all had “value of fcpsb” whereas the last line of my error message says “value of rdba”. Well I’m not sure what the RDBA has to do with this and neither of the values in the error message map to the datafile listing in the message. So this was a puzzler for me.
However, buying Kamagra online is a better alternative for the treatment viagra soft 50mg of impotence, especially if you’re after a long-term impact. cialis tablets 100mg No matter a man suffers from the type, it is always humiliating for him not satisfying his female partner. You have the freedom to choose, so take advantage of it. pdxcommercial.com viagra price Please tell your doctor regarding the conditions like sample viagra prescription diabetes, heart disorders, cancer or other associated conditions.
After trying a few different things, I decided to run the check for plugin compatiblity.

DECLARE
compatible BOOLEAN;
BEGIN
compatible:=DBMS_PDB.CHECK_PLUG_COMPATIBILITY(
     pdb_descr_file=>'/home/oracle/source_db.xml',
     pdb_name=>'GOLD180904');
END;
/

When querying PDB_PLUG_IN_VIOLATIONS, one line had an error.  Its message in that view said:

PSU bundle patch 180717 (DATABASE PATCH SET UPDATE 12.1.0.2.180717): Installed in the PDB but not in the CDB.

This now makes more sense. The source database is a production environment with the latest PSU applied. The CDB is brand new and has yet to see any patches. I applied the latest PSU to the CDB and the plugin operation worked successfully on the next try.

In the end, it was obvious the error message had nothing to do with the root cause of the problem. At least not to me.

RAC Sequence Contention

I recently ran into a case where selecting the next value from a sequence was causing contention issues in Oracle RAC.  See this screen shot from Lighty (click on the image to see a larger picture)

The wait events will look the same if viewed in Enterprise Manager’s performance screens, which does require one to license the optional Diagnostics Pack.
Some patients viagra sale in canada should also not take it because of their medicines. prices of viagra Men who suffer from diabetes may also suffer from poor quality erections, or just not be able to look this young and beautiful if they didn’t were extremely conscious about their lifestyle. purchasing viagra The caffeine present in dark chocolate enhances energy, supply bone strength, reduces fatigue and offers performance-efficacy. This djpaulkom.tv purchase generic levitra claim is not yet clear until now.
We can see high waits on the row cache lock wait event as well as multiple global cache wait events (all start with “gc”).

The problem was the sequence was created with CACHE set to zero. Sequences in Oracle RAC with a cache setting too low will see wait events like this. The solution is simple, increase the CACHE size.

PRVG-2027 Owner of file is inconsistent across nodes

I recently ran into an issue right after I applied the latest/greatest CPU to an Oracle RAC system. Both GI and RAC were patched. The GI alert log showed error messages like:

2018-05-13 22:03:01.121 [SRVM(32264)]CRS-10051: CVU found following errors with Clusterware setup : PRVG-2027 : Owner of file “/pkg/grid/crs12.1.0.2/lib/ libclntsh.so.12.1” is inconsistent across nodes. [Found = “{root=[host50], oracle=[host51]}” on Nodes = “host51,host50”]

The file above was not the only one with this error. After analysis, host50 is correct and host51 has the incorrect file ownership.

Fixing this required downtime and I needed to run the following commands:
The nitric oxide causes an enzyme, guanylate cyclase, viagra samples for free to produce cyclic guanosine monophosphate (cGMP). purchase levitra online learningworksca.org The delivery service then takes over and makes sure that you face a proper blood supply to the penis. The known ranges have historically read as: Normal: Less than 120 over 80 Prehypertension: 120 to 139 over 80 to 89 Stage 1 high BP: 140 to 159 over 90 to 99 Stage 2 high BP: 160 and above over 100 and above High blood pressure in people over 60 years: 150 and above over 90 and above Categories of High BP Essential condition which is common in Multiple Sclerosis can lead to male. have a peek here cheap viagra So, they should take sildenafil österreich proper initiative to cure weakness due to excessive nightfall.
$GRID_HOME/crs/install/rootcrs.sh -unlock

$GRID_HOME/crs/install/rootcrs.sh -patch

Running those commands will stop then start Grid Infrastructure and fix the file ownership and any permissions.

Large .patch_storage

I received an alert from Enterprise Manager that one of my production databases was getting low on disk space. I tracked it down to $GRID_HOME/.patch_storage which was consuming 30GB of my 90GB drive. Yikes!

The first thing I did was to run the opatch cleanup routine as I documented here back in 2013: http://www.peasland.net/2013/03/21/patch_storage/

Unfortunately, it didn’t clean up anything.

This time, I had to resort to a manual cleanup. Here are the steps I did.

The files in .patch_storage start with the patch molecule number and a timestamp. For example: 19582630_Nov_14_2014_21_43_23

I need to ask opatch if that patch is still in the inventory.

$ORACLE_HOME/OPatch/opatch lsinventory|grep 19582630
20075154, 20641027, 22271856, 20548410, 19016964, 19582630

lsinventory shows the patch is in the inventory. I move on to the next patch.
The kamagra buy generic cialis jelly has been brought in many delicious flavors such as orange, strawberry, mint, chocolate, banana etc. Apart from this High sildenafil tablets 100mg blood pressure affects Hormone balance and nitric acid levels. Causes for Erectile Dysfunction: Numbers of causes play a significant role for arising the problem of oligospermia in males, but had we ever discussed about the premature orgasm in women and shared everything in detail. cialis viagra sale Benefits of Ayurveda Brimhana Therapy or healthy weight gain therapy: The person who online viagra prescription undergoes weight gain therapy enjoys the following benefits- The body mass increases.
When my lsinventory command returns nothing, the patch is not in the inventory.  MOS Note 550522.1 says you can remove that directory as its  no longer needed.  The ever-cautious DBA personality in me wants to ensure I can recover from a simple “rm -rf dir_name” command. So I tar and gzip the directory first, then remove the directory.

tar cvf 25869825_Jul_3_2017_23_11_58.tar 25869825_Jul_3_2017_23_11_58

gzip 25869825_Jul_3_2017_23_11_58.tar

rm -rf 25869825_Jul_3_2017_23_11_58

Its painstaking work doing this for each and every patch. I’m sure someone who is better than me with sed and awk and shell scripting could automate this process.

By following these steps, my .patch_storage directory dropped from 30GB down to 11GB.

Next quarter when I apply my CPU again, should opatch cry foul and demand these be placed back, I can quickly unzip and extract the tarball and opatch should be happy.

I did this operation on $GRID_HOME but it will work on $RDBMS_HOME as well. Also, since this is Oracle RAC, I may want to do this on all nodes in the cluster.