 |
| https://ww.deluxe.com/blog/using-facebook-insights-small-business/ |
In this case we were applying the latest Patch Set Updates (PSU) for our Financial Reporting/Reporting and Analysis/BIPlus applications. One of the wrinkles with PSUs, particularly those for BIPlus, is that you have to redeploy the application with the EPM System Configurator/Config Tool. One of the other wrinkles with EPM products is that they are frequently bolted on to the rest of the suite and will go by many names when you try to research a problem.
After successfully running OPatch to update the product I ran configtool.bat. But the list of available products to configure only showed the basic Foundation Services. None of the other products, including the Reporting and Analysis Framework that we just patched were available.
Fortunately this was in our QA environment so there was no chance of causing a production outage. Also fortunately, and oddly, the services were still visible in the Windows Services console. And the OPatch for the PSU completed correctly which confirmed that the required file structure was still in place.
Perhaps the other products needed to be reinstalled? But when I ran the InstallTool.cmd the only products available to re-install were the same Hyperion Foundation products that we saw in ConfigTool.
Research
I opened an SR with Oracle to help me determine the cause. We ran the usual diagnostics and checked files related to inventory such as the HYPERION_HOME/inventory/contentsXML/comps.xml and EPM_INSTANCE/config/foundation/11.1.2.0/reg.properties files. But that didn't show us anything useful.
The Oracle SE hit on the idea of using ProcMon to watch what files were being processed. This pointed us to the HYPERION_HOME/.oracle.products file as the file referenced during configuration. After comparing the .oracle.products file to our production server we confirmed that the file in our QA environment got damaged somewhere along the line.
Solution
The .oracle.products file isn't used for normal operations. It is just referenced and updated during installation and configuration. So the file could be damaged for months or years and you would never notice until you run the configurator or try to install or uninstall products.
One solution was to backup the existing .oracle.products file, remove all of the <InstallabelItem> and <products> nodes from the XML file, then run InstallTool and reinstall the required products. Because this would overwrite all of the existing .ear and .jar files all previous patches would also need to be re-applied.
While this is a reasonable solution it involves several manual steps which can be prone to error. And it can be time consuming to reapply all of the patches.
The other solution was simple and brilliantly elegant. Just take a clean server and run InstallTool to install the products that are on the damaged server, then copy the .oracle.products file from the temporary server to the damaged server.
I used VirtualBox to quickly spin up a virtual machine to handle this task. Note that all I needed to do is install the products to generate the .oracle.products file. There was no need to worry about connecting to an existing WebLogic instance, or accessing databases, or any of the other configuration tasks. Plus I already had a server image I could deploy in VirtualBox so I completed the whole process in an hour.
Once the .oracle.properties file was copied I was able to successfully complete the redeploy and confirm that the PSU fixed the issue we were facing.
Takeaways:
1. ProcMon is a useful tool for finding missing or damaged files and registry keys
2. VMs are useful for troubleshooting and problem resolution. Everybody gives away hypervisors so be sure to keep something handy for when you need it.
3. The .oracle.products file is used by InstallTool and ConfigTool to determine what is installed and configurable on the server.
No comments:
Post a Comment