EAL and IIS

Oracle Essbase Analytics Link for HFM (EAL) is a pretty cool product.  We use it to move data from HFM to Essbase over EAL bridges.  The EAL bridge defines the source and target applications as well as configuration parameters for the data transfer.  Data can be moved on demand but we can also create a transparent bridge which lets us have near real-time replication of our HFM data into Essbase.

https://www.pinterest.com/pin/100416266663858663/

This comes in handy for reporting purposes.  We use HFM to do the translations and consolidations of our financials and leverage Essbase for reporting.  During the close cycle we don't have to sacrifice resources needed for consolidations with reporting activity.  In addition we can leverage the Essbase calculation engine to add other accounts or dynamic scenarios that we use for reporting but don't need for normal close calculations.  So we let Essbase do the calculations for prior year or budget-to-actual comparisons so that HFM and Financial Reporting don't have to do it.

You can also do some management of the metadata during the transfer.  You can filter dimensions so you only bring over the members you need in Essbase.  You can rename dimensions which can be handy if you have legacy spreadsheets or reports that use different names.  You can also migrate security so that the security on the Essbase application matches the HFM application.

Product description

EAL has a few moving parts.  There is the EAL service (Hyperion Essbase Analytics Link Server - Web Application) that oversees and manages operations.  The Data Synchronization Service (Hyperion Essbase Analytics Link - DSS) is responsible for moving data from HFM to Essbase and does the bulk of the work.  (Usually these two services are installed on a server other than the HFM or Essbase servers so as to not impede performance of other HFM and Essbase activity).  The Financial Management Connector lives on one HFM server to facilitate communication with the HFM applications.  And there is an add-in for Essbase Administration Server (EAS) that is used to configure all of the pieces and create bridges to move data from HFM to Essbase.

There are also some back-end pieces these services need.  A RDBMS database is needed to store the configuration information which can be on either Oracle or MS SQL server.  The Financial Management Connector uses IIS to host a website named livelink that provides access the .dlls required to move the data.  If you are running HFM on Windows 2008 or later then you have to enable IIS 6 compatibility.

Problem and Solution

Our situation was a patch for EAL that was rolled back partway through the install.  When patching EAL the current version has to be completely uninstalled before the new version is installed.  Because the configuration data is held in the database that information is retained so the upgrades are usually painless.

After the rollback we tried to refresh the metadata for an existing bridge.  We use dynamic accounts so I deleted the existing region for the HFM application and created a new region to enable the dynamic accounts.  But when I tried to create the new region I saw this message in the EAS status window:

Data Synchronization Server database cannot be created

This points an an issue with the DSS.  After confirming that DSS was correctly configured and the database was valid we checked the dss.log file and saw:

[23 Nov 2015 16:21:39] [dbmgr] ERROR: HR#07722: Cube 'main_cube' of application 'MyApp' is not registered.
[23 Nov 2015 16:21:51] [dbmgr] ERROR: last message repeated 1 more time
[23 Nov 2015 16:21:51] [dbmgr] WARN : HR#01566: can't open directory "D:\oracle\product\EssbaseAnalyticsLink\oem\hfm\MyApp\Default": The system cannot find the path specified. [3]

But the path did exist.  We tried adjusting permissions but that was no help.  So we checked the eal.log which showed:

[2015-Nov-23 16:24:08] java.lang.OutOfMemoryError Exception [HR#09746]: in jni_GetStringUTFChars (src/jvm/api/jni/jni.c:963).
[2015-Nov-23 16:24:08] Attempting to allocate 256M bytes
[2015-Nov-23 16:24:08] There is insufficient native memory for the Java
[2015-Nov-23 16:24:08] Runtime Environment to continue.
[2015-Nov-23 16:24:08] Possible reasons:
[2015-Nov-23 16:24:08] The system is out of physical RAM or swap space
[2015-Nov-23 16:24:08] In 32 bit mode, the process size limit was hit
[2015-Nov-23 16:24:08] Possible solutions:
[2015-Nov-23 16:24:08] Reduce memory load on the system
[2015-Nov-23 16:24:08] Increase physical memory or swap space
[2015-Nov-23 16:24:08] Check if swap backing store is full
[2015-Nov-23 16:24:08] Use 64 bit Java on a 64 bit OS
[2015-Nov-23 16:24:08] Decrease Java heap size (-Xmx/-Xms)
[2015-Nov-23 16:24:08] Decrease number of Java threads
[2015-Nov-23 16:24:08] Decrease Java thread stack sizes (-Xss)
[2015-Nov-23 16:24:08] Disable compressed references (-XXcompressedRefs=false)

This looked like it was a memory issue so we tried changing some of the options in the EAL configuration tool.  There are also some java heap options in the registry at HKLM\SYSTEM\CurrentControlSet\services\Hyperion Essbase Analytics Link Server - Web Application\Parameters in the string value CmdLine.  None of this helped.

Finally we checked the livelink web site configuration and noticed that it was using the DefaultAppPool.  Ordinarily the default pool enables 32-bit applications.  But EAL requires 64-bit applications so the application pool has to have 32-bit applications disabled.  We already had an application pool specifically for EAL but between the partial upgrade and the roll-back that got reset.  Reconfiguring the LiveLink web site to use the 64-bit application pool resolved the problem.

There are other errors that can occur if you don't disable 32-bit applications for livelink.  Doc ID 1997467.1 reports that you can get this error in the eal.log:

An unknown error has occurred in the HsxAuthentication object.

And who doesn't love an unknown error?

Doc ID 1489138.1 reports that the eal.log will show this error:

Server/Cluster is incorrectly configured. Please reconfigure your Cluster or Server connection.
Server/Cluster is incorrectly configured. Please reconfigure your Cluster or Server connection.

And, yes, the error is duplicated in the log file.

These are different errors but they all have the same resolution which is to create an application pool for EAL, set Enable 32-Bit Applications to false, and assign that app pool to the livelink website.  This is an easy fix and an easy thing to check as you are troubleshooting EAL issues.