ETL: Scheduled Jobs Failing

This requires Admin or Data Admin privileges for implementation.

Please Note: If you do not have the correct privileges to access Data Hub you will receive the following upon your login attempt. If you get this message and think it is an error please see your organization's iDashboards Administrator for assistance as Support is unable to give you access.

There are a couple possible causes for failing Scheduled ETL jobs. The most common cause is that there was not enough memory allotted to the software, more specifically Java to complete the job. What happens in this case is there is a java heap space error and it crashes Tomcat/iDashboards service, which is usually set to automatically restart so other jobs may continue to run.

What to look for...

Check the idbdata.log files, through Data Hub

• Go to System
• Select Logs
• On the left set it to Debug
• Click Save
• Then download the log file and open in a text editor

On the server

• C:\Program Files\iDashboards_SRV\ivizgroup\logs
• Then open the log file in a text editor.

Look for the following, it will likely be nearby the name of the failed ETL job:

java.lang.OutOfMemoryError: Java heap space

What to do about it...

On the server open the Java options console by navigating to:

C:\Program Files\iDashboards_SRV\Server\bin

Then double click iDashboardsW.exe

If this does not work then:

1. Open Command Prompt As Administrator
   
2. Type the following:
   cd C:\Program Files\iDashboards_SRV\Server\bin
   
3. Hit Enter
   
4. Type the following:
   idashboardsw //ES//iDashboardsSRV
   
5. Hit Enter
   
6. Then the box should pop up
7. Select Java
8. Edit the settings as necessary for you environment. For this issue we recommend following:
   • Scroll down to the Java Options:
   • Inside the box, scroll down until you see the -Xms/-Xmx option.
     • Nothing needs to be done with: -Xms which determines the minimal RAM that will be used.
     • Set -Xmx to suit your ETL usage: we have 4G, alternatively you can remove the -Xmx line which will allow access to all available memory.
     • Click Apply then OK
9. Restart the iDashboards Service (iDashboardsSRV) for the change to take effect.

If you are still receiving the heap space error after increasing or removing the -Xmx line you may need to increase the overall memory on the server.

Possible Cause #2

This cause was reported by a customer. They found that scheduled ETL jobs we're failing whenever there we're pending Windows Updates on the server. We have not verified this correlation, however there is a lot of information online of people reporting pending updates causing a variety of issues from affecting software to performance issues. If you find this is the reason your Scheduled ETL jobs are failing. You can turn off automatic updates and check for and run updates manually. 

For More Information:

• ETL: Job Optimization
• Java: Out of Memory Error - System Unavailable
• iDashboards Data Hub Manual 8. Extract, Transform and Load
• Java Documentation: 3.2 Understand the OutOfMemoryError Exception

If the above is unable to resolve the issue, then please contact iDashboards Support for further assistance.