Unknown error 1053 starting hpdp-as

As I mentioned in my post about 1053 error for hpdp-idp-cp ( http://blog.ifost.org.au/2014/06/unknown-error-1053-starting-hpdp-idp-cp.html ), error code 1053 on Linux doesn't have any particular meaning. It's just a catch-all to say "something went wrong during service startup".

I've experienced this twice now, and both times it was for different reasons.

Here's the output from omnisv status...

    ProcName      Status  [PID]    
===============================
    crs         : Active  [26506]
    mmd         : Active  [26504]
    kms         : Active  [26505]
    hpdp-idb    : Active  [26466]
    hpdp-idb-cp : Active  [26499]
    hpdp-as     : Down
    omnitrig    : Active
    Sending of traps disabled.
===============================

With a bit of inspired guessing, hpdp-as is supposed to be started by /etc/rc.d/init.d/hpdp-as. Despite what it looks like, this isn't actually used as SYSV init runscript -- it is invoked by /opt/omni/sbin/omnisv start. This in turn is invoked by /etc/rc.d/init.d/omni, which actually is a SYSV init runscript.

/etc/rc.d/init.d/hpdp-as isn't part of any RPM file; neither are any of the other Data Protector start up scripts.

• /etc/init.d/omni is installed by the OB2-CS post-install scriptlet.
• /etc/init.d/hpdp-as is created by the IDBsetup.sh script when it calls an internally-defined updateServices function

The first time I encounted "Unknown error 1053", it was simply because something had gone wrong during installation, and /etc/init.d/hpdp-as wasn't created. I just took the code from IDBsetup.sh (search for hpdp-as and you'll find an init script inside a heredoc) and recreated it. Then I checked that the appropriate /etc/services entry had been created ( "hpdp-idb-as 7116/tcp"  )

If this happens to you, here's a handy /etc/init.d/hpdp-as for reference. Just change lnx.ifost.org.au to whatever your cell manager's hostname is:

#!/bin/sh
# chkconfig: 35 99 08
# description: HP Data Protector Application Server.
# processname: hpdp-as

### BEGIN INIT INFO
# Provides: hpdp-as
# Required-Start: $local_fs $remote_fs $network $syslog
# Required-Stop: $local_fs $remote_fs $network $syslog
# Default-Start: 3 5
# Default-Stop: 0 1 2 4 6
# Short-Description: HP Data Protector Application Server
### END INIT INFO

#Defining AS_HOME
AS_HOME=/opt/omni/AppServer

case "$1" in
start)
echo "Starting the HP Data Protector Application Server..."
nohup su - hpdp -c "${AS_HOME}/bin/standalone.sh -b lnx.ifost.org.au &"
;;
quick)
nohup su - hpdp -c "${AS_HOME}/bin/standalone.sh -b lnx.ifost.org.au > /dev/null &"
;;
stop)
echo "Stopping the HP Data Protector Application Server..."
su - hpdp -c "${AS_HOME}/bin/jboss-cli.sh --connect command=:shutdown"
;;
log)
echo "Showing server.log..."
tail -1000f /var/opt/omni/log/AppServer/server.log
;;
*)
echo "Usage: /etc/init.d/hpdp-as {start|stop|log}"
exit 1
;; esac
exit 0

The second time I encountered this (which was today), /etc/init.d/hpdp-as was present. A bit of digging into it revealed that it calls /opt/omni/AppServer/bin/standalone.sh -b cell-manager-hostname as the user hpdp (or whatever you are running Data Protector as). Helpfully, standard output is redirected to /dev/null, so whatever errors you might encounter are not reported anywhere.

When I ran that manually I saw (buried in the standard output which would otherwise have been discarded):

15:35:14,505 ERROR [org.jboss.msc.service.fail] MSC00001: Failed to start service jboss.logging.handler.FILE: org.jboss.msc.service.StartException in service jboss.logging.handler.FILE: java.io.FileNotFoundException: /var/opt/omni/log/AppServer/server.log (Permission denied)

And indeed, /var/opt/omni/log/AppServer is owned by root, and unwritable by hpdp-as.

[hpdp@cellmgr AppServer]$ ls -ld /var/opt/omni/log/AppServer

drwxr-xr-x. 2 root root 21 Sep 16 22:40 /var/opt/omni/log/AppServer

Various other problems crop up, all to do with permissions. As far as I can tell, the following chown command will fix all of them. 

sudo chown hpdp \
  /var/opt/omni/log/AppServer \
  /var/opt/omni/server/AppServer  \
  /opt/omni/AppServer/standalone/deployments  \ 
  /etc/opt/omni/server/AppServer

sudo /opt/omni/sbin/omnisv start

Greg Baker is an independent consultant who happens to do a lot of work on HP DataProtector. He is the author of the only published book on HP Data Protector (http://x.ifost.org.au/dp-book). He works with HP and HP partner companies to solve the hardest big-data problems (especially around backup). See more at IFOST's DataProtector pages at http://www.ifost.org.au/dataprotector