Sean's Musings as a Service

Sean's Musings as a Service

Upgrading UrbanCode when using a load balancer for webhost

  • Published:
  • categories: ibm
  • tags: urbancode, deploy, upgrade

When trying to upgrade a server with a client using F5 the other day I ran into this error seemed to be the same for 6.1.1.7 and 6.1.1.8, this is the error message from 6.1.1.7

Short version:

 A previously installed version of IBM UrbanCode Deploy is running

Full stack trace:

[echo] Do you accept the license? [y,n]
y

[echo]
[echo] Installing IBM UrbanCode Deploy version 6.1.1.7.683647
[echo] Enter the directory where the IBM UrbanCode Deploy should be installed. [Default: C:\Program Files\ibm-ucd\server]

D:\IBM\UCD\Server

: A previously installed version of IBM UrbanCode Deploy is running. Please shut down the running IBM UrbanCode Deploy and start the installation again.

       at org.apache.tools.ant.taskdefs.Exit.execute(Exit.java:142)
       at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
       at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
       at java.lang.reflect.Method.invoke(Unknown Source)
       at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
       at groovy.util.AntBuilder.performTask(AntBuilder.java:260)
       at groovy.util.AntBuilder.nodeCompleted(AntBuilder.java:220)
       at groovy.util.BuilderSupport.doInvokeMethod(BuilderSupport.java:147)
       at groovy.util.AntBuilder.doInvokeMethod(AntBuilder.java:170)
       at groovy.util.BuilderSupport.invokeMethod(BuilderSupport.java:64)
       at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:45)
       at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:116)
       at ContainerInstaller.checkForUpgrade(ContainerInstaller.groovy:1052)
       at ContainerInstaller.this$2$checkForUpgrade(ContainerInstaller.groovy)
       at ContainerInstaller$this$2$checkForUpgrade.callCurrent(Unknown Source)
       at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent (CallSiteArray.java:46)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
       at ContainerInstaller.installServer(ContainerInstaller.groovy:310)
       at ContainerInstaller.install(ContainerInstaller.groovy:251)
       at ContainerInstaller$install.call(Unknown Source)
       at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
       at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
       at embedded_script_in_c__tmp_ibm_ucd_install_install_dot_with_dot_groo vy_dot_xml.run(embedded_script_in_c__tmp_ibm_ucd_install_install_dot_with_dot_groovy_dot_xml:6)
       at org.codehaus.groovy.ant.Groovy.parseAndRunScript(Groovy.java:440)
       at org.codehaus.groovy.ant.Groovy.execGroovy(Groovy.java:414)
       at org.codehaus.groovy.ant.Groovy.execute(Groovy.java:280)
       at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
       at java.lang.reflect.Method.invoke(Unknown Source)
       at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
       at org.apache.tools.ant.Task.perform(Task.java:348)
       at org.apache.tools.ant.Target.execute(Target.java:357)
       at org.apache.tools.ant.Target.performTasks(Target.java:385)
       at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1337)
       at org.apache.tools.ant.Project.executeTarget(Project.java:1306)
       at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
       at org.apache.tools.ant.Project.executeTargets(Project.java:1189)
       at org.apache.tools.ant.Main.runBuild(Main.java:758)
       at org.apache.tools.ant.Main.startAnt(Main.java:217)
       at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
       at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)

    [echo] Install Failed. (press return to exit installer)

One good thing about the current UrbanCode Deploy installer is that it is written in Groovy so you can just follow the error message in the stack trace back to the source. So inside the .../ibm-ucd/install/install/ContainerInstaller.groovy file we find two places where this specific error is generated, around line 1013

if (new File(installDir + '/var/server.pid').exists()) {
    ant.fail('A previously installed version of ${component.name} is running. ' +
            'Please shutdown the running ${component.name} and start the installation again.')
}

or line 1047

if (webHost && port) {
    try {
        def clientSocket = new Socket(webHost, port)
        clientSocket.close()

        ant.fail('A previously installed version of ${component.name} is running. ' +
                'Please shutdown the running ${component.name} and start the installation again.')
    }

The first one is pretty straight forward, confirm that your service is not actually running and cleanup that file. The second is less obvious because it uses a successful test to assert failure, meta as it is, it makes sense by saying “If I can connect to the server, this is a failure” we get a logical assumption that the server must be running and that we fail out. But it pulls the existing webHost and port from the installed.properties file and attempts to bind to it, if that is successful the installer thinks your system is still running and fails out. The problem here is when webHost is pointing to a load balancer, in that case it is probably pretty likely that this connection will not fail since this server/appliance is still running and even though it cannot forward it to the backend services it will probably still accept the iniital connection.

So how do we get around this, there are a few ways, as part of your upgrade process:

  1. Mark your load balancer url as offline/down, so it does not accept connections. This is not as easy for some teams to do or worth raising the service requests through the network teams for a 15 minute outage.
  2. Modify the servers local machine so that webHost will not connect.
  • Change /etc/hosts file so that webHost resolves to localhost and essentially shortcuts any external calls at the dns level (this will not work if your host happens to be an ip address)
  • Add a firewall rule so we can’t connect to the load balancer ( in this case the error, downgrades to a warning and you will be asked to confirm you have stopped the server, but can continue)
  1. Modify the value in your local installed.properties file to change the value of install.server.web.host to localhost or to the specific server you are upgrading.
  2. Modify ContainerInstaller.groovy to shortcut webHost to always be localhost

I put them in the order I would recommend, but any of them will do. I have also filed a defect with the UrbanCode Deploy team to make this error message more clear as to why this is failing, and potentially downgrade this from a failure to a warning so there is actually no action required.

Reference: