Sean's Musings as a Service

Sean's Musings as a Service

Getting started with UrbanCode Deploy with Patterns 6.1.1.1 on Ubuntu 14.04

  • Published:
  • categories: ibm
  • tags: ubuntu, ibm, urbancode, deploy, openstack, patterns, deployment-automation

Getting your own private cloud is cool, connecting it to UrbanCode Deploy is pretty cool too, but adding a way to perform full stack deployments of complete environments is awesome. This article is a helper to get up and running, from here the sky is the limit to start talking about how you push your unit of deployment up from a code changes and configurations changes to talking complete environment deployments or updates to help improve quality of deployments.

UrbanCode Deploy with Patterns(UCDP) is an extention to UrbanCode Deploy(UCD) that allows for the creation and cultivation of application topolgies as patterns that can be instatiated and updated on managed infrastructure as full-stack environments. That is a lot to take in, but it boils down to a gui interface to compose application topologies from re-usable components that are a mix of capabilities from OpenStack/VSphere and UrbanCode Deploy Components, see the following 10 minute demo1 by Michael Elder, architect and Tech Lead for UCDP, for an introduction:

Pre-Requisites

1. OpenStack

In this case I’ll get a UCDP instance up and running on Ubuntu 14.04 using an all in one OpenStack instance installed using the instructions by stackgeek2. I’m not an OpenStack savvy user at this point so I found Kord Cambell’s instructions, video, and helper scripts were useful to get from a bare metal machine to a working install in an hour or so if you don’t run into any false starts, I recommend that you check it out. At this point UCDP will only support OpenStack IceHouse or Juno3 so a current devstack on 2015.x may not work if you are just playing around, I have not tried that yet to confirm or deny, what I did was install IceHouse per the article once I get a bit more comfortable I’ll look at upgrading or migrating to Juno.

So we have an OpenStack instance running, keystone is present, we have some images installed, and have verified that we can provision an image connect to it and it appears to behave correctly.

2. Install UrbanCode Deploy

This process is pretty straight forward and I already wrote about it a few years back now here, while the features and product has improved the install process is still very straight forward and easy to do. If you already have one, just make sure it matches the level of your UCDP they are highly dependent.

3. UCDP Server

There are some OS specific requirements to get out of the way, they are only listed in yum commands so I tried to do the translation.

yum install -y gcc gcc-c++ kernel-devel mysql-devel python-devel

Translated to apt packages:

sudo apt-get install -y  gcc g++ linux-headers-generic libmysqld-dev python-dev

Install UrbanCode Deploy with Patterns

Ok, so Ubuntu may not be officially supported, hence the reason for this post, it is reasonable to install on Ubuntu but IBM is working to limit testing scope and RHEL-ish support is easier to contain, so in either case we’re going to push forward as Ubuntu is my preference. That being said, the instructions on the Info Center are probably sufficient to get up and running on RHEL 6/7, CentOS 6/7, Fedora 20/21 and are based on yum commands as are the installation scripts for now.

For Ubuntu and other Debian based systems we translate that to the corresponding packages and apt-get commands. In this case we already have a HEAT Engine installed(our OpenStack server) and running so we will extend and existing engine4. This is starting to get into the nuts and bolts of what UCDP does and how it works but it is essentially a GUI interface that can be used to manage and edit the production of HOT Templates for HEAT compliant endpoints. First you can check the available templates, these are the defaults for IceHous:

sgwilbur@lappy:~/workspace/ucdp-installation$ heat resource-type-list
+------------------------------------------+
| resource_type                            |
+------------------------------------------+
| AWS::AutoScaling::AutoScalingGroup       |
| AWS::AutoScaling::LaunchConfiguration    |
| AWS::AutoScaling::ScalingPolicy          |
| AWS::CloudFormation::Stack               |
| AWS::CloudFormation::WaitCondition       |
| AWS::CloudFormation::WaitConditionHandle |
| AWS::EC2::EIP                            |
| AWS::EC2::EIPAssociation                 |
| AWS::EC2::Instance                       |
| AWS::EC2::InternetGateway                |
| AWS::EC2::NetworkInterface               |
| AWS::EC2::RouteTable                     |
| AWS::EC2::SecurityGroup                  |
| AWS::EC2::Subnet                         |
| AWS::EC2::SubnetRouteTableAssociation    |
| AWS::EC2::VPC                            |
| AWS::EC2::VPCGatewayAttachment           |
| AWS::EC2::Volume                         |
| AWS::EC2::VolumeAttachment               |
| AWS::ElasticLoadBalancing::LoadBalancer  |
| AWS::IAM::AccessKey                      |
| AWS::IAM::User                           |
| AWS::RDS::DBInstance                     |
| AWS::S3::Bucket                          |
| OS::Ceilometer::Alarm                    |
| OS::Ceilometer::CombinationAlarm         |
| OS::Cinder::Volume                       |
| OS::Cinder::VolumeAttachment             |
| OS::Heat::AccessPolicy                   |
| OS::Heat::AutoScalingGroup               |
| OS::Heat::CWLiteAlarm                    |
| OS::Heat::CloudConfig                    |
| OS::Heat::HARestarter                    |
| OS::Heat::InstanceGroup                  |
| OS::Heat::MultipartMime                  |
| OS::Heat::RandomString                   |
| OS::Heat::ResourceGroup                  |
| OS::Heat::ScalingPolicy                  |
| OS::Heat::SoftwareConfig                 |
| OS::Heat::SoftwareDeployment             |
| OS::Heat::StructuredConfig               |
| OS::Heat::StructuredDeployment           |
| OS::Heat::UpdateWaitConditionHandle      |
| OS::Neutron::Firewall                    |
| OS::Neutron::FirewallPolicy              |
| OS::Neutron::FirewallRule                |
| OS::Neutron::FloatingIP                  |
| OS::Neutron::FloatingIPAssociation       |
| OS::Neutron::HealthMonitor               |
| OS::Neutron::IKEPolicy                   |
| OS::Neutron::IPsecPolicy                 |
| OS::Neutron::IPsecSiteConnection         |
| OS::Neutron::LoadBalancer                |
| OS::Neutron::MeteringLabel               |
| OS::Neutron::MeteringRule                |
| OS::Neutron::Net                         |
| OS::Neutron::NetworkGateway              |
| OS::Neutron::Pool                        |
| OS::Neutron::PoolMember                  |
| OS::Neutron::Port                        |
| OS::Neutron::ProviderNet                 |
| OS::Neutron::Router                      |
| OS::Neutron::RouterGateway               |
| OS::Neutron::RouterInterface             |
| OS::Neutron::SecurityGroup               |
| OS::Neutron::Subnet                      |
| OS::Neutron::VPNService                  |
| OS::Nova::FloatingIP                     |
| OS::Nova::FloatingIPAssociation          |
| OS::Nova::KeyPair                        |
| OS::Nova::Server                         |
| OS::Swift::Container                     |
| OS::Trove::Instance                      |
+------------------------------------------+
sgwilbur@lappy:~/workspace/ucdp-installation$ heat resource-type-list|grep IBM

In the install I did, the HEAT engine was not installed, so I added it in and got to the point where I could do a heat resource-type-list command successfully by following the OpenStack instructions for IceHouse on Ubuntu. At this point we aready to add some new types here that are used by UCDP, check what user Heat is running as and assign the appropriate permissions before restarting.

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ sudo grep plugin_dir /etc/heat/heat.conf
#plugin_dirs=/usr/lib64/heat,/usr/lib/heat
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ ls -als /usr/lib/heat /usr/lib64/heat
ls: cannot access /usr/lib/heat: No such file or directory
ls: cannot access /usr/lib64/heat: No such file or directory
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ sudo mkdir /usr/lib//heat	
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ sudo tar xvzf resoruces/ibm-ucd-patterns-heat-plugins-6111.tgz -C /usr/lib/heat/
ibm-cloud-ext/
ibm-cloud-ext/resources/
ibm-cloud-ext/resources/aws/
ibm-cloud-ext/resources/softlayer/
ibm-cloud-ext/resources/vmware/
ibm-sw-orch/
ibm-sw-orch/heat/
ibm-sw-orch/heat/debug_utils/
ibm-sw-orch/heat/ucd/
ibm-cloud-ext/README.txt
ibm-cloud-ext/__init__.py
ibm-cloud-ext/requirements.txt
ibm-cloud-ext/resources/__init__.py
ibm-cloud-ext/resources/aws/__init__.py
ibm-cloud-ext/resources/aws/boto_utils.py
ibm-cloud-ext/resources/aws/ec2_auto_scaling_group.py
ibm-cloud-ext/resources/aws/ec2_auto_scaling_policy.py
ibm-cloud-ext/resources/aws/ec2_base.py
ibm-cloud-ext/resources/aws/ec2_cloudwatch_alarm.py
ibm-cloud-ext/resources/aws/ec2_eip.py
ibm-cloud-ext/resources/aws/ec2_internet_gateway.py
ibm-cloud-ext/resources/aws/ec2_launch_config.py
ibm-cloud-ext/resources/aws/ec2_port.py
ibm-cloud-ext/resources/aws/ec2_router.py
ibm-cloud-ext/resources/aws/ec2_security_group.py
ibm-cloud-ext/resources/aws/ec2_server.py
ibm-cloud-ext/resources/aws/ec2_subnet.py
ibm-cloud-ext/resources/aws/ec2_volume.py
ibm-cloud-ext/resources/aws/ec2_volume_attachment.py
ibm-cloud-ext/resources/aws/ec2_vpc.py
ibm-cloud-ext/resources/aws/elasticache_cluster.py
ibm-cloud-ext/resources/aws/elasticache_parameter_group.py
ibm-cloud-ext/resources/aws/elasticache_replication_group.py
ibm-cloud-ext/resources/aws/elasticache_subnet_group.py
ibm-cloud-ext/resources/aws/rds_instance.py
ibm-cloud-ext/resources/aws/rds_parameter_group.py
ibm-cloud-ext/resources/aws/rds_subnet_group.py
ibm-cloud-ext/resources/aws/route53_hosted_zone.py
ibm-cloud-ext/resources/aws/route53_record_set.py
ibm-cloud-ext/resources/aws/s3_bucket.py
ibm-cloud-ext/resources/everglade_secrets.py
ibm-cloud-ext/resources/ibm-cloud-discovery.conf
ibm-cloud-ext/resources/resource_extender.py
ibm-cloud-ext/resources/softlayer/__init__.py
ibm-cloud-ext/resources/softlayer/hello_world.yaml
ibm-cloud-ext/resources/softlayer/softlayer_base.py
ibm-cloud-ext/resources/softlayer/softlayer_compute.py
ibm-cloud-ext/resources/softlayer/softlayer_port.py
ibm-cloud-ext/resources/vmware/__init__.py
ibm-cloud-ext/resources/vmware/basic_template.yaml
ibm-cloud-ext/resources/vmware/userdata.iso
ibm-cloud-ext/resources/vmware/vcenter_base.py
ibm-cloud-ext/resources/vmware/vcenter_network.py
ibm-cloud-ext/resources/vmware/vcenter_port.py
ibm-cloud-ext/resources/vmware/vcenter_server.py
ibm-cloud-ext/resources/vmware/vcenter_subnet.py
ibm-sw-orch/__init__.py
ibm-sw-orch/heat/__init__.py
ibm-sw-orch/heat/debug_helper.py
ibm-sw-orch/heat/debug_utils/__init__.py
ibm-sw-orch/heat/debug_utils/debug_helper.py
ibm-sw-orch/heat/sw_config.py
ibm-sw-orch/heat/sw_deploy.py
ibm-sw-orch/heat/ucd/__init__.py
ibm-sw-orch/heat/ucd/ucd_resource_tree.py
ibm-sw-orch/heat/ucd/ucd_sw_config.py
ibm-sw-orch/heat/ucd/ucd_sw_deploy.py
ibm-sw-orch/heat/ucd_client.py
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ ps aux|grep heat
heat     26571  0.0  0.1 149332 46828 ?        S    04:00   0:00 /usr/bin/python /usr/bin/heat-api
heat     26587  0.0  0.1 241972 54980 ?        S    04:00   0:01 /usr/bin/python /usr/bin/heat-engine
sgwilbur 32126  0.0  0.0  10468   944 pts/0    S+   05:55   0:00 grep --color=auto heat
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ sudo groups heat
heat : heat
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ sudo chown -R heat:heat /usr/lib/heat
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ alias|grep heat
alias start-heat='sudo /etc/init.d/heat-api start;sudo /etc/init.d/heat-api-cfn start;sudo /etc/init.d/heat-engine start'
alias stop-heat='sudo /etc/init.d/heat-api stop;sudo /etc/init.d/heat-api-cfn stop;sudo /etc/init.d/heat-engine stop'
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ stop-heat
 * Stopping Heat API heat-api
   ...done.
 * Stopping Heat API heat-api-cfn
start-stop-daemon: warning: failed to kill 26579: No such process
   ...done.
 * Stopping Heat Engine heat-engine
   ...done.
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ ps aux|grep heat
sgwilbur 32229  0.0  0.0  10464   944 pts/0    S+   05:57   0:00 grep --color=auto heat
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ start-heat
 * Starting Heat API heat-api
   ...done.
 * Starting Heat API heat-api-cfn
   ...done.
 * Starting Heat Engine heat-engine
   ...done.
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ heat resource-type-list

ERROR: Timeout while waiting on RPC response - topic: “engine”, RPC method: “list_resource_types” info: “

But this was not working as expected, I can’t get the ressource-type-list to verify they new templates! Opening up /var/log/heat/heat-engine.log` I see what looks like a problem, we’re missing a module?

015-04-04 06:04:24.929 32732 ERROR heat.common.plugin_loader [-] Failed to import module heat.engine.plugins.ibm-cloud-ext.resources.softlayer.softlayer_compute
2015-04-04 06:04:24.929 32732 CRITICAL heat [-] ImportError: No module named IPy
2015-04-04 06:04:24.929 32732 TRACE heat Traceback (most recent call last):
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/bin/heat-engine", line 65, in <module>
2015-04-04 06:04:24.929 32732 TRACE heat     srv = engine.EngineService(cfg.CONF.host, rpc_api.ENGINE_TOPIC)
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/dist-packages/heat/engine/service.py", line 286, in __init__
2015-04-04 06:04:24.929 32732 TRACE heat     resources.initialise()
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/__init__.py", line 44, in initialise
2015-04-04 06:04:24.929 32732 TRACE heat     _load_global_environment(global_env)
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/__init__.py", line 49, in _load_global_environment
2015-04-04 06:04:24.929 32732 TRACE heat     _load_global_resources(env)
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/dist-packages/heat/engine/resources/__init__.py", line 54, in _load_global_resources
2015-04-04 06:04:24.929 32732 TRACE heat     manager = plugin_manager.PluginManager(__name__)
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/dist-packages/heat/engine/plugin_manager.py", line 58, in __init__
2015-04-04 06:04:24.929 32732 TRACE heat     self.modules = list(modules())
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/dist-packages/heat/common/plugin_loader.py", line 91, in load_modules
2015-04-04 06:04:24.929 32732 TRACE heat     module = _import_module(importer, module_name, package)
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/dist-packages/heat/common/plugin_loader.py", line 72, in _import_module
2015-04-04 06:04:24.929 32732 TRACE heat     module = loader.load_module(module_name)
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/python2.7/pkgutil.py", line 246, in load_module
2015-04-04 06:04:24.929 32732 TRACE heat     mod = imp.load_module(fullname, self.file, self.filename, self.etc)
2015-04-04 06:04:24.929 32732 TRACE heat   File "/usr/lib/heat/ibm-cloud-ext/resources/softlayer/softlayer_compute.py", line 43, in <module>
2015-04-04 06:04:24.929 32732 TRACE heat     from IPy import IP
2015-04-04 06:04:24.929 32732 TRACE heat ImportError: No module named IPy

So I guess we need to add that, not running the gradle installer that must be something that was installed, let’s see if there is anthying else we should know about. Just used a one liner to pull all the used modules in the templates.

First from the installer, looking for yum commands, I already have thei heat and keystone clients so this looks ok:

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ grep -R 'yum ' engine-install/
engine-install/buildSrc/src/main/groovy/tasks/InstallEngineTask.groovy:                         + "\n\tyum install -y gcc gcc-c++ kernel-devel mysql-devel python-devel"
engine-install/buildSrc/src/main/groovy/tasks/InstallEngineTask.groovy:                         + "\n\tyum remove -y akonadi"
Binary file engine-install/buildSrc/build/classes/main/InstallEngineTask.class matches
engine-install/media/engine/bin/configure-python-modules.sh:# pre-reqs to install MySQL-python: yum -y install gcc gcc-c++ kernel-devel python-devel mysql-devel
engine-install/media/build.gradle:// TODO: yum clean all
engine-install/media/build.gradle:    args '-c', "yum -y --nogpgcheck install ${INSTALL_PKG_DATABASE}"
engine-install/media/build.gradle:    args '-c', "yum -y --nogpgcheck install ${INSTALL_PKG_AMQP}"
engine-install/media/build.gradle:    args '-c', 'yum -y --nogpgcheck install keystonemiddleware openstack-heat python-heatclient'

Then the actual Python module setup the installer would have done, realizing that this IPy issue could be the first of many , I already installed the package easy enough with apt-get install python-ipy lets try and do it right the first time :)

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine$ less bin/configure-python-modules.sh
#!/bin/sh

WORKING_DIR="`pwd`/../python-modules"

echo "--------------------------------------------------------------------------------"
if command -v easy_install ; then
  echo "easy_install already installed."
else
  echo "Installing easy_install..."
  tar -xzf $WORKING_DIR/ez_setup-0.9.tar.gz
  python $WORKING_DIR/ez_setup-0.9/ez_setup.py
  echo "Done."
fi

if command -v pip ; then
  echo "pip already installed."
else
  echo "Installing pip..."
  python get-pip.py
  echo "Done."
fi


cd $WORKING_DIR

# pre-reqs to install MySQL-python: yum -y install gcc gcc-c++ kernel-devel python-devel mysql-devel
echo "
docopt-0.6.2.tar.gz
requests-2.6.0-py2.py3-none-any.whl
six-1.9.0-py2.py3-none-any.whl
prettytable-0.7.2.tar.gz
pyvmomi-5.5.0.tar.gz
poster-0.8.1.tar.gz
IPy-0.82a.tar.gz
boto-2.36.0.tar.gz
softlayer-python-3.2.0.tar.gz
" > pip-modules.txt

if command -v mysqld_safe ; then
 echo "MySQL-python-1.2.5.zip" >> pip-modules.txt
fi

OLD_MASK=$(umask)
umask 022
pip install --upgrade -r pip-modules.txt
umask $OLD_MASK

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine$ cd python-modules/
sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules$ ls -als
total 2856
   4 drwxr-xr-x 2 sgwilbur sgwilbur    4096 Apr  4 06:42 .
   4 drwxr-xr-x 8 sgwilbur sgwilbur    4096 Apr  3 05:43 ..
1380 -rw-r--r-- 1 sgwilbur sgwilbur 1412318 Mar 31 07:06 boto-2.36.0.tar.gz
 388 -rw-r--r-- 1 sgwilbur sgwilbur  396609 Mar 31 07:06 distribute-0.6.14.tar.gz
  28 -rw-r--r-- 1 sgwilbur sgwilbur   25901 Mar 31 07:06 docopt-0.6.2.tar.gz
   8 -rw-r--r-- 1 sgwilbur sgwilbur    6577 Mar 31 07:06 ez_setup-0.9.tar.gz
  68 -rw-r--r-- 1 sgwilbur sgwilbur   67852 Mar 31 07:06 ibm-ucd-patterns-heat-plugins.tgz
  32 -rw-r--r-- 1 sgwilbur sgwilbur   31588 Mar 31 07:06 IPy-0.82a.tar.gz
 108 -rw-r--r-- 1 sgwilbur sgwilbur  108935 Mar 31 07:06 MySQL-python-1.2.5.zip
  12 -rw-r--r-- 1 sgwilbur sgwilbur   12274 Mar 31 07:06 poster-0.8.1.tar.gz
  28 -rw-r--r-- 1 sgwilbur sgwilbur   24784 Mar 31 07:06 prettytable-0.7.2.tar.gz
 160 -rw-r--r-- 1 sgwilbur sgwilbur  161913 Mar 31 07:06 pyvmomi-5.5.0.tar.gz
 460 -rw-r--r-- 1 sgwilbur sgwilbur  469802 Mar 31 07:06 requests-2.6.0-py2.py3-none-any.whl
  12 -rw-r--r-- 1 sgwilbur sgwilbur   10222 Mar 31 07:06 six-1.9.0-py2.py3-none-any.whl
 160 -rw-r--r-- 1 sgwilbur sgwilbur  163469 Mar 31 07:06 softlayer-python-3.2.0.tar.gz

So I created a file with the adhoc requirements the script was going to create in engine-requirements.txt and placed in the python-modules directory since all the files are relatively pathed I’ll run the pip install from that directory:

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules$ cat engine-requirements.txt 
docopt-0.6.2.tar.gz
requests-2.6.0-py2.py3-none-any.whl
six-1.9.0-py2.py3-none-any.whl
prettytable-0.7.2.tar.gz
pyvmomi-5.5.0.tar.gz
poster-0.8.1.tar.gz
IPy-0.82a.tar.gz
boto-2.36.0.tar.gz
softlayer-python-3.2.0.tar.gz

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules$ sudo pip install --upgrade -r engine-requirements.txt
Unpacking ./docopt-0.6.2.tar.gz
  Running setup.py (path:/tmp/pip-P7Ld8X-build/setup.py) egg_info for package from file:///home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules/docopt-0.6.2.tar
.gz

Unpacking ./requests-2.6.0-py2.py3-none-any.whl
Unpacking ./six-1.9.0-py2.py3-none-any.whl
Unpacking ./prettytable-0.7.2.tar.gz
  Running setup.py (path:/tmp/pip-tpsoPR-build/setup.py) egg_info for package from file:///home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules/prettytable-0.7.
2.tar.gz

Unpacking ./pyvmomi-5.5.0.tar.gz
  Running setup.py (path:/tmp/pip-C1c9Ue-build/setup.py) egg_info for package from file:///home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules/pyvmomi-5.5.0.ta
r.gz

Unpacking ./poster-0.8.1.tar.gz
  Running setup.py (path:/tmp/pip-_d3TnA-build/setup.py) egg_info for package from file:///home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules/poster-0.8.1.tar
.gz

Unpacking ./IPy-0.82a.tar.gz
  Running setup.py (path:/tmp/pip-VbpmZB-build/setup.py) egg_info for package from file:///home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules/IPy-0.82a.tar.gz

Unpacking ./boto-2.36.0.tar.gz
  Running setup.py (path:/tmp/pip-ZAewpz-build/setup.py) egg_info for package from file:///home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules/boto-2.36.0.tar.
gz

    warning: no files found matching 'boto/mturk/test/*.doctest'
    warning: no files found matching 'boto/mturk/test/.gitignore'
Unpacking ./softlayer-python-3.2.0.tar.gz
  Running setup.py (path:/tmp/pip-w04BCx-build/setup.py) egg_info for package from file:///home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/engine-install/media/engine/python-modules/softlayer-python
-3.2.0.tar.gz

Installing collected packages: docopt, requests, six, prettytable, pyvmomi, poster, IPy, boto, SoftLayer
  Running setup.py install for docopt

  Found existing installation: prettytable 0.7.2
    Uninstalling prettytable:
      Successfully uninstalled prettytable
  Running setup.py install for prettytable

  Running setup.py install for pyvmomi

  Running setup.py install for poster

  Found existing installation: IPy 0.81
    Uninstalling IPy:
      Successfully uninstalled IPy
  Running setup.py install for IPy

  Found existing installation: boto 2.20.1
    Uninstalling boto:
      Successfully uninstalled boto
  Running setup.py install for boto

    warning: no files found matching 'boto/mturk/test/*.doctest'
    warning: no files found matching 'boto/mturk/test/.gitignore'
    changing mode of build/scripts-2.7/sdbadmin from 644 to 755
    changing mode of build/scripts-2.7/elbadmin from 644 to 755
    changing mode of build/scripts-2.7/cfadmin from 644 to 755
    changing mode of build/scripts-2.7/s3put from 644 to 755
    changing mode of build/scripts-2.7/fetch_file from 644 to 755
    changing mode of build/scripts-2.7/launch_instance from 644 to 755
    changing mode of build/scripts-2.7/list_instances from 644 to 755
    changing mode of build/scripts-2.7/taskadmin from 644 to 755
    changing mode of build/scripts-2.7/kill_instance from 644 to 755
    changing mode of build/scripts-2.7/bundle_image from 644 to 755
    changing mode of build/scripts-2.7/pyami_sendmail from 644 to 755
    changing mode of build/scripts-2.7/lss3 from 644 to 755
    changing mode of build/scripts-2.7/cq from 644 to 755
    changing mode of build/scripts-2.7/route53 from 644 to 755
    changing mode of build/scripts-2.7/cwutil from 644 to 755
    changing mode of build/scripts-2.7/instance_events from 644 to 755
    changing mode of build/scripts-2.7/asadmin from 644 to 755
    changing mode of build/scripts-2.7/glacier from 644 to 755
    changing mode of build/scripts-2.7/mturk from 644 to 755
    changing mode of build/scripts-2.7/dynamodb_dump from 644 to 755
    changing mode of build/scripts-2.7/dynamodb_load from 644 to 755
    changing mode of /usr/local/bin/dynamodb_dump to 755
    changing mode of /usr/local/bin/lss3 to 755
    changing mode of /usr/local/bin/taskadmin to 755
    changing mode of /usr/local/bin/mturk to 755
    changing mode of /usr/local/bin/launch_instance to 755
    changing mode of /usr/local/bin/route53 to 755
    changing mode of /usr/local/bin/instance_events to 755
    changing mode of /usr/local/bin/elbadmin to 755
    changing mode of /usr/local/bin/s3put to 755
    changing mode of /usr/local/bin/list_instances to 755
    changing mode of /usr/local/bin/dynamodb_load to 755
    changing mode of /usr/local/bin/cq to 755
    changing mode of /usr/local/bin/pyami_sendmail to 755
    changing mode of /usr/local/bin/bundle_image to 755
    changing mode of /usr/local/bin/cwutil to 755
    changing mode of /usr/local/bin/fetch_file to 755
    changing mode of /usr/local/bin/asadmin to 755
    changing mode of /usr/local/bin/glacier to 755
    changing mode of /usr/local/bin/kill_instance to 755
    changing mode of /usr/local/bin/cfadmin to 755
    changing mode of /usr/local/bin/sdbadmin to 755
  Running setup.py install for SoftLayer

    Installing sl script to /usr/local/bin
Successfully installed docopt requests six prettytable pyvmomi poster IPy boto SoftLayer
Cleaning up...

Stop and start our servers and see how we did.

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install$ heat resource-type-list|grep IBM
| IBM::EC2::Alarm                          |
| IBM::EC2::AutoScalingGroup               |
| IBM::EC2::AutoScalingPolicy              |
| IBM::EC2::EIP                            |
| IBM::EC2::EIPAssociation                 |
| IBM::EC2::InternetGateway                |
| IBM::EC2::LaunchConfiguration            |
| IBM::EC2::Port                           |
| IBM::EC2::RouteTable                     |
| IBM::EC2::RouteTableAssociation          |
| IBM::EC2::SecurityGroup                  |
| IBM::EC2::Server                         |
| IBM::EC2::Subnet                         |
| IBM::EC2::VPC                            |
| IBM::EC2::Volume                         |
| IBM::EC2::VolumeAttachment               |
| IBM::ElastiCache::Cluster                |
| IBM::ElastiCache::ParameterGroup         |
| IBM::ElastiCache::ReplicationGroup       |
| IBM::ElastiCache::SubnetGroup            |
| IBM::RDS::Instance                       |
| IBM::RDS::ParameterGroup                 |
| IBM::RDS::SubnetGroup                    |
| IBM::Route53::HostedZone                 |
| IBM::Route53::RecordSet                  |
| IBM::S3::Bucket                          |
| IBM::SoftLayer::Port                     |
| IBM::SoftLayer::Server                   |
| IBM::UrbanCode::ResourceTree             |
| IBM::UrbanCode::SoftwareConfig::UCD      |
| IBM::UrbanCode::SoftwareDeploy::UCD      |
| IBM::VCenter::Port                       |

Cool, first step done, giving ourselves a pat on the back we can now continue.

PASTING

sgwilbur@lappy:~/workspace/ucdp-installation/ibm-ucd-patterns-install/web-install$ sudo ./install.sh
[sudo] password for sgwilbur:
Downloading file:/home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/web-install/gradle/wrapper/gradle-1.12-bin.zip
........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
Unzipping /root/.gradle/wrapper/dists/gradle-1.12-bin/5do04mrfk2g077n2l4dnkv26b5/gradle-1.12-bin.zip to /root/.gradle/wrapper/dists/gradle-1.12-bin/5do04mrfk2g077n2l4dnkv26b5
Set executable permissions for: /root/.gradle/wrapper/dists/gradle-1.12-bin/5do04mrfk2g077n2l4dnkv26b5/gradle-1.12/bin/gradle
International Program License Agreement
Part 1 - General Terms

REMOVED FULL LICENSE TEXT

 Program's Server component is installed as well as sufficient Managed Virtual Server entitlements to cover the relevant servers where the Program's Agent component is installed.
L/N:  L-PASH-9G4JCM
D/N:  L-PASH-9G4JCM
P/N:  L-PASH-9G4JCM


--------------------------------------------------------------------------------
Do you accept the license? [y, n]
y

Response: y


Installing UrbanCode Deploy with Patterns version 6.1.1.0

--------------------------------------------------------------------------------
Specify the directory where UrbanCode Deploy with Patterns should be installed.

[Default /opt/ibm-ucd-patterns]: /opt/ibm-ucd/patterns

Response: /opt/ibm-ucd/patterns

--------------------------------------------------------------------------------
The specified directory does not exist. Do you want to create it? [y, n]

[Default Y]:

Response: Y

--------------------------------------------------------------------------------
Enter the public IP address or host name of this system.

[Default lappy]: 192.168.1.139

Response: 192.168.1.139

--------------------------------------------------------------------------------
Do you want the design server to always use secure connections using SSL? [y, n]

[Default Y]:

Response: Y

--------------------------------------------------------------------------------
Enter the port on which the design server should listen for secure HTTPS requests.

[Default 8443]:

Response: 8443

--------------------------------------------------------------------------------
Enter the port to which the design server should redirect unsecured HTTP requests.

[Default 8080]:

Response: 8080

--------------------------------------------------------------------------------

        If the path is a directory, all license files in the directory will be used.
        A list of paths must be separated by colons on *nix systems and semicolons on
        Windows systems.

Enter the RCL server path(s). (e.g. 27000@example.com)


[Default 27000@localhost]: 27000@192.168.1.84

Response: 27000@192.168.1.84


--------------------------------------------------------------------------------

        When installing a server as a part of an existing cluster or when
        using a pre-populated database, it is not necessary to create the database
        schema. This step must be performed when installing a standalone server to a
        fresh database or when installing the first server in a cluster.

Create database schema? [y, n]

[Default Y]:

Response: Y

--------------------------------------------------------------------------------

Enter the database type to use.

[Default derby]: mysql

Response: mysql

--------------------------------------------------------------------------------
Enter the database driver.

[Default com.mysql.jdbc.Driver]:

Response: com.mysql.jdbc.Driver

--------------------------------------------------------------------------------
Enter the database connection string. Eg. jdbc:mysql://localhost:3306/ibm_ucdp

[Default jdbc:mysql://localhost:3306/ibm_ucdp]: jdbc:mysql://localhost:3306/ibm_ucp

Response: jdbc:mysql://localhost:3306/ibm_ucp

--------------------------------------------------------------------------------
Enter the database username.

[Default ibm_ucdp]: ibm_uc

Response: ibm_uc

--------------------------------------------------------------------------------
Enter the database password.

[Default password]: Rat1onal

Response: Rat1onal

        You have specified a database driver which is not in your classpath ("com.mysql.jdbc.Driver").
        Without a valid, classpath-resolveable database driver, your design server will not be
        able to access the database.

        Note: The MySQL Connector for Java is available at http://dev.mysql.com/downloads/connector/j/


--------------------------------------------------------------------------------
Enter the the path to the database library jar file to copy into your installation.

[Default /tmp/mysql-jdbc.jar]: /usr/share/java/mysql-connector-java-5.1.28.jar

Response: /usr/share/java/mysql-connector-java-5.1.28.jar

        The library jar was successfully copied into /home/sgwilbur/workspace/ucdp-installation/ibm-ucd-patterns-install/web-install/media/server/opt/tomcat/lib/mysql-connector-java-5.1.28.jar.


--------------------------------------------------------------------------------
Enter the URI for the IBM UrbanCode Deploy server.

[Default https://localhost:8443]: https://192.168.1.139:8443/

Response: https://192.168.1.139:8443/

--------------------------------------------------------------------------------

        In order for the design server to access your application components, 
        a connection must be established to query the REST API from the deploy 
        server. A valid authentication token is required to connect from the 
        design server to the deploy server.

        The token can be generated at: https://192.168.1.139:8443/#security/tokens 

        You may choose to skip this step and enter "None". You may update 
        this value in the System Settings page of the design server. 

Enter the authentication token to access the deploy server. 

[Default None]: 

Response: None
Installing server files into /opt/ibm-ucd/patterns...Creating new encryption key.
Creating Security Database Schema ...
Creating Security Database Foreign Keys ...
Creating Security Database Indexes ...
Creating IBM UrbanCode Deploy with Patterns Database Schema ...
Seeding IBM UrbanCode Deploy with Patterns Database ...

--------------------------------------------------------------------------------


You may now start the design server from your console using the following command.

        /opt/ibm-ucd/patterns/bin/server start

After starting the server, you may access the design server by pointing your web-browser at

        https://192.168.1.139:8443/landscaper 


--------------------------------------------------------------------------------


You may now start the cloud discovery service from your console using the following command.

        /opt/ibm-ucd/patterns/opt/udeploy-patterns-cloud-discovery-service/runserver

After starting the server, you may test to make sure the service is available by pointing 
 your web browser at

        http://192.168.1.139:7575/ 


--------------------------------------------------------------------------------
Product Installation Complete. (press return to exit installer)

Ok, start it up the servers and check that we are in fact up and running, note that Tomcat will start and daemonize, but we will need to start up the discovery services a different way to push it to the background and capture the logged output.

sgwilbur@lappy:/opt/ibm-ucd/patterns$ sudo ./bin/server start
Starting IBM UrbanCode Deploy with Patterns: Tomcat started.
sgwilbur@lappy:~$ sudo /opt/ibm-ucd/patterns/opt/udeploy-patterns-cloud-discovery-service/runserver start
[sudo] password for sgwilbur:
Running app with gevent
192.168.1.139 - - [2015-04-04 07:15:36] "GET / HTTP/1.1" 200 157 0.001087
192.168.1.139 - - [2015-04-04 07:15:36] "GET / HTTP/1.1" 200 157 0.000820
192.168.1.139 - - [2015-04-04 07:15:37] "GET / HTTP/1.1" 200 157 0.000716
192.168.1.139 - - [2015-04-04 07:17:37] "GET / HTTP/1.1" 200 157 0.000631
192.168.1.139 - - [2015-04-04 07:17:37] "GET / HTTP/1.1" 200 157 0.001134

All looks good, we can browse http and https for landscaper http://192.168.1.139:8080/landscaper/login, https://192.168.1.139:8443/landscaper/login and get the default ‘IBM Cloud Discovery Service is Running…’ at http://192.168.1.139:7575/

UCDP Login

IBM Cloud Delivery Service

So go ahead and login to UCDP, and we will connect to Openstack

UCDP Dashboard - Users Menu

Fill in your OpenStack Keyston connection, an admin user, and do not uncheck the option to also add as default orchestration engine:

UCDP Users - Add User Realm

This should import the users in OpenStack into UCDP, I ran into a snag as I deleted the users and tried to re-import them…. I had to manually update the SQL in sec_user to re-enable those userids, but I am not sure what happens when two users with the same name exist will have to look that one up later.

UCDP Users- Request Import

UCDP Users - Imported Users

Realized the ids all share and email, so let’s update them in Keystone just to show the update process.

OpenStack - Keyston Users

And trigger the User Update:

UCDP Users - Trigger Update

And voila!

UCDP Users - Trigger Update

Logout and confirm you can login with one of these users, then logoff, and log back in as ucdpadmin.

Next Steps:

  • Create an OpenStack Project

  • UCDP: Settings > Clouds - Configure the Authorization for your project with a functional id associating it with a project in OpenStack.

  • UCDP: Create a Team in UCDP, add members, and associate the Cloud Authorization with your project.

  • Modeling environments for OpenStack and OpenStack-based clouds

Good luck!

Reference:
FootNotes