Getting Start with UrbanCode Deploy with Patterns: A full-stack Application designer for Clouds

This blog post is about IBM UrbanCode Deployment with Patterns Blueprint and a how to guide, to help you manage the deployment solutions in the cloud that enables you to design, deploy and update full-stack environment for multiple clouds.

In the traditional on-premise IT teams are struggling to keep up with having to manage and deploy applications into various environments. It is a lot of work trying to get an environment and resources setup before we can start to deploy our application. Each deployment requires a lot of manual processes, where lots of people in the delivery pipelines such as the IT Architects and specialists need to be involved in setting up the virtual machine, network, storage, settings the different configurations in order to get the environment ready for applications deployment. This whole process of finding, setting up an environment and deploying the application can take an average of 3-4 weeks to complete. This is a lot of time wasted. As times goes by, the new business requirements came in and what if we need to make changes in the multiple environments in DEV, TEST and PROD, there are a bunch of manual processes and re configurations that needs to take place yet again.

Therefore in the new cloud business model such as pay-as-you-go in the Infrastructure as a Service (IaaS), it’s definitely a lot easier if we can get these resources from the Cloud. Cloud can help us speed up our development and deployment for new applications in the heterogeneous datacenter (private, public and on premise) as business operational demands increase or decrease very quickly.

This way, when we need an environment to deploy our application we can just simply request one through a cloud environment and have the environment provisioning for us without having to wait 3 to 4 weeks. When your business operational demands increase or decrease, you have an ability to add needed business application quickly and integrate and scale them up or down as needed.

In this series of blog post, I will be demonstrating to you how easy it is to use IBM Blueprint designer to:

Part 1: Create a Provision environment on clouds through OpenStack

Part 2: Deploy my application to the OpenStack environment on the cloud

The value BluePrint Designer brings

  • Design complete environment patterns that include your applications along with the infrastructure and middleware that supports them.
  • Enables specialist and integrators to capture their knowledge and expertise in the artifacts in the Heat Orchestrator Template (HOT) to represent the information about images, networks, storages and applications.
  • Capture the heterogeneous expertise of the total DevOps team, from solution architects to cloud infrastructure specialists
  • Manage and version those full-stack environment patterns through Git repositories, like you do with application source codes
  • Provision a full-stack environment in minutes from a pattern document that includes your application, the infrastructures, middleware and everything needed to run it
  • Cloud-portability through a blueprint. Ability to port your blueprint design in the different virtualized infrastructure to any cloud provider (OpenStack, Amazon EC2, SoftLayer and Vware Vcenter ) across the heterogeneous datacenter (private, public and on premise) as you wish.

Prerequisites

  1. Blueprints Design Server is connected to OpenStack and also has already been configured to connect to UrbanCode Deploy (UCD) server
  2. IBM UrbanCode Deploy with Patterns is fully configured and it’s up and running
  3. OpenStack Cloud environment has already been configured
  4. User responsible for creating blueprint has been granted access to an OpenStack Cloud.

Note: Without the access to the OpenStack cloud environment, we will not be able to use UrbanCode Deploy with Patterns (UCDP) to access resources such as compute images, networks, storage and security from OpenStack to build our blueprints

Part 1: Creating and provisioning new Environment to OpenStack Cloud

In this section, we will be using a Blueprint designer to model, provision and update cloud environments from OpenStack Heat-based blueprints. We will be using a blueprint designer to describe topology of machine, network, storage resources and use the blueprint to provision and install UrbanCode Deploy agents, which references UCD components and processes.

Objectives

How to use IBM Blueprint Designer to create a blueprint that we can then later on use to help us setting up a new environment to deploy the application on.

In this section you will:

  • Learn how to work with the blueprint designer
  • Lean to design a complete environment pattern that includes your application along with its infrastructure and middleware that supports them
  • Experience how a blueprint design can be designed by technical experts once and then be deployed and reused across the enterprise

At the end of this section you will have a blueprint that describe an environment. You defined Linux nodes, the networks that would be attached to the two Linux nodes and the software components to be deployed on each Linux system.

1.1 Create a Blueprint in Blueprint Designer

  1. Log on to your landscaper
  2. lanscaper.png

  3. Select Blueprint
  4. new_blueprint.png

  5. Click “New” in the add a new file dialog and type in your blueprint name and “Save” it. You will then be landed in the graphical editor mode of the blueprint.
  6. add_new_file

    Notice that on the right hand side of the blueprint designer, there is the palette for different resources for you to select from such as compute images, networks, storage and security from the OpenStack Cloud that you can just simply drag and drop into our design diagram canvas in the middle and start to build a blueprint for your environment.

    There are two methods that you can use for building your blueprint. The first method is via the graphical editor and the second method is via the Heat Orchestration Template (HOT) source code of blueprint directly.
    Caution: If you don’t see any of these resources that means, your UrbanCode Deployment with Patterns is not properly configure to access the OpenStack Cloud. You have to make sure that user who is responsible for creating the blueprint design has been authenticated to your OpenStack Cloud

  7. From the Network drawer of the palette on the right select and drag the networks that you would like to use with this blueprint. In my case, I am going to drag the private network to the designer’s diagram canvas.
  8. drawer.png

  9. Select the virtual image. Expand the images drawer; these are the images that are available on my OpenStack Cloud system. Drag and drop the Ubuntu1404x64 image onto the private network resource on the diagram canvas. This will automatically draw a connecter from the Private network to the image
  10. ubuntu.png

  11. Select the image to edit its properties and then select the pencil icon. Set Key and Name field to “dbserver
  12. db_server.png

  13. Repeat the previous step, drag and drop the Ubuntu1404x64 image onto the private network resource on the diagram canvas. This compute resource will be the host for my Websphere Liberty node
  14. Set Key and Name filed “appserver
  15. App_server.png

  16. Expand the Components drawer. Notice the components and tags that are available from UrbanCode Deploy server. When we drag and drop these components tag to our virtual machine, all of those components on the UCD server for those tags will be added to our blueprint.
  17. From the Component Drawer, I then select the MySQL Server component and drop it onto the first compute resource, which is “dbserver“. Basically, what I am doing at this step is that I’m adding the components from my UrbanCode Deploy server to the image.
  18. Choose UCD Application, “JKE” application is selected in the dialog that pops up, and click OK
  19. JKE_component.png

    Note: I need to choose the UCD application, so, that the UrbanCode Deploy server knows which application I am modeling. This will come in handy once we switch from a blueprint designer to UrbanCode Deploy Server later on.

  20. Repeat the previous step to add db component to the same compute resource.
  21. Next I will add the WebSphere Liberty Profile and war component to the second compute resource.

    jke_db.png

  22. Set the component process for MySQL server, select “CHOOSE” and select “deploy“. If I want, I could also give it a version name as well.
  23. deploy.png

    Note: that both of the compute resources for dbserver and appserver are connected to the private network. It means that they can access each other but will not be accessible outside the OpenStack server. To have the web application that is deployed on the Liberty node accessible from externally, I need to have OpenStack assign a floating IP address to the WebSphere Liberty node on the “public” network.

  24. In the Blueprint Designer, click the circle icon in the bottom-right hand corner of the Liberty Compute resource on the blueprint designer canvas to assign a floating IP to the node in OpenStack. Once it’s done notice that it will turn blue.
  25. static_ip.png

  26. Save the blueprint
  27. Next we will mark the deployment sequence order in which the UrbanCode deploy components should be deployed. Select the Policies drawer, and drag-and-drop Deployment Sequence onto each of the components in the sequence of

    1. MySQL Server
    2. db
    3. Websphere Liberty Profile
    4. war

    sequence.1.png

    Note: If you set the deployment sequence incorrectly, you can click the Deployment Sequence icon of any of the component and change the sequence in the dialog.

    sequence.png

  28. Save your current progress

1.2 Exploring the UrbanCode Deploy

  1. Open a new tab in your browser and type in the address of your UrbanCode Deploy Server.
  2. 1.1

  3. Login to UrbanCode Deploy with your credential
  4. IE: username: admin
    password: your_password

  5. In UrbanCode Deploy, select Components, and select the war component.
  6. Screenshot_3_2_16__12_28_PM.png

  7. Select the Configuration tab on the second row, and open the Environment Property Definition tab
  8. configuration_tab.png

  9. Notice that the default value for “db.host “is set to “localhost”.
  10. I will have to direct OpenStack to extract the value of the IP address allocated to the MySQL compute resource on the private network and set this as the value of the jke.db.host property.
  11. Switch back to the Blueprint Designer tab
  12. Select “war” component, set “jke.db.host” to “Private IP address for dbserver
  13. Private_iP.png

  14. Save the Blueprint. Now I have my blueprint to deploy my application and what cloud I want to deploy it on.

 

Advertisements

IBM UrbanCode Deploy with Patterns and OpenStack: Part I: Installing DevStack Kilo on Ubuntu 14.04.03 LTS for newbies

Even though, there are ton of instructions on the Internet on how to configure DevStack environment on Ubuntu 14.0.4 LTS. I have to admit that it took me several tries and many hours to troubleshoot my environment until I finally got the hang of it and to truly understand what was going on.

Since this is simply a test environment and I want to keep things very simple, I am going to use the same interface for all of OpenStack networks. In this blog post you will find a straightforward instruction on how to quickly setup OpenStack environment via DevStack. Once the OpenStack environment is up and running, I will show you how to quickly install and integrate with the IBM UrbanCode Deploy with Patterns so, that you can quickly:

  • “Stand up continuous delivery pipeline environment for new applications
  • Provision short-lived testing sandboxes for specific infrastructure
  • Promote infrastructure changes through environments just like application changes
  • Collaborate by composing the work of infrastructure specialists into full environment
  • Drive environment consistency from testing to production even when you are using multiple clouds” –developer.ibm.com

See the demo of UrbanCode Deploy with Patterns

Prerequisite

  1. Download Ubuntu 14.04.03 LTS
  2. Install OS / Configure Ubuntu environment to have it ready for the OpenStack Environment
  3. Configure Ubuntu and Prepare for devstack configuration
    • $ sudo apt-get update
    • $ sudo apt-get upgrade
    • $ sudo apt-get dist-upgrade

DevStack Preparation

  1. Install Git
 $ sudo apt-get install git
  1. Clone devstack.git from branch stable/kilo
 $ git clone https://github.com/openstack-dev/devstack.git -b stable/kilo
  1. Create a localrc file in the root DevStack directory
DEST=/opt/stack

# Logging
LOGFILE=$DEST/logs/stack.sh.log
VERBOSE=True
LOG_COLOR=False
SCREEN_LOGDIR=$DEST/logs/screen

# Credentials
ADMIN_PASSWORD=MyStack
MYSQL_PASSWORD=MyStack
RABBIT_PASSWORD=MyStack
SERVICE_PASSWORD=MyStack
SERVICE_TOKEN=MyToken

# HOST IP
HOST_IP=173.17.137.142

# Network ranges
FIXED_RANGE=10.254.1.0/24
NETWORK_GATEWAY=10.254.1.1

# Heat
ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng

# Neutron Load Balancing
ENABLED_SERVICES+=,q-lbaas

# Neutron Networking Service
disable_service n-net
enable_service n-cauth
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service horizon
disable_service tempest

Install DevStack

  1. cd into devstack directory
  2. Install Devstack
     $ ./stack.sh
  1. If DevStack was successfully installed, you should see the following screen
            This is your host ip:173.17.137.142
            Horizon is now available at http://173.17.137.142/
            Keystone is serving at http://173.17.137.142:5000/
            The default users are: admin and demo
            The password: MyStack
  1. Confirming heat is responding (Note: We need an the heat orchestration engine to connect to each cloud system. If you don’t want configure Heat here you also have an option to have UrbanCode Deployment with Patterns to install the heat orchestration engine for you later on as well.)
     $ source admin-openrc

Note: admin-openrc file can be download by log in to the OpenStack Dashboard > Choose Your project> Compute > Access & Security > Open API Tab then click Download OpenStack RC File

  1. Make sure that heat service is registered with Keystone
 $ keystone service-list | grep heat
| 6326cd8a316743a79d77d5b9a0ee7bf9 | heat     | orchestration | Heat Orchestration Service   |
| 89333a24d6214a7283e08cbf86d56196 | heat-cfn | cloudformation | Heat CloudFormation Service |
  1. Confirm that Heat is running and responding
 $ heat stack-list
+----+------------+--------------+---------------+
| id | stack_name | stack_status | creation_time |
+----+------------+--------------+---------------+
+----+------------+--------------+---------------+
  1. Generate a keypair for Heat to launch instances
 $ nova keypair-add heat_key > heat_key.priv
 $ chmod 600 heat_key.priv
  1. Launch a stack by running a heat stack-create

Note: Your image_id might be different than the example that I am showing here

 $ heat stack-create teststack -u http://git.openstack.org/cgit/openstack/heat-templates/plain/hot/F20/WordPress_Native.yaml -P key_name=heat_key -P image_id=cirros-0.3.4-x86_64-uec
+--------------------------------------+------------+--------------------+------------------| id             | stack_name | stack_status       | creation_time       |
+--------------------------------------+------------+--------------------+------------------
| ffa93ba6-f1fa-43b0-8f8b-5e87c9b700f0 | teststack | CREATE_IN_PROGRESS | 2015-12-23T17:24:17Z |
+--------------------------------------+------------+--------------------+------------------
  1. List Stacks
 $ heat event-list teststack
  1. List stack events

Referencing: https://wiki.openstack.org/wiki/Heat/GettingStartedUsingDevstack

Extending the existing Heat Engines

We need to extend the existing Heat Engines that we have configured with DevStack so, that we can use it with the UrbanCode Deploy with Patterns

  1. Assuming that you already download the UrbanCode Deploy with Patterns
  2. Edit the configuration file for the engine at /etc/heat/heat.conf and uncomment /usr/lib64/heat and /usr/lib/heat (Note: In my heat.conf configuration file. I didn’t have any plugin-dirs parameters setup, therefore, I need to first create a plugin-dirs directory at /usr/lib/heat and update my plugin_dirs parameters in the /etc/heat/heat.conf to point to it.)
  1. From the UrbanCode Deploy with Patterns with Engine extract ibm-ucd-patterns-install/engine-install/chef.tar.gz file and then
     $ tar -xvf chef.tar.gz

    and then find the cookbooks/ibm-orchestration-ext/files/default/plugins.tar.gz file to the plugs-in folder

 $ tar -xvf plugins.tar.gz -C /usr/lib/heat/
  1. Restart the Heat Engine
 $ service openstack-heat-engine restart
  1. Restart the Heat API
 $ service openstack-heat-api restart
  1. Verify the new resource types from UrbanCode Deployments with Patterns are exist
 $ heat resource-type-list