Tracking the status of your builds and getting their results

Our team was asked recently how we go about getting access to the CLM software downloads.  The primary answers were of course, the jazz.net product downloads while others use a ready made virtual or cloud image.  I tend to support customers who are at different version levels.  I also tend to want to run different versions of CLM natively.  Some times I get the download from jazz.net, which is really the official channel (not to mention IBM Passport Advantage site).  However, the typical download will involve use of IBM Installation Manager, which works great, but when you want to install multiple versions, you end up with multiple similarly named program groups.  I choose a method which allows me to install from a zip archive.

The point of this blog entry is not to advocate my method of getting the download.  In fact I wouldn’t recommend it for any production deployment.  What I want to do is use my method to illustrate some cool features of RTC that you can apply in your environment, not for getting our downloads but to allow others in your organization access to those you produce and to monitor progress along the way.

As I said, I like to use the zip archive installs.  This works for me since I don’t need an enterprise install and can typically get away with Derby and Tomcat.  The jazz.net downloads have zip installs for the RTC client, Jazz Build Engine and a combined JTS/CCM but not a full CLM install (JTS, CCM, RQM, RDNG).  If I want the zip install of CLM, I have to go elsewhere.

So let’s say I want the GA version of CLM 5.0 along with the RTC Eclipse client and Jazz Build Engine.  I will end up using combinations of the following capabilities:

• Build Queue
• Build Result
• Track Build Item

Note once again, this scenario is for illustrative purposes only to highlight those capabilities and not promoting it as how you should go about getting your Jazz downloads.

Firstly, I know that the Jazz CLM development team has a Jazz build definition to produce one of its releases.  Using the Team Concert Web UI, I can browse the Jazz Collaborative ALM project area to see the list of build definitions for the project and in particular, the Jazz Collaborative ALM Development team area.

I select the build definition for the build producing the bits for the release I am interested, in this case, calm.30-50(aka calm.jcb) for the version 5.0 release I want.

Note the list of most recent build results for the calm.30-50(aka calm.jcb) build.  See the Tags column.  RTC allows build results to be tagged.  The development team uses  tags for different purposes (e.g. passing a certain test stage) but one is to denote which build result is declared final for a particular milestone or release.  Here I can see that build result CALM-I20140509-1731 completed successfully, is green, is tagged 5.0-ga and thus produced the final bits for the 5.0 generally available (GA) release of CLM. 

I select the result link to navigate to the CALM-I20140509-1731 build result.  Of interest to me here are the Downloads and External Links tabs. 

The items on the Downloads tab represent artifacts produced by the build.  These can be published to the build result using the artifactLinkPublisher Ant task, available via the Jazz Build Toolkit.  In this scenario, links to the various platform CLM Installation Manager installers and full CLM zip archive install are published with the build result.  I want a 64-bit Linux install so I will select the Workbench-for-CLM-Linux64-Redhat_5.0.0.0-CALM-I20140509-1731.zip link.

Once downloaded, I can expand the archive, navigate to the server folder and start the Jazz Team Server and go through the setup and import the appropriate licenses.

This only gets me the CLM installation.  I also want to get the corresponding RTC Eclipse Client and Jazz Build Engine.  Since this is a GA release, I can get these from the typical jazz.net downloads page, but to further my illustration, I will use the External Links tab.

On this tab, I see the list of contributing builds from other projects.  I am interested in the contributing build from the Team Concert project area, that is, RTC – rtc.jcb RTC-I20140509-1417.  Note these links can be produced using the linkPublisher Ant task.

Navigating to the result of the contributing RTC build, I see the build is tagged for 5.0 and has it’s own Downloads and External Links tabs. 

For my purposes, I am interested only in the Downloads tab so I can find the download link for the zip install version of the Build System Toolkit and Eclipse Client.

Once downloaded, I can expand and run. 

One last feature, which is very useful if I am trying to get access to the milestone or release bits early or just to be aware of how we are progressing at declaring its final build, is the Track Build Item work item type. 

The development team uses these work items to collaborate across the teams and disciplines on the builds being produced, their contributors, schedule, test status, blockers found and finally, which will be designated as final/green for the milestone/release.  For example, the Track Build Item 324131 was used to track the CLM 5.0.1 RC1 milestone release. 

The description section tells me a lot of useful information about the release and its primary build and contributors. 

Looking through the discussion comments, I see a great deal of collaboration leading to green declaration of the final build.

A build is ready for test:

A problem occurred with the build

A build may be ready to be declared final but the team needs to coordinate on some approvals.

The team is declaring the latest build as green and ready to move on to the next release.

With all this useful information available, should I want to stay current on the release and pick and choose which builds to get early access to, I would subscribe to the work item and monitor the comments.

In conclusion then, by browsing the build queues to find a build result that has been tagged then using the result’s Downloads and External Links, coupled with the Track Build Item work item, development teams have a powerful way to monitor and collaborate on builds throughout the release cycle and obtain their results.

Automating changes to a build workspace configuration

Here’s the scenario.  A build engineer gets a request to establish a build workspace and corresponding build workspace.  He creates the build workspace, gets the desired components added to it, sets the flow targets then changes ownership of the workspace to the build user’s functional ID.  After that, he creates the build definition and associates it with the newly created workspace.  All is good and developers can begin to use the new build definition.  After a time, a request comes in to change the configuration of the build workspace, specifically the components to use.  Since the build workspace is owned by the build user, any changes to it must be made by that user.  Many organizations eschew the use of functional IDs or at least minimize who knows their credentials and are concerned about extra maintenance brought on by password expiration rules.  What to do?

This specific scenario came up recently with a customer of mine.  In particular, their build workspaces have three flow targets.  The current/default is the stream with the application source and it is scoped to include a subset of the components in the stream.  The other two streams are build script and configuration related.  At times, the release team needs to change the components included in the scope of the application source stream.  They do so currently by logging in as the build user, something they detest doing.

What they much prefer is support for team ownership of repository workspaces but that isn’t currently possible (though requested via 271760).  We instead proposed a solution that put the workspace configuration change in the build script which is already performed as the build user and owner of the workspace.

As of 4.0.1, the SCM CLI includes the capability to add/change workspace flow targets.  It was later refactored in 4.0.6 to the current verb-oriented form.

${scmExe} –non-interactive set flowtarget -r ${repositoryAddress} -u ${userId} -P ${password} ${team.scm.workspaceUUID} ${targetStreamUUID} –flow-components ${componentsToLoad}

Where:

• scmExe – path to scm CLI executable
• repositoryAddress – URL of CCM server
• userId – build user ID
• password – password of build user
• team.scm.workspaceUUID – build repository workspace UUID
• targetStreamUUID – stream flow target to set component scope
• componentsToLoad – space delimited list of components

Unfortunately the SCM CLI does not understand the password file format used by the Ant tasks. You either need to give it as plain text using –P shown above or login outside the build system with the option to remain logged in (on each build machine).

A simple way to get the UUID of a workspace or stream is to open it up in the Eclipse editor and select Copy URL workspace editor menu in the view header or browse to it in the Web UI to get the URL.  The end portion of the URL is the UUID.  For example, the UUID for the repository workspace URL shown below is _GVqXYLRpEeOdavKqgVc36Q

https://clm.jkebanking.net:9443/ccm/resource/itemOid/com.ibm.team.scm.Workspace/_GVqXYLRpEeOdavKqgVc36Q

For components with spaces in their name, care must be taken to offset them by appropriate quotes.  For my tests, single quotes worked on Linux and double on Windows.

The command will set the scope of the specified flow target (e.g. application development stream) for the specified workspace (e.g. build workspace) to include only those components in the specified list.  The current configuration of the component(s) in the target stream is used.

Note if multiple flow targets exist for the workspace and a component listed is included in multiple flow targets, then scoped flows need to be used to avoid conflicts.  See How should my source code be loaded from Jazz SCM?

Let’s take a look at an example.  Below, the build.brm.continuous repository workspace, owned by the build user, has six components and three flow targets in its configuration. Banking Logic, Database, Java UI and Prerequisites are from the BRM Stream.  Build comes from Build Scripts and Build Config comes from the stream of the same name.

The BRM Stream is scoped to only include only a subset of the components.

Assume that we wanted to add the Database component to the scope.  Using SCM CLI command similar to that shown in the screenshot below, the scope can be changed to add it in.

This results in the flow target scope being changed.

Now to add this to the build script so it can be automated.  Observe in the editor screenshot below that the command has been added to an exec statement in an Ant build script. 

The targetStreamUUID and componentsToLoad values need to be passed to the build script.  Add these as properties of the build definitions.  For example:

When the build is requested, the componentsToLoad value can be changed.  In our original example, the Database component can be added in by editing the componentsToLoad build property at the time of the build request.

Should you be concerned that adding the ‘set flowtarget’ command to the build script will add unnecessary overhead, albeit very minimal, to every build execution even when component list isn’t changed, you can create a build script and definition that only performs the ‘set flowtarget’ command and run it when needed.

The example shown was for changing the flow target component scope.  The technique used can be applied for other build workspace manipulations needed and supported by the SCM CLI.

Thanks to Nick Edgar, RTC Build component lead, for making me aware of the ‘set flowtarget’ command and suggesting its application to the problem described.