More guidance on using IBM IoT CE solution with configuration management

Last year I announced several new guidance articles on use of our configuration management capability in Guidance on adopting IBM CLM configuration management across the lifecycle. Our workgroup continued and has several guidance articles in progress; we’ve just published the first in a series we expect to complete by end of year.

As the articles are published, I’ll update the list above with the proper links.  All of our guidance articles can be accessed here.

Advertisements

How many artifacts do I have in my Jazz application repository?

One element of sizing the servers for the IBM Continuous Engineering (CE) solution is the current and projected data scale (along with data shape, user scale and workload). There are also recommended artifacts limits to keep an application performing well, such as 200K artifacts per DNG project area (as of v6.0.5 and noted here).

Whether you are trying to project future growth based on current sizing or ensure you are staying withing recommended limits, it is useful to know how many artifacts currently exist in a repository (or other “container” such as a project area). Each application provides different means of getting this information.

DOORS Next Generation (DNG)

Vaughn Rokosz has written a very good article on the impact of data shape on DNG performance. He provides several SQL and SPARQL queries to monitor artifact counts.  I won’t repeat them here but go to the link to minimally get the queries for total number of artifacts and versions in the repository and artifacts in the project areas.

Rational Team Concert (RTC), Rational Quality Manager (RQM) and Rational Model Manager (RMM)

Since these applications share a common storage service, they have similar means to get to the artifact counts. As a Jazz Admin you can run a repotools command or a web service.

Option 1: use repotools from command line
repotools-<context>.bat -listItemStats adminUserId=<jazz admin ID> adminPassword=<jazz admin password> repositoryURL=https://<server:port>/<context>logFile=<filename>

Option 2: use web service from browser

https://<server:port>/<context>/service/com.ibm.team.repository.migration.internal.stats.IDBTableSizeHttpService/

for <context>, use ccm, qm or am for Change Configuration Management, Quality Management or Architecture Management applications.

Note that both of these options can take some time to execute so be aware of possible load put on the server. I suggest running them during lighter load times. You can first run in a test environment with production like data to get a sense of timing and load.

Sample CCM artifact counts output

 

Sample QM artifact counts output

Starting with v6.0.3, administrators can monitor Jazz application metrics through the use of JMX MBeans. One of the MBeans is Item Count Details which contains similar information as provided by the listItemStats repotools command and IDBTableSizeHttpService web service. The Item Count Details MBean, once enabled can be viewed from RepoDebug or an enterprise monitoring tool capable of receiving published JMX inputs. This is the preferred method as you can capture that data over time, see trends, set alerts and thresholds and correlate with other monitored data.

CCMMBeans

Item Count Details MBean Objects

Attachment

Attachment Item details

Monitoring Jazz Applications using JMX MBeans

I recently published a blog post on jazz.net regarding our serviceability strategy and use of JMX MBeans to monitor Jazz Applications. If you’ve heard me speak on this topic, you know that I believe that having an monitoring strategy is a best practice and essentially imperative for any deployment involving our global configuration management capability. I would even extend that to deployment of RTC clustering as well.

Have a look at the blog post here:
Monitoring Jazz Applications using JMX MBeans

Guidance on adopting IBM CLM configuration management across the lifecycle

Earlier this year, Kathryn Fryer and I formed a workgroup to share experiences we’ve had with customers adopting the IBM CLM configuration management solution. The purpose being to elicit our shared recommended practices and guidance.  The workgroup consisted of representatives from Services, Support, Development, Test, Enablement and Product Management.

When forming your Configuration Management plan we recommend it consider the following key topics:

  • End to End Process Flow (context for use of configuration management)
  • Component Strategy
  • Stream Strategy
  • Baseline Strategy
  • Change Management (including DNG change sets)
  • Cross-stream Updates
  • Reviews and Approvals
  • Traceability, Link Validity and impact analysis
  • Naming, Tagging, Custom Attribute Conventions
  • Roles and Permissions
  • Configuration aware Reporting
  • Integrations (including non-CLM)
  • Communication Plan

Our aim is to create guidance that encompasses each of these topic areas to aid you in creation of your configuration management plan.

The first such guidance is now available on jazz.net.

Our focus has been on general adoption guidance along with component and stream strategies (both are critical at the outset and go hand in hand). Next focus areas are yet to be determined but could include finishing out the stream strategies (a couple more patterns remain) baselining strategy and change management.

We value your feedback on the guidance to date and input on areas to focus on next.

If you are at the IBM Watson IoT: Continuous Engineering Summit 2017 this week in New Orleans, be sure to say hello. Check out the talk I have with Ian Zimmerman on Friday at 2:30pm (Azalea 1): CM05 – Adopting Configuration Management: What You Need to Know!

Resource-intensive scenarios that can degrade CLM application performance

About a year ago, I was asked to begin considering what scenarios could drive load on a Collaborative Lifecycle Management (CLM) application server that could lead to outages or overall diminish the end user’s quality of service.  These aren’t necessarily long running scenarios but those that are known to use large amounts of system resources (e.g. high CPU, memory, heap usage).  As such, they have been known at times to degrade server performance and negatively impact user experience.

After reviewing a number of problem reports and escalations plus several discussions with Support, Services and Development resources, I identified scenarios for several of the applications.  We coined the term ‘expensive scenarios’ though our User Assistance team recently indicated that it could be misconstrued and a more apt name would be ‘resource-intensive’.

The first set of scenarios were published in v6.0.3 and documented as Known Expensive Scenarios.  The title will be changed in the next release to be Known Resource-intensive Scenarios.

For each of the identified scenarios, there is a description of what it is and under what conditions it could become resource-intensive.  Further, if there are any known best practices to avoid or mitigate the scenario from becoming resource-intensive, these too are captured.  These practices could include adjusting some application advanced properties that tunes the scenario behavior some or a change in work practices for when and how the scenario is invoked.

For example, importing a large number of requirements into DOORS Next Generation (DNG) can consume high resources as subsequent to the import, indexing of the newly imported artifacts occurs, which can block other user activity.  When the volume of imported data is high and/or several occur at once, system performance could degrade.  The wiki describes this scenario, identifies that there are some advanced properties that limit the number of concurrent ReqIF imports as well as the recommendation that these imports be kept under 10K requirements or be performed when the system is lightly loaded.

Knowing these scenarios help in a couple of ways.  First, as your process and tools teams define usage models for one of these applications, knowing that a particular usage pattern can potentially drive load on the server leading to degraded performance allows that usage model to be adjusted to avoid or reduce the likelihood of that occurring. Second, in situations of poor performance or worse, knowing if these scenarios are occurring could help identify root cause.

This latter case is helped by the logging of start and stop markers when a resource-intensive scenario occurs.  Each marker includes the Scenario ID (from Table 1) and a unique instance ID.

ScenarioStartStopTo get additional details when the scenario occurs and to aid in understanding its characteristics, advanced (verbose) logging can be enabled.  This can be done from the Serviceability page of an application’s admin UI.  Note the enabling verbose logging does not require a server restart.

ScenarioEnableAdvLogging

Now when a performance or system anomaly occurs and the application logs are reviewed, should it have occurred during a resource-intensive scenario, you may have a clue as to cause.  The additional logging should at a minimum include the data specified in Table 2.

ScenarioAdvLogging

As part of our serviceability improvements in v6.0.3, the CLM applications publish various JMX MBeans that may be collected and trended by enterprise monitoring tools such as IBM Monitoring, Splunk, LogicMonitor and others.  MBeans exist for several application metrics including counts/occurrences of resource-intensive scenarios.

Each MBean to be published must first be enabled from an application’s admin UI advanced properties page.

MBeansEnable

After doing so, the monitoring application can be configured to capture that data and displayed on a dashboard.

MBeansStats

Having a comprehensive enterprise monitoring strategy is essential for a well-managed CLM environment.  Tracking occurrences of these scenarios and correlating them against other environment measurements give administrators (and IBM Support) insight when troubleshooting anomalies or proactively evaluating environment performance.  In a subsequent post, I will talk further about what to monitor.

 

Adopting the IBM Continuous Engineering (CE) solution Configuration Management Capability

Adopting the IBM Continuous Engineering (CE) solution Configuration Management Capability is the title of a webinar that Kathryn Fryer and I recently presented.  We’ve been working with ‘new’ configuration management capability since it was in development prior to its launch in v6.0.  Adopting it takes careful consideration in order to successfully realize its benefits.

Objectives of the presentation

In version 6, the IBM CE solution added exciting new configuration management capabilities across the lifecycle, better enabling parallel development and strategic reuse. Simply enabling these capabilities won’t help you realize their potential; you must consider changes to your process and usage model to achieve results. This presentation describes current considerations, limitations and strategies for adopting configuration management.

  • Configuration management overview
  • Trade-offs and considerations – as of current release (6.0.2)
    • Primary factors
    • Reporting
    • OSLC integrations
    • Linking
    • QM utilities
    • Additional considerations
  • Enabling configuration management
  • Upgrade and migration
  • Adoption path and additional resources

If you are interested in this presentation, you can find the replay of the webinar here in the DOORS Enlightenment Series.

The slides are shared here.

Additional Reading

Scaling the Collaborative Lifecycle Management (CLM) solution across an enterprise

It’s critical that clients understand the implications when scaling your Jazz topology by adding additional servers for a given application. Ralph and I have been documenting these considerations and discussing them with customers over the last few years. We recently gave a webinar on this topic. See the following blog post for details.

rsjazz

Scaling the Collaborative Lifecycle Management (CLM) solution across an enterprise, is the title of the webinar Tim and I presented recently. Tim and I have worked on this content for a while and consider it important. Please also see  Tim’s related post Getting to a right-sized Jazz environment. There is additional reading below.

Objectives of the presentation

Scaling the v6.0.x Collaborative Lifecycle Management (CLM) solution across an enterprise often includes multiple instances of a given Jazz application. What multi-Jazz application options are available and what are the considerations?

  • What topologies and general multi-Jazz application options are available
  • How Jazz applications such as Change Configuration Management (CCM), Quality Management (QM) and Requirements Management (RM) relate to a Jazz Team Server (JTS)
  • The impact, advantages and disadvantages of multiple CLM Applications
  • What to consider when scaling and developing usage models adopting one of these deployment options to avoid surprises

If you…

View original post 73 more words