Towards Distributed GQM

Towards Distributed GQM
Alessandro Bianchi*, Danilo Caivano*, Filippo Lanubile*, Francesco Rago°, Giuseppe Visaggio*
*Dipartimento di Informatica, Università di Bari, Italy
{bianchi, caivano, lanubile, visaggio}@di.uniba.it,
°Italy Solution Center, EDS Italia, Caserta, Italy, francesco.rago@
Abstract
Global software development has a big impact on current software engineering practices. Current quality
frameworks and improvement strategies need to be adapted for distributed software engineering
environments. GQM is an approach to measurement that can be used within the context of a more general
strategy to software quality improvement. We present a variant of GQM, called D-GQM (Distributed-
GQM), whose goal is to address new requirements from distributed and cooperative organizations.
Introduction
Business globalization and in particular high-technology business are growing due to a continuously expanding market full of new opportunities and resources. Global Software Development (GSD) [IE01], i.e., developing software from remotely located sites, is increasingly being adopted by software suppliers because of the following reasons:
- new business markets
- new business opportunities
- to capitalize on the global resource pool, wherever located;
- to take advantage of proximity to the market
- to take advantage of expert’s knowledge wherever located
- to take advantage of time zone differences for software development;
As reported by Herbsleb and Moitra in [HM01], “This change is having a profound impact not only on marketing and distribution, but also on the way products are conceived, designed, constructed, tested and delivered to the customer”. However, a series of disadvantages like the need of ad hoc methodologies to manage larger and geographically distributed work groups [Coc00], tools for exchanging and sharing knowledge [NFK97, SY99], extra overhead to face and overcome communication problems with staff [ED01], must to be taken in to account.
Herbsleb and Moitra characterize the dimensions of the GSD problem [HM01]:
- strategic issues: the decisions made to divide and distribute work among different sites, would let the sites operate as independently as possible while providing for an adequate communication across them;
- cultural issues: due to the fact that staff involved in co-operative work have different cultural backgrounds;
- inadequate communication: the distribution of work in different sites increases the cost of formal communication between team members and limits informal exchange of information that in a usual working environment contributes to sharing experience and co-operating for the achievement of a common goal.
- knowledge management: the knowledge management is more difficult in a distributed context. This means that information may not be exchanged informally without a standard procedure and adequate rapidity; also the consequences of poor management can negatively influence and limit reuse.
- project and process management issues: in distributed environments the project management activity is a critical success factor. The distribution of sites negatively impacts on synchronization and scheduling of activities.
Furthermore, cultural differences and attitudes must be considered too.
- technical issues: the distribution of sites in disperse locations imply the use of different hardware and software platforms, with derived problems related to incompatible data formats exchanges or the use of different tool versions.
All these dimensions have also a strong impact on product and process quality. Thus, the quality frameworks and improvement strategies [WCR97] need to be adapted to distributed contexts. The Goal-Question-Metric (GQM) [BR88, BW88], represents a structured and systematic approach for data collection, based upon the specific needs of the project and the organization. Furthermore, as reported in [BCR94], “It can be used in isolation or, better, within the context of a more general approach to software process improvement.”.
This work suggests a new measurement approach, called Distributed-GQM (D-GQM), developed to overcome the limits of the classic GQM in distributed contexts.
The GQM
The main idea behind GQM is that measurement should be goal-oriented and based on context characterization.
According to [BCR94], the measurement model has three
levels:
- Conceptual Level (GOAL): a goal is defined for a
specific purpose based on the needs of the organization, for a variety of reasons, with respect to various quality
models, from various points of view, relative to a
particular environment.
- Operational Level (QUESTION): a set of questions is used to characterize the way the achievement of a
specific goal is going to be performed.
- Quantitative Level (METRIC): a set of collectable data is associated with every question in order to
quantitatively answer them.
In the interpretation phase measurements are used to answer
the questions and to conclude whether or not the goal is
attained. Thus, GQM uses a top-down approach to define
metrics and a bottom-up approach for analysis and
interpretation of measurement data.
GQM defines a dynamic quality models [LV94] on which basing an effective measurement program. In a colocated project, quality goals reflect the business strategy and GQM is used to identify and refine goals based on the characteristics of software processes, products and quality perspectives of interest. This is done considering the entire organization as shown in Figure1. Furthermore a colocated organization can be considered internally homogeneous referring to competencies, maturity and processes in use.
If we refer to distributed environments, what has been mentioned to this point is not always true. The sites that make up a distributed organization can be heterogeneous and then a quality model, built using GQM, need to be refined in different ways (depending on site characteristics), or a different quality model should be adopted for each site. This poses new problems at different levels:
- Conceptual Level: one of the most important aspects of GQM is the domain characterization through the inclusion
of the context in the goal specification. In the case of distributed environments the sites may differ for ability, maturity and responsibilities. Therefore characterization of one site is not necessarily extendible to (and then valid for) other sites. Each site might focus only on a small portion of the quality goals accepted by the entire organization. In fact if we consider that software processes are spread amongst the sites, each site might have different goals based on the business process it supports.
- Operational Level: the questions used to refine a goal might change because of site heterogeneity. Therefore a site
may not be interested in a certain aspect of a goal expressed in a question. Based on the maturity of a site the same goal can be refined with different questions that focus on certain aspects rather than others.
- Quantitative Level: the choice of metrics used to quantitatively evaluate a question is a critical aspect. Collected
measures should be familiar, easy to use and tailored to the sites in which they are used. A quality question can be refined using different metrics for each different site, so that metrics are well known in that context and reuse of existing metrics is promoted. In fact, the introduction of new and unfamiliar metrics causes some disadvantages such as purchasing new measurement tools, training people, resistance to adoption.
The D-GQM
A distributed organization is formed of N geographically distributed sites. Therefore we can assume that each site can consider all or only part of the goals, depending on the supported software processes. For example, a software factory can be composed by the following sites: s 1, analysis site; s 2, project site; s 3, s 4, coding sites, s 5,s 6 testing sites . The different characteristics of each site, suggest that they have different goals. Thus, each site will contribute to satisfy a specific subset of goals.
As shown in Table 1, site s 2 supports the quality goals g 2 and g 3 and the site s 3 supports the quality goals g 3 and g 5. The resources available, the levels of expertise in various technologies, tool used, processes maturity, capability and so on, characterize each site.
Figure 1: GQM used in a colocated organization
Even if two sites share the same goals and support the same processes, they might have the need to refine the quality model in two different ways. A typical example is the case of a large organization that takes over a smaller one. The integration takes time because the smaller organization will need to adapt to new standards, procedures, and technologies. Suppose that both goals g 4 and g 5 (including questions and metrics) are defined as shown in tables 2 and 3. Sites s 3 and s 4 (table 2) pursue the same goal and execute the same processes (coding). Goal g 4 is refined by both sites s 3 and s 4, using the questions q 1 and q 5. Every site is different in terms of competencies, resources and abilities. Therefore code metrics such as Halstead’s complexity metric [Hal77] may be too expensive for site s 3, because it does not have a measurement tool and manual measurement requires too much effort. On the other hand site s 4 does not have the same problem because it can automatically measure this metric. Site s 3 has a tool to measure McCabe’s cyclomatic complexity [MB89]. In such cases, although the same questions are used, D-GQM allows specializing the metrics in order to tailor them to the characteristics of each site. In this way, s 3 can use McCabe’s cyclomatic complexity to measure code complexity rather than Halstead’s metric, and still be consistent with the question. This is one of many possible ways of measuring complexity as mentioned in [Zus91].
In table 3, it can be seen how site s 5 uses only one question to refine the goal g 5 while site s 6 uses both q 7 and q 9. Furthermore s 5 uses one metric m 17 to refine question q 7 of goal g 5. For example, suppose that site s 5 performs testing only on safety-critical systems. For these systems, software quality is the higher priority, whatever effort is spent to achieve defect-free software. Under these conditions the measurement of both m 8 and m 10 might be useless in terms of achieving g 5 because it is not necessary to make a tradeoff between quality and cost (determined by testing effort). Thus, only question q 7 will be considered.
On the other hand, site s 6 performs testing on business information systems. In order to achieve g 5, we can think of measuring the density of defects found (q 7) and the effort spent on testing (q 9). This choice makes it possible to find the desired tradeoff between cost and quality, in order to maximize the economical benefits for the organization.
Each goal can be refined with different questions. A quality factor expressed by a goal is identified by many questions, finally each question is analyzed in terms of what measurements (in terms of metrics) are needed in order to be answered. The D-GQM allows using different quality models depending on the characteristics of the site.
s 1 s 2 s 3 s 4 s 5
s 6 g 1 ■
g 2
■ g 3
■ g 4
■ ■ g 5
■ ■
Table 1: cross-reference
goals-sites
g 4: Analyze the coding activity for the purpose of evaluation
with respect to effectiveness
from the viewpoint of the
software engineer.
s 3 s 4 m 11: Halstead complexity
■ q 1: what is the
complexity if the code produced?
m 15 : v(G)for
each module
■ q 5: what is the
productivity of each
programmer?
m 16: LOC/hour
■ ■
Table 2: Decomposition of goal 4 g 5: Analyze the system test process for the purpose of evaluation with respect to effectiveness from the viewpoint of the software engineer. s 5 s 6 q 7: What is defect density? m 17: number of defects / LOC ■ ■ m 8: effort per activity ■ q 9:What is the effort for test execution? m 10: effort per person ■
Table 3: Decomposition of goal 5
The interpretation process of a quality model can be adequately represented with decision trees. In fact if we consider the structure of GQM, we can see how it is possible to go through the GQM-tree starting from the goals down to the lower levels corresponding to the metrics. At this point strengths and weaknesses are located by comparing values of metrics and baselines fixed. These values are a reference for the improvement process following the interpretation phase.
The D-GQM tailors the classic GQM with respect to the conceptual, operational and quantitative levels overcoming the limits mentioned.
−At the conceptual level it is possible to distribute the goals amongst the sites that compose the organization as shown in figure 2. This allows assigning goals to those sites that can actually satisfy them rather than to others.
Each goal can then be characterized to the specific context. Furthermore, the strengths (+) and weaknesses (-) p1....p n deriving from the interpretation phase are not necessarily distinct for each site, as shown in figures 3 and 4.
−At the operational level the same goal can be refined with different questions as shown in table 3. In figure 3 the goal g5 is refined in two different ways. Therefore the decision tree associated to the goal is different. The tree in figure 3.a leads to different strengths and weaknesses than those in figure 3.b. In fact in this last case it is not possible to identify the two weaknesses p3 and p4. Nevertheless both trees refer to the same goal g5. This is due to the fact that the decision tree in figure 3.b does not use one of the questions used in the tree represented in figure
3.a (The excluded question is colored in gray).
−At the quantitative level the same question can be quantified through different metrics. Figure 4 shows an example.
This kind of approach allows using all the resources of a site as shown in the example in table 2. In this case the decision tree associated to the goal (as in figure 4.a and 4.b) leads to the same strengths and weaknesses although different metrics are used.
The interpretation phase plays a fundamental role in the successful utilization of the D-GQM. It makes possible to combine results from different question and metrics into a coherent view according to the quality model adopted.
Conclusions
In this work, the limits of the GQM approach, applied within geographically distributed organizations, have been discussed and a new approach, called D-GQM, has been proposed. D-GQM is an extension of the GQM to be used in distributed contexts. This approach allows interpreting collected data, based on the characteristics of each site.
The D-GQM may provide the following feedback to the project manager: (1) achieved goals, (2) where the goals have been achieved from (which site of the organization), (3) how the goals have been achieved. Although dispersed sites might use different metrics, the project manager can evaluate the conformance of organization to the adopted quality model .
Encouraged by the potential benefits of the D-GQM in distributed environments, we will further investigate the method in order to highlight its strengths and weaknesses. This investigation will be conducted through a rigorous formalization of the method and experimentation in an industrial environment.
Acknowledgments.
This work is the result of collaboration between SER_LAB (Software Engineering Research LABoratory) from the University of Bari and EDS Italia, within the Software System Quality project.
Our thanks to Teresa Baldassarre for her insightful remarks on a first draft of this proposal.
References
[BCR94] V.R. Basili, G. Caldiera, H.D. Rombach, “Goal Question Metric Paradigm”, Encyclopedia of Software Engineering, John Wiley & Sons, Volume 1, 1994, pp. 528-532.
[BR88] V.R. Basili, H.D. Rombach, “The TAME Project: Towards Improvement Oriented Software Environments”, IEEE Transaction on Software Engineering, Vol. 14 no. 6, 1988, pp. 758-773
[BW88] V.R. Basili, D.M. Weiss, “A Methodology for Collecting Valid Software Engineering Data”, IEEE Transaction on Software Engineering, Vol. 10 no. 6, 1984, pp. 728-738
[Coc00] A. Cockburn, "Selecting a Project's Methodology", IEEE Software, July-August 2000, pp.64-71.
[ED01] C. Ebert, P. De Neve, "Surviving Global Software Development", IEEE Software, Mar-Apr 2001, pp.62-69
[Hal77]M.H. Halstead, “Elements of Software Science”, Operating, and Programming Systems Series Volume ,. New York, Elsevier, 1977.
[HM01] J.D. Herbsleb, D. Moitra, "Global Software Development", IEEE Software, Mar-Apr 2001, pp. 16-20. [IE01] IEEE Software, The Global View, Mar-Apr 2001
[LV94] nubile, G.Visaggio, “Quality evaluation in software reengineering based on fuzzy classification”, in Frontier Decision Support Concepts, John Wiley & Sons Inc, 1994, pp.119-134.
[MB89] T.J. McCabe, C.W. Butler, "Design Complexity Measurement and Testing." Communications of the ACM 32, 12 (December 1989): 1415-1425.
[NFK97] K. Nakamura, et al., "Distributed and Concurrent Development Environment via Sharing Design Information", Proc. of the 21st Intl. Computer Software and Applications Conference, 1997.
[SY99] J. Suzuki, Y. Yamamoto, "Leveraging Distributed Software Development", Computer, Sep 1999, pp.59-65
[WCR97] Y. Wang, I. Court, M. Ross, G. Staples, G. King and A. Dorling , “Quantitative Evaluation of the SPICE,CMM, IS0 9000 and BOOTSTRAP”, Proceeding of the 3rd International Software
Engineering Standards Symposium, Walnut Creek, CA, 1997, pp.
[Zus91] H. Zuse, Software Complexity measure and methods, Walter de Gruyter, 1991.。