Summary of this Chapter

Chapter from book Effective software Maintenance and Evolution: Reuse-based Approach, Copyright (C) 2007 Stan Jarzabek Chapter 15 Evaluation of the mixed-strategy approach
Summary of this Chapter:
We start by summarizing salient features of the mixed-strategy approach and describing a typical project application scenario. Then, we evaluate trade-offs involved in adopting the mixed-strategy approach.
15.1 Summary of the mixed-strategy approach
Mixed-strategy supports generic design and enhances software changeability. The main motivation for building a mixed-strategy solution is to exploit opportunities offered by similarity patterns. By doing this, we gain extra levels of changeability, which matters in maintenance, and genericity, which matters in reuse. From the mixed-strategy perspective, there is no difference between maintainability and reusability. Change specifications and generic design mechanisms address these two important, but difficult to achieve in practice, software engineering goals in a unified, simple yet effective way.
Mixed-strategy is meant for developers and maintainers of complex, evolving software systems. It is applied on top of conventional programs, in synergy with design techniques supported with current programming paradigms and component platforms (such as .NET or J2EE). The main idea behind mixed-strategy is to separate concerns related to genericity and changeability, from the concerns of how programs are designed and implemented in terms of class/component architecture, platforms used, or required program runtime characteristics. By extending programs with XVCL, we deal with issues of generic design and change, without conflicts with - and in separation from - other programming concerns.
The main benefit of mixed-strategy is to avoid implementing similar concepts all over again, and to ease changes. The expected return on investment in applying mixed-strategy is from savings on program maintenance, long-term evolution and/or better reuse.
process
15.2 Mixed-strategy
Thee design of a software component architecture, and at least partial implementation of components, is a prerequisite for building a mixed-strategy solution. The reason for that is that while some of the similarity patterns are inherent in an application domain, many others become evident as we design and implement a program. Furthermore, the intended architecture and components of program(s) to be supported by a mixed-strategy solution influence the structure of that solution. Therefore, the architecture must be envisioned first. Below, we describe how we designed a mixed-strategy solution for buffer classes, and then outline a project scenario for re-engineering an existing program into a mixed-strategy solution. To make process description more meaningful, we annotated specific process steps with Remarks containing methodological guidelines.
15.2.1 Arriving at the mixed-strategy solution for the Buffer Library
To build a Java/XVCL generic representation for the Buffer library, we gained general understanding of buffer classes by manual domain analysis [20] first. At the time of the first experiment, no clone detection tools were available to us. Due to the small size of the library and its clear design, manual analysis was relatively simple. We aimed at design-level similarities (structural clones) and similar code fragments (simple clones). We identified groups of classes that we suspected would be similar. We examined methods and attribute declarations within each group which confirmed that there was much similarity among classes. We designed meta-components for each group of classes, eliminating redundant code fragments as follows: For each group of similar fragments, we created a suitable meta-fragment. As for meta-fragments that appeared in different contexts with changes, we parameterized them with meta-variables, <break> points, <select>, <insert>, <adapt> and other XVCL commands to cater for required variations. Finally, we incorporated suitable adaptation commands to the corresponding meta-classes and SPC. It took 10 days for one person, with no prior knowledge of the Buffer library but with good knowledge of XVCL, to complete the job.
15.2.2 Re-engineering of the existing program into a mixed-strategy representation Objectives: to simplify a library/program by enhancing its genericity and changeability; to enhance visibility and automate adaptive changes involved in customization of generic solutions; to reuse similar design solutions within the scope of a library or program; to design generic program solutions for possible reuse in yet other programs.
1.Apply one of the clone detection tools [3][4][11][17] to identify the extent of cloning in
the subject program.
Remark: Most of current clone detectors find only similar code fragments differing in parametric ways (e.g., in variable names, keywords or operators). This is not enough to identify design-level similarities (e.g., similar classes in the Buffer library). Clone Miner [3] applies data mining heuristics to infer possible design-level similarities.
2.Analyze application domain to identify design-level similarity patterns.
Remark: Design-level similarity patterns result in recurring program structures that may involve a number of collaborating classes. Unification of such structures is very beneficial for maintainability. It is essential to complement bottom-up semi-automated analysis of clones with manual top-down domain analysis: The top-down domain analysis and the bottom-up analysis of clones feed one another with useful inputs.
3.Analyze the subject program to identify design-level program structures that exhibit much
similarity.
Remark: Typically, there will be much cloning among such structures. Yet other structures may implement similar analysis or design concepts, with variants.
4.Identify program areas that are heavily maintained and select critical areas for re-
engineering.
5.Re-engineer the subject program incrementally to form a mixed-strategy solution. In each
iteration, build a partial mixed-strategy generic solution to unify selected groups of similar program structures.
Remark: A mixed-strategy solution must be carefully designed, and iterations must be planed accordingly in order to minimize the effort of refining the mixed-strategy solution (refining the mixed-strategy solution is analogous to refactoring of a conventional program [12]). While some re-work is unavoidable, we found that focusing on design-level, large granularity similarity patterns (as opposed to simple clones) leads to relatively stable mixed-strategy solutions. Smaller granularity similarities are then dealt within the solution frame provided by generic solutions for design-level similarities.
6.Future maintenance is done via the mixed-strategy solution which never becomes
disconnected from the base code.
15.3 Trade-offs involved in applying the mixed-strategy approach Adopting and applying techniques like XVCL does not come for free. Brining in a new technique necessarily entails cost and involves trade-offs. Therefore, to be attractive, a new technique must solve some important engineering problems that cannot be solved otherwise, providing benefits that outweigh the cost. In the attempt to meet this challenge, XVCL provides strong capabilities for designing generic, highly adaptable generic structures, with full automation and visibility of adaptive changes at architecture, detailed design and code levels. The source of XVCL’s expressive power is that mixed-strategy representation is unrestricted by the rules of the underlying programming language. The inherent dangers of moving beyond the level strictly controlled by a programming language are mitigated by maintaining 100% integration between mixed-strategy representation and program structures, and by keeping XVCL simple.
In this section, we evaluate trade-offs involved in applying the mixed-strategy approach.
15.3.1 Strengths
A mixed-strategy representation displays a clear view of commonalities and differences among program structures within or across programs. XVCL generic structures can unify a similarity patterns at any abstraction level, from design to code, and of any granularity – from a sub-system, to component, to class and to program statement in class implementation. We can specify almost arbitrary differences among similar program structures. This enhanced genericity is due to the unrestricted parameterization mechanism, which, unlike conventional generics [13], is decoupled from the language core: The overall generic design solution is decomposed into a hierarchy of parameterized meta-components, called x-frames. Mixed-strategy representation partitioning is parallel to partitioning of a program along component boundaries (at architecture level) and class, function/method and statement boundaries (at the program level). The boundaries of mixed-strategy partitioning are not restricted by rules of a programming language or the semantics of a programming problem being solved. The boundaries are solely dictated by the concerns of generic design.
We can achieve genericity at the level of mixed-strategy representation, if it is so required for ease of maintenance, and still keep clones in executable programs, which is often unavoidable [8][14][15][16][19].
Mixed-strategy representation is also operational: The XVCL Processor instantiates generic structures and automates the process of deriving specific, custom programs from the generic representation, based on the specification of the required properties of the program that is needed.
An important principle of XVCL design is clean separation of various sources of change affecting a program. Each source of change (e.g., a combination of features in our earlier examples) can be traced to codes affected by this change. Non-redundancy of a mixed-strategy representation reduces the number of points at which affected classes must be modified. Changes done to one x-frame consistently propagate to all the contexts into which the x-frame is adapted. If the impact of change is not uniform in all such contexts, the exceptions can be handled at the specific adaptation point, without directly modifying the code fragment involved. X-frames explicitly reflect the impact of change on program elements. The traceability information is helpful in program understanding and maintenance. Mixed-strategy approach shows its strength when addressing changes (or variant features in a product line) that have global impact on a system. The impact of such variants cannot be localized into a single component identified through conventional modularization approaches such as Object-Oriented analysis and design. We accommodate these variants into meta-components by parameterizing components for change, and by designing lower-level meta-components that contain the impact of variants. As it is often the case, changes (or variant features) affect many meta-components. XVCL Processor automatically propagates changes to all the affected components, releasing programmers from manually repeating routine changes.
The need for genericity and change arises in any application domain. Concepts of the mixed-strategy apply to any domain and any programming language. XVCL distils these concepts into a minimum set of domain-independent, and language-independent rules which, in a very simplified way, can be described as a “composition with adaptation” mechanism, capable of selecting software building blocks, and composing them after suitable adaptations. Depending on the level of abstraction and the goal of generic design, a “software building block” may mean a subsystem, a component, component interface, class, function or any other element that is a useful in software construction. The above concepts are universal and in one form or another appear in any application domain and implementation technique.
The XVCL solution is transparent - a programmer can intervene in any detail of the transition from the meta-level to programs. This allows XVCL to escape the problem of the techniques based on abstractions disconnected from the base code, which are found difficult to work with by maintenance programmers [8].
XVCL complements rather than competes with programming language constructs and conventional design techniques: a developer using his/her favorite programming language and design technique can escape to XVCL to tackle problems that cannot be easily solved at the program level. Therefore, the XVCL solution can be neatly integrated into other programming methodologies and environments.
15.3.2 Weaknesses and how they are addressed
Despite those benefits, two-level design in terms programming language and XVCL induces its own difficulties. Mixed-strategy introduces extra steps into the conventional software development process, and requires developers to embed usual program solutions into generic design structures. Therefore, the development of a mixed-strategy solution takes longer than the development of a conventional program, which is generally expected for the design of generic, reusable solutions. The feedback from our industry partner indicates that, in practice, the benefit of being able to deal with complexity at two levels outweighs the cost of the added
complexity [14]: the learning curve and development effort of an XVCL solution can be reasonable even for large programs (provided that an XVCL expert is also familiar with an application domain and program itself). At the same time, the return on investment may be quick and substantial. In case of the Buffer library study, it took at most 10 days for a skilled XVCL developer with no prior knowledge of the Buffer library, to analyze buffer classes and develop a Java/XVCL mixed-strategy solution. We are collecting further empirical evidence to better assess trade-offs involved.
It is more difficult to understand a generic mixed-strategy representation than a concrete program. It is also difficult to validate a mixed-strategy representation as we can derive many concrete programs from it. These problems can be mitigated by skillful design and by tools. Mixed-strategy representation can be organized into a layered architecture, based on the usual principles of the abstraction and separation of concerns. “Good design” can minimize the scope of a meta-program that has to be analyzed at any time. As lower level x-frames become stable and reliable over time, potential errors tend to be located only in top-most, context-specific and still fragile meta-components. In mixed-strategy representation, patterns of similar, recurring program structures (such as groups of similar buffer classes) are represented in a generic way. A single x-frame provides a window into multiple program structures, pattern instances, that can be obtained from a generic solution.
Tools may further simplify application of meta-programming techniques and may have significant impact on their performance. In our lab, we are working on an XVCL Workbench that helps in editing, visualizing, debugging and static analysis of XVCL solutions.
Could we do better by raising the level of abstraction of XVCL? We consider the current form of XVCL an assembly language of change specifications. XVCL contains the minimum constructs to specify any type of changes, but specifications can get tedious and overly complex. We consider adopting specified composition points, such as joint points in Aspect-oriented Programming [18], in addition to composition points explicitly marked with <break> commands. This can simplify specification of changes in certain cases. But in general, at this point, we do not know how to raise the level of abstraction without compromising the expressive power of the current change mechanism. In the future, we hope to discover meta-level abstractions that will allow us to define higher-level forms of XVCL, equally expressive but free of current pitfalls. Domain-specific, abstract specifications are certainly useful and application generators exemplify neat solutions that can be obtained in that way. While the approach works fine in narrow application domains, it does not scale to general programming paradigms in which we cannot make assumptions about the semantics of the underlying domain. Abstract specifications are intellectually appealing, but they can get easily disconnected from the actual code. In case of change specification, such discontinuity is undesirable for practical reasons [8].
As applying mixed-strategy is not the “all-win” situation, it is essential to apply qualitative and quantitative evaluations of the results. This can be done by (1) conducting controlled experiments in which two teams perform the same program modifications using traditional and mixed-strategy approach, (2) comparing the productivity index achieved with mixed-strategy against an industry productivity index (such as described in [1]), (3) analytical argumentation, and (4) metrics-based evaluation of equivalent program solutions developed using different techniques. The limitation of method (1) is that differences in productivity among programmers participating in experiments may significantly influence the results.
Method (2) can be used for mature technologies for which much experimental data is already available (for example, original frames from Netron, Inc. have been assessed in that way [1]). Method (3) is always required as it explains the rationale for the expected benefits of the technique and its pitfalls.
We use analytical argumentation, metrics (e.g., physical size and conceptual complexity), and controlled experiments (e.g., to evaluate effort involved in changes) to assess mixed-strategy solutions.
Workbench
15.4 XVCL
Figure 1. A snapshot of the XVCL Workbench
Tools may considerably simplify application of mixed-strategy. In our Lab, we are developing an XVCL IDE called XVCL Workbench that helps in editing, visualizing, debugging and static analysis of x-frameworks (Figure 1). XVCL Workbench helps developers to browse/analyze x-frameworks (in the Project Explorer window) and x-frames (in the Outline Window). structure in XML-free format. The upper right-hand-side window shows an x-frame in raw XML format. In the future, a developer will be able to ask queries about properties of x-frames, and run the Processor in a debugging mode. XVCL Workbench is implemented as a plug-in to the Eclipse platform.
References
[1]Bassett, P. Framing software reuse - lessons from real world, Yourdon Press, Prentice Hall, 1997
[2]Basit, H.A., Rajapakse, D.C., and Jarzabek, S. “Beyond Templates: a Study of Clones in the STL and Some General
Implications,” Int. Conf. Software Engineering, ICSE’05, St. Louis, USA, May 2005, pp. 451-459
[3]Basit, A.H. and Jarzabek, S. “Detecting Higher-level Similarity Patterns in Programs,” ESEC-FSE'05,European
Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering, ACM Press, September 2005, Lisbon, pp. 156-165
[4]Baxter, I., Yahin, A., Moura, L., Sant’Anna, M., and Bier, L. “Clone detection using abstract syntax trees,” Proc. Int.
Conf. on Software Maintenance 1998, pp. 368–377
[5]Bosch, J. Design and Use of Software Architectures – Adopting and evolving a product-line approach, Addison-Welsey,
2000
[6]Brooks, P.B The Mythical Man-Month, Addison Wesley, 1995
[7]Clements, P. and Northrop, L. Software Product Lines: Practices and Patterns, Addison-Wesley, 2002
[8]Cordy, J. R., "Comprehending Reality: Practical Challenges to Software Maintenance Automation," Proc. 11th IEEE
Intl. Workshop on Program Comprehension, (IWPC 2003), pp. 196-206
[9]Czarnecki, K. and Eisenecker, U. Generative Programming: Methods, Tools, and Applications, Addison-Wesley, 2000
[10]Deelstra, S., Sinnema, M. and Bosch, J. “Experiences in Software Product Families: Problems and Issues during Product
Derivation,” Proc. Software Product Lines Conference, SPLC3, Boston, Aug. 200
[11]Ducasse, S., Rieger, M. and Demeyer, S. 1999. “A language independent approach for detecting duplicated code,”Int.
Conference on Software Maintenance, ICSM’99, September 1999, Oxford, UK pp. 109-118
[12]Fowler M. Refactoring - improving the design of existing code, 1999, Addison-Wesley
[13]Garcia, R. et al., “A Comparative Study of Language Support for Generic Programming,” Proc. 18th ACM SIGPLAN
Conf. on Object-oriented Programming, Systems, Languages, and Applications, 2003, pp. 115-134.
[14]Jarzabek, S. and Li, S. "Eliminating Redundancies with a “Composition with Adaptation” Meta-programming
Technique,” Proc. ESEC-FSE'03, European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering, September 2003, Helsinki, pp. 237-246
[15]Jarzabek, S. and Li, S. ”Unifying clones with a generative programming technique: a case study,” Journal of Software
Maintenance and Evolution: Research and Practice John Wiley & Sons, Volume 18, Issue 4, July/August 2006, pp.
267-292
[16]J arzabek, S. “Genericity - a “Missing in Action” Key to Software Simplification and Reuse,” to appear in 13th Asia-
Pacific Software Engineering Conference, APSEC’06, IEEE Comp. Soc., 6-8 December 2006, Bangalore, India
[17]Kamiya, T., Kusumoto, S., and Inoue, K. “CCFinder: A multi-linguistic token-based code clone detection system for
large scale source code”, IEEE Trans. Software Engineering, 2002, 28(7): pp. 654-670
[18]Kiczales, G, Lamping, J., Mendhekar, A., Maeda, C., Lopes, C., Loingtier, J-M., Irwin, J. “Aspect-Oriented
Programming,” Europ. Conf. on Object-Oriented Programming, Finland, Springer-Verlag LNCS 1241, 1997, pp. 220-242
[19]Kim, M., Bergman, L., Lau, T. and Notkin, D. “An Ethnographic Study of Copy and Paste Programming Practices in
OOPL,” Proc. Int. Symposium on Empirical Software Engineering , ISESE’04, August 2004, Redondo Beach, California, pp. 83-92
[20]Pettersson, U., and Jarzabek, S. “Industrial Experience with Building a Web Portal Product Line using a Lightweight,
Reactive Approach,” ESEC-FSE'05,European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering, ACM Press, September 2005, Lisbon, pp. 326-335
[21]XVCL (XML-based Variant Configuration Language) method and tool for managing software changes during evolution
and reuse,
[22]Yang, J. and Jarzabek, S. “Applying a Generative Technique for Enhanced Reuse on J2EE Platform,” 4th Int. Conf. on
Generative Programming and Component Engineering, GPCE'05, Sep 29 - Oct 1, 2005, Tallinn, Estonia, pp. 237-255。