5.26. KobrA - Vending machine UML diagram
11.1.2. Verification and validation of component-based systems
Notions
• Verfication checks that software satisfies the specification.
• Validation checks that software fulfills customer requests.
11.1.3. Verification and validation of component-based systems
Figure 1. Development dimensions
11.1.4. 11.1.2 Historical overview
11.1.5. Verification and validation of component-based systems
Historical overview
• Program verification related researches began at 60's of the last century.
• Reasons:
• Computers began to become more widely used
• Hardware components became more cheaper, which extended the possibilities of computing. Without efficient, reliable software development models, technologies and tools the software crisis became true.
• Software crisis
• Computer hardwares were developed more rapidly and their speed increased radically, while the increase of software creation efficiency remain backward.
11.1.6. Verification and validation of component-based systems
Historical overview
• While experts was searching for the reasons, it became clear that the poor quality of softwares came from the high complexity of problems being solved and also from the high complexity of the software systems.
• Programming methodology was introduced against the complex problems of software development. The goal is to provide such technologies and software development methods, that can guarantee the production of reliable softwares in the available time using the given budget.
11.1.7. Verification and validation of component-based systems
Historical overview
• There are two ways to achieve the goal above.
• The analytical way is using verification tools to create correct (sequential or parallel) programs according to the given specification.
• Nowadays, software development technologies combine the advantages of the two approach, extending them with the usage of automatic and visual tools.
11.1.8. 11.1.3 Correctness proof methods
11.1.9. Verification and validation of component-based systems
Correctness proof methods
• Several methods are known to prove the correctness of sequential or parallel programs.
• Key elements of a proof method:
• A specification language
• General part - a deduction system
• Logical part - axioma schemas to help to derive a sort of axiomas from the text of the program
• Special part - a description of the program domain, e.g. Peano axiomas in case of integer numbers.
• A programming language
• Next, the notions of testing will be outlined and then the special testing problems of component based systems will be discussed.
11.1.10. 11.1.4 Testing
11.1.11. Verification and validation of component-based systems
About testing
• Testing is a widely used and accepted software verification and validation method.
• Testing is used to decide whether a software fulfills the requirements, and the specification given in the documentation.
• Testing can be used only to show that the software is working properly for the used test cases, but it is not enough to say that the software is error-free.
• A good example is the explosion of the Ariane 5 rocket in June 4., 1996.
• An exception raised, when the software converted a 64bit floating point number to 16bit signed integer.
Software did not handle this case, which lead to a computer crash, and then the rocket exploded.
11.1.12. Verification and validation of component-based systems
Testing process
• Testing process should begin at latest after the analyzing phase of development.
• Testing requires so detailed preparation as software development.
• The testing scenario should be made, which consists of
• test cases and those
• testing steps
• These should be written so detailed that everybody can perform testing.
11.1.13. Verification and validation of component-based systems
Performing test It is important to test the system from as much viewpoint as we can.
• The ergonomics test is performed at first.
• Displays should be viewed so as a user does that.
• Place of buttons
• Order of fields
• Amount of controls in windows etc.
• Creating testing environment
• It is a complex task and here the test environment creation of components is discussed in details.
• Functional test
• Here we checks that system functions satisfy the requirements.
• Required fields are placed to where the user asked.
• We get the required output for a given input.
11.1.14. Verification and validation of component-based systems
Performing test
• Performance test checks that a software modification does not cause a significant change in the operation.
• When a new version is created, modifications should be tested again.
• This test is followed by the regression test, which checks that system modifications did not cause improper operation in the unmodified software parts.
• Testing can not be done in an ad hoc way. Component based software development has special properties, those affect also the testing process. We will review these and draw up recommendations.
11.1.15. 11.1.5 Test environment
11.1.16. Verification and validation of component-based systems
Creating test environment for components
• Main challenges of component based software testing
• Component test in a new environment
• The internal operations of a component are usually unknown in the case of reusage.
• What kind of test should we choose and how long should we test our component based system?
• We try to find a solution for these suggestions.
11.1.17. Verification and validation of component-based systems
Unified Modeling Language (UML)
• UML is an industrial standard, that is created to specify and design software systems.
• UML is supported by many business and open source tools:
• Rational's Rose
• Verimag's Tau
• Sybase's Power Designer
• etc.
• Modeling and testing can be combined following two orthogonal dimensions.
• Model based testing
• Test modeling
11.1.18. 11.1.6 Model based testing
11.1.19. Verification and validation of component-based systems
Model based testing
• Support of testing with UML models
• Defining test cases
• Checking the final implementation of the system
• For traditional testing methods, basic categories are:
• Error classification
• Using UML we can define our requirements and test specification with same notations.
11.1.19.1. 11.1.6.1 UML based testing
11.1.20. Verification and validation of component-based systems
UML model based testing
• There are many similarities between the UML model based testing methods and the regular code based testing techniques.
• Source code can be taken as the representation of a concrete software system.
• It contains more details, but less information about the perspective of the system.
• UML models embody a more abstract representation of these systems.
• It contains less information, but shows more of the complete system.
11.1.21. Verification and validation of component-based systems
Importance of UML in testing
• UML has unified notations for description and testing of the system from the top abstraction level to the most concrete description.
• During code base testing, we give scenarios for a given covering criteria.
• We can do similarly, if instead of testing the system, we test a more abstract description, the UML model of the system.
• A new criteria appears naturally at the level of models: the covering of models.
11.1.21.1. 11.1.6.2 Black box testing
11.1.22. Verification and validation of component-based systems
Black box testing
• During the functional test, we ignore completely the internal mechanisms of the system or component, and we focus on the outputs of the system, that is responsed for the selected inputs. It is a typical black box testing method.
• At functional testing, we analyse how test cases can be derived from the functional specification.
11.1.23. Verification and validation of component-based systems
Extending black box testing for models
• Functional test cases generating techniques are based on analysis and partitioning of the input domains.
• A domain defines a part of the input space, which is related to the tested component.
• Domains can be defined by algebraic expressions and inequalities.
• Partitions can define equivalent classes for the input space.
• This method can be extended to UML models by behavioral equivalent classes, which exist in behavioral model form.
• One kind of the UML behavioral models is the flow chart, with we can examine behavioral equivalences.
11.1.24. 11.1.7 Testing techniques
11.1.25. Verification and validation of component-based systems
Testing techniques
• Domain Analysis and Partition Testing
• Equivalence Partitioning
• In this case, we are partitioning the input space using equivalence relations.
• Category Partitioning
• This method is traditionally used in industrial applications, when the design specification is transformed to test specification.
• State-Based Testing
• State-based testing checks the correctness of the component implementation according to the state model of the component.
• The testing scenario is based in this case on the states and state transitions.
11.1.26. Verification and validation of component-based systems
Testing techniques 2.
• Method Sequence-Based Testing
• This method is used to check the correctness of the realization of a component based system.
• For component based systems, this is the most important test.
• The design of test cases is based on the behavioral models, which can be defined with UML flow charts.
11.1.27. Verification and validation of component-based systems
Testing techniques 3.
• Message Sequence-Based Testing
• This testing method can be used to check the collaboration of different components.
• The design of test cases is based on UML collaboration diagrams.
• Such UML models are given as sequence or interaction diagrams.
• Hereafter, we examine the usability of each models in testing.
11.1.28. Verification and validation of component-based systems
Model based testing We can create diagrams with UML about different views of the system, that is being made.
These views can be categorized into two groups
• User views
• Use case diagrams
• Architectural views
• Structural view
• Class and object diagrams
• Functional and behavioral views
• Sequence, collaboration and activity diagrams, flow chart
• Implementation view
• Component and deployment diagrams
• Environmental view
With UML diagrams we can specify
• what should our system do
• how should the system behave
• how can we realize the system
11.1.29. 11.1.8 Model based testing
11.1.29.1. 11.1.8.1 Use case diagrams
11.1.30. Verification and validation of component-based systems
Use case diagram based testing
• Two kind of testing are possible
• Use case diagrams based testing
• Use case patterns based testing
• Similar to black box testing, but here testing is performed in a higher abstraction level.
• Use case diagrams based testings do not support the regular test case designing methods, the pre- and post-condition based methods, because at this level of details the solution does not exist.
11.1.31. Verification and validation of component-based systems
Elements of use case diagrams for testing - 1.
• Important elements for testing
• An actor may exist in more use cases.
• Test should reflect here the usage of the system from the view of the actor.
• A use case may contain more actors.
• In this case, concurrency hould be examined in the system.
• A use case can be a variant of an other use case ( extends )
• Similar correctness tests should be performed here, as in the case of inheritance relations testing of the object oriented programs.
11.1.32. Verification and validation of component-based systems
Elements of use case diagrams for testing - 2. A use case may contain use cases ( uses )
• It can be handled as the previous case.
• Use cases refer to high level components.
• Each referred object in use cases can appear as a component in the architecture of the system, so a relation can be made between the structural diagrams and the use cases of the system.
• A use case can be realized with one or more nested or interacting components.
11.1.32.1. 11.1.8.2 Requirement based testing
11.1.33. Verification and validation of component-based systems
Requirement based testing
• In case of requirement based testing, we should try to cover all the involved components.
• It is really important in those cases, where we change the requirements.
• We have to check the architecture of the component and we may have to modify the components as well.
• A regression test should be performed to be sure about the correctness of the modifications.
11.1.34. Verification and validation of component-based systems
Elements of use case diagrams for testing - 3.
• A component can support more use cases
• In this case the architecture of the component is not functionally coherent, the cohesion is small.
• This often
• can be taken as a bad practice or
• use case may imply a usage of a given component, so testing should be designed so, that we consider concurrency.
11.1.35. Verification and validation of component-based systems
Use case diagrams in testing process
• Use case diagrams are mainly used to determine, identify the goal of testing and to achieve a high level of test case covering.
• In reception tests we can define the following criterias
• Testing or at least covering of
• all use cases
• all actor use cases
• all includes , extends and uses combinations.
• This corresponds to that all nodes and arrows are covered in the use case diagram.
• It is consistent with the criteria of node and arrow covering in flow charts.
• Based upon the available information we can only determine which user functionality should be tested, but we do not know how.
11.1.36. Verification and validation of component-based systems
Example - Vending Machine For use case diagram in next slide, we define a testing component given in the table after the figure.
11.1.37. Verification and validation of component-based systems
Figure 2. Concepts of a use case diagram, specification of the vending machine
11.1.38. Verification and validation of component-based systems
Table 1. Identification of external tester components for the VendingMachine from the use case diagram.
11.1.39. Verification and validation of component-based systems
Figure 3. Containment hierarchy of the test organization.
11.1.39.1. 11.1.8.3 Use case and functionality based testing
11.1.40. Verification and validation of component-based systems
Use case and functionality specification based testing
• Use case diagrams have few information, so they are unable to give a specification of the system, which tells what functionality we require.
• Use case patterns can be used better.
• Patterns are collected in Table 2.
• KobrA specification patterns of methods in Table 3. are closely related to the patterns defined here.
• If a component is taken as a single system, then the specification of the methods and the corresponding system level use case specifications are essentially the same.
11.1.41. Verification and validation of component-based systems
Use case and functionality specification based testing
• Examining the tables, we can determine that descriptions contain similar information at different levels.
• It is obvious that doing use case pattern based testing and the method's specification based testing is a functional or black box testing.
• Use case descriptions specify the functionality at system level, while the operation specification describes the functionality at object or component level.
11.1.42. Verification and validation of component-based systems
Table 2. Identification of external tester components for the VendingMachine from the use case diagram.
11.1.43. Verification and validation of component-based systems
Table 2. Identification of external tester components for the VendingMachine from the use case diagram - continued.
11.1.44. Verification and validation of component-based systems
Table 3. Operation specification template according to the KobrA method and Fusion.
11.1.45. Verification and validation of component-based systems
Use case descriptions of the example
• Because of use case descriptions and the specification of the methods have basically the same information on different abstraction levels, therefore functional testing methods can be based also on use case descriptions, but in a higher abstraction level.
• In vending machine example, test cases related to the first use case can be found in Table 4.
• Table 5. contains the use case definitions of PurchaseItem and based upon this, a relation can be defined between use case descriptions and derived tests.
11.1.46. Verification and validation of component-based systems
Table 4. Abstract tests, derived from the use case description for PurchaseItem.
11.1.47. Verification and validation of component-based systems
Table 5. Definition of the use case PurchaseItem from the vending machine usage model.
11.1.48. Verification and validation of component-based systems
Testing techniques All of the abstract tests in Table 4. are based on deductive testing techniques
• Testing basic cases
• Testing the required sequence of events according to a use case.
• Testing special cases
• Testing unexpected events.
• Requirements testing
• Testing all traceable use case.
• Testing properties defined in user documentation.
11.1.48.1. 11.1.8.4 Structural model based testing
11.1.49. Verification and validation of component-based systems
Structural model based testing
• Structural diagrams specify the architectural relations among the components.
• It may seem like strange to combine structural questions with testing activities, because testing is always based on a functionality or behavior, not on a structure.
• We can get the information from structural models that what should we test in a system, which is built from a lot of interacting entities.
• Structural models play similar role in testing as use cases.
11.1.50. Verification and validation of component-based systems
Component Containment Diagrams and Testing
• Primary components of a system are typically identified at the level of requirement analysis, related to the modeling of use cases.
• After the typical components of the system are identified, we can create a draft structure of the system.
• The structure of the vending machine is shown in Figure with UML notations.
• The modified specification structural model of the vending machine is shown in Figure 5., where a newer activity of the operator is also named.
• Figure 6. shows the testing component of the vending machine, which is derived from the containment model.
• In Figure 6. we show only those test cases, which simulate the currently tested parts of the vending machine.
• If all tests were successful, then the integration of the components is successful.
11.1.51. Verification and validation of component-based systems
Figure 4. Development-time containment tree for the vending machine.
11.1.52. Verification and validation of component-based systems
Figure 5. VendingMachine amended specification structural model.
11.1.53. Verification and validation of component-based systems
Figure 6. Tester components for the vending machine derived from the containment model.
11.1.53.1. 11.1.8.5 Class, Package, Component, Deployment diagrams and testing
11.1.54. Verification and validation of component-based systems
Class, Package, Component, and Deployment Diagrams and Testing
• These UML diagrams are used to describe the implementation and deployment of components.
• Class diagrams are mainly used in component specifications and implementations, but they can be also used to describe how to split logical software components for hardware nodes.
• Classes and packages are very similar at first sight and the formerly introduced testing methods can be used to them.
• They also have custom properties
• The basic difference between classes and packages that e.g. connections among classes stay longer fixed.
Components are designed to reused in different environments and they can also change, while classes are the basic building rocks of components, so they can be reused not so easy.
11.1.55. Verification and validation of component-based systems
Class, Package and Component Diagrams and Testing
• From the view of testing these differences are not significant. The difference is the time when we test them.
• Class diagrams show the connections between the basic building rocks of a component. These connections are more or less constant in the component's lifecycle, so it is sufficient to test them only once.
• Components are building rocks of a component based system, and each time a component is used in a new application, the integration test should be performed.
11.1.56. Verification and validation of component-based systems
Deployment Diagrams and Testing
• Deployment diagrams represent an other aspect of the same problem.
• Components are assigned to nodes in distributed environment. In this case, a problem is that some kind of nodes are untouched in the life cycle of the system, so we only need an initial test, while other nodes are always changing and the integration should should be performed after each change.
11.1.57. Verification and validation of component-based systems
Structural diagrams
• Structural diagrams are essential in the development of the testing architecture.
• Structural diagrams are essential in the development of the testing architecture.