System testing involves integrating two or more components that implement any system functions or features and then testing this integrated system. In an iterative development process, system testing is concerned with testing an increment to be delivered to the customer.
In the case of a waterfall development process, system testing is concerned with testing the entire system. Therefore, system testing may have two distinct phases:
1. Integration testing. After integrating a new component the integrated system is tested.
When a problem is discovered, the developers try to find the source of the problem and identify the components that have to be debugged.
2. Functional testing. The version of the system that could be released to users is tested.
Here, the main objective is to validate the system that it meets its requirements and ensure that the system is dependable. Where customers are involved in release testing, this is called acceptance testing. If the release is good enough, the customer may then accept it for use.
Fundamentally, the integration testing is considered as the testing of incomplete systems composed of clusters or groupings of system components. Functional testing is concerned with testing the system release that is intended for delivery to customers. Generally, the priority in integration testing is to discover defects in the system and the priority in system testing, is to validate that the system meets its requirements.
9.2.1 Integration testing
The process of system integration involves building a system from its components and testing the resultant system for problems that arise from component interactions. Integration testing checks how these components work together across their interfaces.
System integration integrates clusters of components that deliver some system functionality and integrating these by adding code that makes them work together.
Sometimes, the overall skeleton of the system is developed first, and components are added to it. This is called top-down integration. Alternatively, the infrastructure components are
integrated first providing common services, such as network and database access, and functional components added after. This is bottom-up integration.
In order to locate errors easier an incremental approach to the system integration and testing is suggested. Initially, a minimal system .configuration should be integrated and tested.
Then the new components are added to this minimal configuration and tested after each added increment. If any problem arises in these tests, this probably means that they are due to interactions with the new component.
Before the process of integration, it is necessary to decide the order of integration of components. Usually, system integration is driven by customer priorities. When the customer is not involved in the developments, the components that have the most frequently used functionality are integrated first. Integrating and testing a new component can change the already tested component interactions. Errors may occur that were not exposed in the tests of the simpler configuration. This means that when a new increment is integrated, it is important to rerun the tests for previous increments as well. Rerunning an existing set of tests is called regression testing.
9.2.2 Functional testing
A release of the software is its final version that will be distributed to customers. The objective of release testing is to show that the software product delivers the specified functionality, performance and dependability, and that it does not fail during normal use.
Another name of release testing is functional testing because it is only concerned with the functionality and not the implementation of the software.
Release testing is usually a black-box testing process where the tests are derived from the system specification. Figure 9.2. shows the model of black-box testing. The testing process presents inputs to the component or the system and examines the corresponding outputs. If the outputs are not those predicted, i.e. if the outputs are in set Oe, then the test has detected a problem with the software.
Figure 9.2. Black-box testing model.
During release testing, software is often tested by choosing test cases that are in the set Ie in Figure 9.2. These inputs have a high probability of generating system failures i.e. outputs in set Oe.
To validate that the system meets its requirements, the best approach to use is scenario-based testing, where test cases are developed from scenarios. Usually, the most likely scenarios are tested first, and unusual or exceptional scenarios considered later. If use-cases are used to describe the system requirements, these use-cases and associated sequence diagrams can be a basis for system testing.
9.2.3 Performance testing
The objective of performance tests is to ensure that the system can process its intended load. An effective way to discover defects is the stress testing, i.e. to design tests around the limits of the system. In performance testing, this means stressing the system by making demands that are over the design limits of the software. Stress testing has two functions:
1. It tests the failure behaviour of the system under circumstances where the load placed on the system exceeds the maximum load.
2. It continuously stresses the system and may cause such defects that would not normally be discovered.
9.3 Exercises
What are the objectives of software testing?
Review the process of software testing!
What are the types of test?
What does the defect testing mean?
What software components are tested by unit test?
What is the regression testing?
What is the different between integration and functional testing?
What is the different between release and acceptance testing?
Explain the functional testing!
What is the objective of performance testing?
10 Embedded system development
An embedded system is a computer system with a dedicated function within a larger mechanical or electrical system that serves a more general purpose, often with real-time computing constraints. It is embedded as part of a complete device often including hardware and mechanical parts.
Embedded systems are designed to do some specific task, rather than be a general-purpose computer for multiple tasks. Some also have real-time performance constraints that must be met, for reasons such as safety and usability; others may have low or no performance requirements, allowing the system hardware to be simplified to reduce costs. Since the embedded system is dedicated to specific tasks, design engineers can optimize it to reduce the size and cost of the product and increase the reliability and performance.
The processors used in embedded systems may be types ranging from rather general purpose to very specialized in certain class of computations, or even custom designed for the application at hand. A common standard class of dedicated processors is the digital signal processor (DSP).
The program instructions written for embedded systems are referred to as firmware, and are stored mainly in read-only memory or Flash memory chips. They run with limited computer hardware resources: little memory, small or non-existent keyboard or screen. As with other software, embedded system designers use compilers, assemblers, and debuggers to develop embedded system software.
The embedded system interacts directly with hardware devices and mostly must respond, in real time, to events from the system’s environment. In the real-time systems the embedded real-time software must react to events generated by the hardware and issue control signals in response to these events.
Embedded systems control many devices in common use today. They are commonly found in consumer, cooking, industrial, automotive, medical, commercial and military applications.
Physically, embedded systems range from portable devices such as digital watches and MP3 players, to large stationary installations like traffic lights, factory controllers and largely complex systems like hybrid vehicles, MRI. Telecommunications systems employ numerous embedded systems from telephone to cell phones. Many household appliances, such as microwave ovens, washing machines and dishwashers, include embedded systems to provide flexibility, efficiency and features. Transportation systems from flight to automobiles increasingly use embedded systems.
Computers are used to control a wide range of systems from simple domestic machines to entire manufacturing plants. These computers interact directly with hardware devices. The software in these systems is embedded real-time software that must react to events generated by the hardware from the environment of system and issue control signals in response to these events.
Software failures are relatively usual. In most cases, these failures cause inconvenience but no serious, long-term damage. However, in some systems failure can result in significant economic losses, physical damage or threats to human life. These systems are called critical systems. Critical systems are technical or socio-technical systems that people or businesses depend on. If these systems fail to deliver their services as expected then serious problems and significant losses may result. Modern electronic systems increasingly make use of embedded computer systems to add functionality, increase flexibility, controllability and performance. However, the increased use of embedded software to control systems brings
with it certain risks. This is especially significant in safety critical systems where human safety is dependent upon the correct operation of the system.
The objective of this chapter is to introduce main characteristics of critical systems and the implementation techniques that are used in the development of critical and real-time systems.