Chapter 3: Reduced Set of Indicators Best Describing Child Well-Being
3.2 Identifying indicators: concepts and decision points
The aim here is to present a list of potential indicators that could be used to benchmark and monitor the risk of child poverty across the EU. Here the purpose of the exercise needs to be clearly distinguished from both academic exercises and general international comparative evaluations. The indicator set chosen must serve the ongoing purpose of feeding into a regular monitoring process, the aim of which is not to ‘name and shame’ by ranking country performance, but to identify indicators that can together provide a tool for individual Member States to benchmark their policy achievements and better identify areas in need of increased policy attention.
For this, policy-responsive indicators are needed that cover dimensions open to policy influence.
When a limited set of child well-being indicators is suggested, it is understood that, as well as providing the means of monitoring policies, they must be informative for the development of strategies aimed at reducing child poverty and promoting social inclusion among children in Member States. So far as the technical aspects are concerned, according to the streamlined 2006 Social OMC indicator selection criteria, an indicator should:
• capture the essence of the problem and have a clear and accepted normative interpretation;
80 See Lelkes, Platt and Ward (2009).
• be robust and statistically validated;
• provide a sufficient level of cross-country comparability, as far as practicable with the use of internationally applied definitions and data-collection standards;
• be built on available underlying data, and be timely and susceptible to revision; and
• be responsive to policy interventions, but not subject to manipulation.81 These broad requirements have been used to inform the selection of indicators from the wide range suggested in the literature.82
The criterion of general relevance is sometimes difficult to judge. There are a large number of relevant indicators, and for many there is academic support in the economic, sociological and psychological literature. To take this criterion into account, therefore, it may be useful to differentiate between various types of indicator.
The literature on human capital formation and on child development points forcefully to the use of forward-looking indicators that encompass the chances children have to accumulate knowledge and societal skills and to enhance the physical development necessary for successful adulthood. In this regard, those indicators of well-being are considered, in the first place, that are good proxies for, or predictors of, children’s future prospects, and consequently of their performance as adult members of society. Since policies to reduce poverty have long-lasting effects if – and only if – they can break the intergenerational transmission chain of poverty and social exclusion, the indicators selected need to give more attention to life-cycle elements, developmental aspects and life trajectories.
The focus of concern is also on the distribution of well-being. More specifically, in addition to the general level of well-being of children, it is suggested that the social gap between the poorer and excluded on the one hand, and the better off on the other, should be monitored. Concern with the disadvantage of children, as opposed to their well-being, is consistent with the focus of the study on identifying the combination of policies that is most likely to reduce poverty and exclusion among children in EU countries, while the conceptualisation of some positive indicators is also of importance.
The need for an indicator to capture the essence of the problem raises the issue of the aggregation level of indicators. While it is clear that, in some cases, aggregation of elementary variables from micro-surveys might help to achieve a more robust indicator (such as, for example, counting the number of items missing in a household to define material deprivation or, at a country level, averaging immunisation rates for various diseases), the calculation of so-called ‘composite indices’, which encompass information from across a range of dimensions, is not considered a suitable option here. Not only does the issue of relative weighting of the various elements need to be resolved, but also the ‘more composite’ an indicator, the less guide does it provide to policy.83[C1]
81 The setting of these criteria goes back to the Social Protection Committee 2001 decisions endorsed at Laeken. For the streamlined OMC process, see: EC DG EMPL 2006, ‘Proposal for a portfolio of overarching indicators and for the streamlined Social Inclusion, Pensions, and Health portfolios’ (adopted at 22 May 2006 SPC).
82 While we went through a large number of potentially useable indicators, we do not see the point of presenting a ‘full’ or comprehensive list here. The UNICEF paper prepared by Bradshaw, Hoelscher and Richardson (2006) reports that, in building up the UNICEF report cards, a full account of 614 separate indicators was taken; the (then) OECD list of child well-being indicators contained 40 of these elements, grouped into 18 components and aggregated into six dimensions. What is more important, however, is to present those that are relevant from the point of view of the Social OMC. Nevertheless, the list we suggest as a ‘pool’, from which the ISG can select, will still be very large.
83This is not to say that ‘composite’ indicators cannot serve as important tools for raising awareness. On the contrary: league tables based on overall child well-being indices help identify overall policy failures in the national context and are also very important in giving momentum to child mainstreaming in many countries. Also, when it is decided that composite indicators will
Statistical validation in this context means an assessment of various statistical features of the selected indicators. Especially in the case of survey-based indicators, statistical reliability (in terms of sample sizes and confidence intervals) needs to be assessed. Related to this, indicators should preferably show (statistically significant) variance across countries and be reasonably stable over time. In the case of those indicators where we have access to microdata, we try to assess the breakdowns for which we think robustness can be maintained.84
For cross-country comparability, it is important not only that there should be methodological harmonisation, but also that the risk of distortions resulting from differences in culture and custom (wording, understanding, etc.) should be minimised.
As regards availability, it is important to ensure that Member States can produce the data for those indicators selected within a reasonable interval of time. Consistent and comparable data should be available for a sufficient number of countries. However, current availability is not an absolute criterion – it might be possible to institute new surveys and data-collection activities for some indicators.
Another important criterion is policy relevance, which is interpreted as meaning indicators that can support the design, implementation and evaluation of policies. In consequence, the indicators suggested are those that can be directly used to measure the impact of specific policies or policy packages, and that are measurable, transparent and robust to policy changes. Most importantly, those with policy relevance should be responsive to policy change: that is, they should indicate phenomena for which the causal chain between policy shifts and societal outcomes is relatively straightforward.
The establishment of selected indicators of child outcome should encourage governments to consider either the effectiveness or the equity aspects (or both) when promoting and implementing policies. The latter point means that not only should favourable outcomes be pursued on average, but they should also be spread across the population of children or social groups. We take these aspects into consideration, though this distinction cannot serve as a criterion for choosing between the various potential indicators.
In addition to the above criteria, it is also important that the indicators chosen should be useful in creating links between policy analysis and the analysis of EU, national and sub-national data. The use of indicators in the policy experience of the Member States and of the OMC is instrumental in facilitating the identification of explanations for inter-country variance in policy performance, and also in measuring progress made in respect of the agreed common objectives. The indicators should, therefore, be designed in such a way that they allow a reconciliation of comparative EU-level monitoring figures with national ‘headline’ figures. In addition, the use of indicators in national settings can help in developing ‘joined-up government’, as well as a more concrete target-setting for national and sub-national agencies.85
be developed, this, in itself, creates need for collecting each element – which is positive function of for the development of monitoring tools. This is well illustrated by the impact of the child well-being indices developed under the auspices of UNICEF (see Bradshaw, Hoelscher and Richardson 2006, Bradshaw and Richardson 2009). Nevertheless, it is better to keep composites as instruments for the above. To quote Richardson (2009) on this: ‘Dimensions are useful for passing on the quick message of relative success or failures; they say where countries are in relation to comparators and competitors; but do not address why they are where they are, or indeed what or how to change. In order to answer the why, what and how questions, one must refer to the raw data.’ And this is the basic rationale behind the suggestions for individual indicators to use in a child-related portfolio for the OMC.
84 We concentrate on cell sizes that remain in various breakdowns and also on width of confidence intervals for the various point estimates. As may be expected, the robustness understood this way varies from country to country for each of the variables and each of the breakdowns. The validation ‘marks’ express the extent and severity of the emerging cell size problems.
85See more on the use of indicators in Marlier et al. (2007: 46–54).