Analytical methods of impact assessment

Researchers of impact assessment projects quite often succeed in collecting sufficient data that enables them to perform certain quantitative calculations. This body of data is the basis of the analysis that results in the identification of the impacts. In its simplest form the analytical steps consist of generalisation, summation and comparison. However, impact assessment may rely on more complex analytical methods.

The selection of the appropriate analytic method can be facilitated by studying impact assessments made in other countries of the same regulation or of a regulation that serves similar aims. In particular, RIAs analyzing the introduction of certain pieces of European legislation should rely on the findings of the equivalent RIAs produced previously in other countries being in similar situations. By using these RIAs as benchmarks, previously hidden details of the regulatory environment can be revealed.

The most frequently applied analytical methods can be classified according to the following typology.

(A) Cost-benefit analysis. For many years cost-benefit assessments were the most wide spread impact assessment methods. Their purpose is to quantify, in the financial sense, the positive and negative consequences of interventions, in particular decisions concerning infrastructure development or concerning the implementation of policy programmes.

The calculation goes as follows. The researchers define a time interval in which the impacts of the decision will appear. For the calculation they need two time series to be available: in case of ex ante analysis this is a forecasting exercise which must produce the time series of costs and the time series of benefits that are attributable to the intervention. These costs and benefits must be quantified in monetary terms. The researchers use various heuristic hypotheses and/or mathematical models to estimate the two time series of costs and benefits and the resulting indicators.

• Cash flows. Then they calculate the time series of cash flows (which for every year equals the difference of costs and benefits).

• Pay-back time. Following this, the researchers calculate the duration of the investment being returned.

• Net Present Value. Subsequently the researchers calculate the net present value (NPV) of the investment. The net present value is calculated by discounting the yearly differences of costs and benefits (i.e. the cash flow stream) with the help of a previously externally determined discount rate. In other words, for every year of the evaluated time interval, following the planned intervention, the values of these yearly cash flows are projected to the present time with the help of an appropriate discount interest rate. ²³ If the net present value is positive, this indicator can be used for justifying the planned intervention.

• Internal Rate of Return. Additionally, impact assessors frequently use the so-called internal rate of return (IRR) of the investment. The internal rate of return is a calculated discount interest rate which if applied, yields a zero net present value to the investment ²⁴. If the calculated internal rate of return is negative or low the donor may prefer alternative projects, which yield better interim cash flows.

Following this, a sensitivity analysis is carried out. This is the study of how sensitive the predicted outcomes such as costs, benefits, present values and rates of returns are to changes in the assumptions.

The practical realisation of cost-benefit assessment, however, has its limits, because the credibility of the forecasted costs and benefits may be low. Moreover, there are certain impacts which cannot be expressed in monetary values. Although attaching monetary value to saving a life is an everyday practice in the insurance industry, however it is not ethical to make decisions based on a comparison of values of human lives on the one hand, and other assets on the other hand. Similarly, the monetisation of environmental benefits has its limits.

Moreover, cost-benefit analysis applies certain models to forecast costs and benefits to occur in future years, and these models are often debatable. For the above reasons, in recent years quantitative cost-benefit analyses are approximated, substituted or enriched by qualitative analyses, e.g. by comparative case studies.

(B) Cost-effectiveness analysis. The problem of monetising benefits can be avoided, if the researchers apply so-called cost-effectiveness analysis. The starting point of this analysis is that while the costs attributable to an intervention can be more or less precisely quantified, benefits cannot. That is why this approach either (a) compares alternative plans that guarantee the same benefits and chooses from among these the cheapest one, or (b) compares various alternatives characterised by the same cost level and chooses the one which promises the most favourable benefits.

(C) Compliance cost analysis. This type of analysis applies algorithms for computing the costs of complying with the regulation for those units or stakeholders which are affected by the legal rule (companies, households, local governments, banks, etc.). Compliance costs include both administrative costs (i.e. information obligations) and investment costs attributable to the regulation (e.g. a filter to avoid pollution). While compliance cost analysis is a relatively straightforward method to quantify the regulatory burdens, its disadvantage is that it does not take into consideration the indirect and long term economic effects of the regulation.

(D) The Standard Cost Model. This method measures a subset of compliance costs, the so-called administrative costs. In 2003 the governments of certain European countries have established an international network for impact assessment in order to focus on the analysis and decrease of company level administrative burdens. The member countries of the network committed themselves to a uniform methodology of administrative burden measurement, the so-called Standard Cost Model (SCM) . The following countries are parts of the network:

Austria , Belgium , the Czech Republic, Denmark, the United Kingdom , Estonia, Finland , France, Ireland , Poland, Latvia, Luxembourg , the Netherlands , Hungary , Germany , Norway, Italy and Sweden . The OECD has also entered the network and has applied the method of the Standard Cost Model to rate its member countries from the point of view of the extent of administrative burdens imposed on enterprises, government agencies and citizens. ²⁵

The Standard Cost Model (SCM) ²⁶ serves for the quantification of the administrative burdens imposed on enterprises by regulations. This method can be applied for measuring the impacts of a single legal rule or the aggregate effects of a group of regulations. The method clearly defines, which costs of the enterprises are to be regarded as administrative burdens caused by a specific regulation. According to the cost typology of the SCM,

23 See e.g. [Török-Papanek 2004].

24 See e.g. [Török-Papanek 2004].

25[OECD 2003]

26[SCM 2005]

a regulation may cause the following costs to the enterprises: direct financial costs, compliance costs and long term structural costs.

• Direct financial costs are normative obligations of the enterprise to pay a certain sum to the enforcing authority such as fees, taxes and procedural duties which have to be paid when applying for a permit.

• Substantive compliance costs are investment costs of developing the company in terms of equipment, infrastructure or organisation, in order to satisfy the requirements of a regulation. For example, an environmental regulation may compel the company to install a filter in order to decrease pollution to a new level. Or a labour safety regulation may compel the firm to improve the physical conditions of labour safety.

• Administrative compliance costs arise when the firm must create a documentation or collect data in order to satisfy the requirements of a regulation. Such obligations arise when the firm must manage and administer the maintenance of an equipment which was previously purchased in order to comply by a regulation.

Alternatively, some regulations impose the duty of collecting some information on enterprises, the costs of which are classified in the SCM method as administrative compliance costs.

As of the latter item, the Standard Cost Model measures only those administrative costs which an enterprise must pay only for meeting the demands of the examined regulation. Consequently, regular administrative costs necessary for managing the enterprise, which would be parts of company expenditures even if the regulation was removed, are not included in the administrative burdens calculated by the SCM.

While applying the SCM method for administrative costs, the relevant administrative activities are monetised by collecting and calculating the following cost parameters:

• The unit price of an administrative activity can be calculated from the hourly rates paid for this activity within the enterprise, by taking into consideration also the overhead costs. If the administrative activity is outsourced, the hourly fee paid is applied in the formula.

• Duration, the quantity of time devoted to an administrative activity in one firm.

• Frequency of the administrative activity: how often a given administrative activity has to be carried out annually in one company.

• Size of enterprise population (number of enterprises) affected by the regulation.

With the help of the above parameters the aggregate costs of a given administrative activity can be calculated for all affected firms: the administrative costs arising as the consequence of the regulation by applying the SCM formula:

Yearly administrative costs attributable to the regulation =

= Unit Price x Duration x Frequency x Size of affected enterprise population

An example for the application of the SCM formula: an administrative activity takes 3 working hours (duration), the hourly pay of the administrative employees is 10 EUR (unit price). The activity is carried out in 100.000 enterprises (enterprise population), and it is to be done twice a year (frequency). Thus the cost total of the administrative activity will be: 10 x 3 x 100.000 x 2 = 6.000.000 EUR.

To compute the total burden for the enterprises, the above sum should be complemented by (a) the direct financial costs paid by the affected companies and by (b) the substantive investment that is necessary for being in compliance with the regulation, as estimated for the aggregate of the population.

The basic unit of observation of the SCM method is the so-called ―normally efficient business". This refers to a typical enterprise within the target group of the regulation, which is doing its administrative activity with an efficiency that is neither much better, nor much worse than what is characteristic for other enterprises within the target group. ―Normally efficient businesses" can be defined and their procedures may be revealed by making expert interviews with stakeholders of the affected company population or with experts of regulatory agencies, consultancies, associations of interest representation or NGOs.

(E) Risk analysis characteristically analyses hazards related to safety, health and environmental issues. This is a method to estimate (a) the level, i.e. the extent of disadvantageous consequences of decisions and (b) their probability, i.e. the likelihood of these unfavourable events to occur. In the context of policy analysis, studies applying risk analysis often compare risks under alternative regulative scenarios, one of these scenarios being the baseline scenario, i.e. that no regulation is issued at all.

(F) Multicriteria analysis is a structured approach aimed at reaching several different aims or satisfying multiple preferences by choosing one option from various decision alternatives. It is supposed that each option will have consequences against various social, environmental or economic criteria. In order to find the optimal choice, a scoring, ranking or weighting must be defined, which expresses the importance or monetary value attached to each of these criteria. As a rule, these scores are determined with the help of expert surveys or stakeholder surveys. This approach may involve subjective elements or expert judgment.

(G) Sectoral and macroeconomic models. The impact mechanism of certain regulations upon the production, turnover and prices of various economic sectors can be expressed by a system ofeconometric equations.

Analogously, it may be straightforward to calculate the impact of some regulations on the supply or demand of some production factors (e.g. on labour force). Sectoral and /or macroeconomical models based on such equations may help to calculate the subsequent indirect effects of increasing or decreasing compliance costs related to a planned regulation. Another class of macroeconomic impact assessment models explain the level of GDP of several countries by using several explanatory variables, where some of these variables are characterising the regulatory behaviour of the government.