EFFECTS OF BENCHMARKING OF ELECTRICITY DISTRIBUTION COMPANIES IN NORDIC COUNTRIES - COMPARISON BETWEEN DIFFERENT BENCHMARKING METHODS

EFFECTS OF BENCHMARKING OF ELECTRICITY DISTRIBUTION
COMPANIES IN NORDIC COUNTRIES – COMPARISON BETWEEN
DIFFERENT BENCHMARKING METHODS

Honkapuro Samuli1, Lassila Jukka, Viljainen Satu, Tahvanainen Kaisa, Partanen Jarmo
Lappeenranta University of Technology
1)Samuli.Honkapuro@lut.fi


ABSTRACT

This paper describes the different benchmarking methods used in benchmarking of electricity
distribution companies in Finland, Sweden and Norway. Data Envelopment Analysis (DEA) is
used in benchmarking in Norway and Finland. Network Pe rformance Assessment Model (NPAM)
is used in benchmarking in Sweden. Theory of these methods is presented briefly. Main focus is
on the comparison of these two very different methods.


INTRODUCTION

DEA (Data Envelopment Analysis) and many other benchmarking methods have been developed
for benchmarking of non-commercial units like schools and hospitals. In these cases the results of
benchmarking do not usually have economical effects. These benchmarking methods have then
been adapted to benchmarking of electricity distribution companies. When benchmarking is used
as a regulatory tool it has direct effects to companies profits. Therefore benchmarking methods
have very high demands on correctness and fairness.

Benchmarking of electricity distribution companies differs significantly from benchmarking of
any other industries. For example the operational environments of companies vary greatly from
one company to other. The costs of electricity distribution vary greatly depending on geographic
and demographic issues. Even companies, which seem to have similar operational environments,
could have very different costs structures.

Correctness of benchmarking is strongly dependent on input parameters. If there are too few
parameters, the differences in the operational environments of companies may not be taken
correctly into account in the benchmarking. Efficiency of company may seem to be poor if
reference companies operate in different environments with smaller cost requirements. In the
other hand, if number of parameters is too large the benchmarking may not find true
inefficiencies.


BENCHMARKING METHODS

Companies, which operate in state of monopoly, do not have pressure from the markets to keep
prices and costs at reasonable level. Therefore regulator must supervise the prices of electricity
and costs of companies. Reasonable level of costs is usually determined with efficiency

1

---

benchmarking. Most of benchmarking methods use the most efficient companies to form an
efficiency frontier in which all the companies are compared to; these methods are called frontier
methods. One of most used frontier method is Data Envelopment Analysis. Other benchmarking
philosophy is to use company itself as a reference in benchmarking. The efficient cost-level of
company is calculated with efficient fictive network built today in the same area. This is the main
idea of Network Performance Assessment Model (NPAM). There is also one major difference
between DEA and NPAM, which should be taken on account when comparing these two
methods. DEA is only a benchmarking tool, which is used as one part of regulation. NPAM is a
regulation tool, including benchmarking and regulation.


Data Envelopment Analysis

Data Envelopment Analysis (DEA) is a method for benchmarking Decision Making Units
(DMU). These units can be non-commercial units, companies, business units of one company etc.
DEA is a linear programming application where the goal is to maximize the ratio of weighted
outputs to weighted inputs. Only constraints are that weights must be positive and weights of one
company must not provide efficiency score larger than 1 to any other company. Input oriented
DEA method with variable returns on scale is very commonly used in benchmarking of
electricity distribution companies. Theory of this method is presented in equations (1) – (4).
Theory of DEA can be found more detailed in (Cooper et al. 2002).

?nu y c
j
j0 +
0
j =1
Max h






(1)
0 =
?mv xii0
i =1
s.t.

n
u y
c
j
jk +
?
0
j=1
? 1
;k = 1,…,K



(2)
?mv xiik
i 1
=




uj, vi ? ? ;i = 1,…,m; j = 1,…,n




(3)




c0 unrestricted






(4)


Where

h
= efficiency score

u
= weight of output

v
= weight of input

y
= output

x
= input


m
= number of inputs

n
= number of outputs
K
= number of DMUs
?
= small positive constraint


2

---

DEA-model, as any linear programming problem, can be presented in 2-dimension figure when
there are only 1 input and 1 output. The graphical example of DEA is shown in fig. 1.


D
Output .
C
F
B
G
E
A
H
Input

Figure 1 Example of efficiency frontier in 1 input 1 output case.

The inputs and outputs of 8 (A…H) companies are plotted in figure 1, inputs in x - axis and
outputs in y - axis. Companies A, B, C and D have been founded to be efficient since those
companies have best output to input ratio. These efficient companies forms efficient frontier,
which “envelops” points plotted in the figure. Therefore the name of method is Data
Envelopment Analysis.


DEA in benchmarking of electricity distribution companies

There are few features in the DEA model, which should be considered when using this method as
benchmarking tool. The number of utilities must be large enough for reliable results. If there are
too few utilities in benchmarking, similar reference companies may not be found for every
company. Because of this DEA model could be unsuitable for benchmarking method in countries
where number of distribution companies is very small.

Input and output parameters have obviously very strong effect to correctness of benchmarking. In
input oriented DEA model the input parameters are only parameters, which are considered to be
controllable from companies’ point of view. Therefore environmental and output parameters are
treated same way in the model. The number of parameters have direct effect to number of
effic ient companies, more parameters more efficient companies.

Weights of each parameter are chosen to show the company in the best possible light. Therefore,
weight of some parameters may be very small and changes in these parameters do not affect to
efficiency score. This kind of situation can be problematic when there is some output parameter
that has to have influence on efficiency of company. That parameter could be for example power
quality.


3

---

The smallest and largest companies are founded as efficient in all cases. In figure 1 company A is
utilising least amount of input. Therefore it is founded as efficient, no matter how much output it
is producing. The same way company D is founded as efficient since it produce largest amount of
output. Efficiency of company D remains the same, no matter how much inputs it uses. Company
D is said to be “superefficient” company. The superefficient companies are problematic, since
true efficiencies of such companies cannot be estimated with DEA.


Network Performance Assessment Model

Network Performance Assessment Model (NPAM) is a combination of rate of return and revenue
cap regulation. Reasonable level of revenues is based on costs of fictive network. Fundamentals
of NPAM are shown in fig. 2.
Capital expenses
Operational expenses
Customerspecific
Transport Labor
expenses
Network
Performance
Transmission losses
Regional network
Quality Adjustment

Figure 2 Fundamentals of NPAM.

The basic idea of NPAM is to determine reasonable revenue for each company based on costs of
fictive network. This reasonable level of revenue is called network performance, which reflects
the price level customers are willing to pay.

Input parameters of model are geographical position, energy consumption, voltage level and
yearly income from every customer and every boundary points. Fictive network is constructed
based on this information. Fictive network is radial network, which reflects the optimum network
built today. Fictive network has 4 voltage level and there is only 1 conductor type used in each
voltage level. Lines are routed directly form one point to other, without any bends. Actual
network cannot be built directly since there are buildings, routes etc. on the way. This is taken
account on length of fictive network with geometrical adjustment factor.

Network performance is calculated based on repurchase value of fictive network. Repurchase
value of fictive network is based on amount of components in fictive network and unit costs of
components, determined with cost function. The cost function used in NPAM is shown in
equation (5).

C = (
k
k + k * tanh k * x ? k






(5)
1
2
( 3 (
) ) 0
4
Where

k0…k4 = constants 0…4

x
= density of customer (length of line / customer)

4

---

This cost function, with different constants, is used to calculate the costs of lines, costs of
transformers, area costs of substations, network losses, outage cost parameters, reserve capacity
needed, expected outage costs and geometrical adjustment factor. Cost function of medium
voltage line is presented in fig. 3. Cost function of distribution transformer is presented in fig. 4.

100
90
80
70
60
50
40
30
20
10
Price of conductor [€/meter] .
0
0
50
100
150
200
Customer density (length of line [m] / customer)

Figure 3 Cost of medium voltage line in NPAM.
25 000
20 000
15 000
10 000
5 000
Price of transformer [€] .
0
0
200
400
600
800
1 000
1 200
Capacity of transformer [kVA]

Figure 4 Cost of distribution transformer in NPAM.

Quality adjustment in NPAM is calculated with equation (6).

QA = Costs of reserve capacity – (Reported outage costs – Expected outage costs) (6)

Capital cost is calculated with certain interest rate, which consists of risk-free interest and risk
supplement, and 40 years depreciation time. Operational costs are 1 percent of repurchase value
for lines and 2 percent for transformers. Customer specific costs are fixed and depend only on
voltage level of customer. (Larsson 2004)




5

---

NPAM in benchmarking of electricity distribution companies

There are some main characteristics in NPAM, which should be taken on account when using it
as a regulation tool. The fictive network is used as regulatory asset base in NPAM and the
existing network is totally ignore d. The actual network could be oversized because of historical
investments. For example growth of population in certain area could be overestimated and
network could then be oversized. However, company should be able to maintain network once
built to retain good level of reliability of supply.

Most of parameters in NPAM are calculated with cost function, which is dependent only on
customer density. Therefore the total costs of fictive network determined with NPAM are very
sensitive to changes in customer density. It should be noted that customer density is determined
with length of fictive network. The actual density of population in same area could differ
significantly from that. Customer density is also dependent on design parameters of fictive
network, for example maximum length of low voltage network.


BENCHMARKING IN NORDIC COUNTRIES

Although Nordic countries have somewhat similar geographic and demographic circumstances,
benchmarking methods used in Finland, Sweden and Norway are different. Finland and Norway
are using DEA with different parameters in benchmarking. Sweden has developed NPAM for
regulation and benchmarking.


Benchmarking of electricity distribution companies in Finland

Finnish regulator Energy Market Authority has been chosen DEA for benchmarking method to
be used in regulation. The input parameter is operational costs, output parameters are value of
delivered energy and power quality and environmental parameters are length of network lines and
number of customers. Parameters of benchmarking are shown in table 1. (Korhonen et al. 2000)

Table 1. Parameters of benchmarking in Finland.
Input
Output
Environmental factor
OPEX
Power quality
Length of network

Value of delivered energy
Number of customers


Original idea was that the results of efficiency benchmarking have direct effect to reasonable
return on capital. Since some inaccuracy in the DEA benchmarking was noticed, the error
marginal of 0,1 was included in efficiency requirement. The reasonable level of operational costs
(RC) was then be determined with DEA-score as shown in equation (7). (EMA 2002)




6

---

RC = (DEA-score + 0,1)*OPEX





(7)

Where

RC
= Reasonable operational costs

OPEX = Actual operational costs


However, Energy Market Authority has considered that it is not reasonable to decrease inefficient
company’s reasonable level of return on capital afterwards, when company was not informed it’s
inefficiency and effects of inefficiency. Therefore the authority has decided to use DEA-scores
only for rewarding companies, not for punishment. (EMA 2003) The problem is that DEA-score
of each company is calculated and informed to company afterwards, but reasonable level of
operational costs should be known in advance for pricing and planning of business.

The new regulation model comes into use at beginning of 2005. In new model, there is not
individual efficiency requirement, only the general efficiency requirement. However, individual
efficiency requirement is probably coming to use to second regulatory period (years 2008 –
2012). The efficiency requirement will most likely be based on benchmarking done by DEA.


Benchmarking of electricity distribution companies in Norway

Norway has been a pioneer in the deregulation of electricity markets. Therefore Norwegian
regulator has great experience of regulation and benchmarking of electricity distribution
companies. Regulation method in Norway is revenue cap and benchmarking method is DEA. The
X-factors, which have direct impact on revenue cap, are based on benchmarking of each
company and general efficiency requirement. Parameters of benchmarking are shown in table 2.
(Gammeltvedt 2003)

Table 2. Parameters of benchmarking in Norway.
Input
Output
Environmental factor
Network assets
Number of customers
Length of network
Number of man-labour years
Delivered energy
Normalised interruption time
Interruption time


Network losses


OPEX



Different age profiles for different grids influence to companies book values. Therefore NVE has
decided to use two different versions of benchmarking, one with book values and other with
replacement values. The X-factor is based on most favourable result for each company.
(Gammeltvedt 2003)

Benchmarking of electricity distribution companies in Sweden

Swedish Energy Agency (STEM) has been developing the Network Performance Assessment
Model since 1998. There have been many pilot tests with model and the final version is most

7

---

likely coming to use during 2004. In the regulation, the revenue of company is compared to
network performance and that ratio is called Relative Billing Ratio. If Relative Billing Ratio is
higher than 1 the company is overcharging the customers. STEM has informed that it is using
NPAM to find out certain companies, which has too high Relative Billing Ratio. Pricing of these
companies will then be investigated more closely. However, method of closer investigation has
not been published yet. (Gammelgård et al. 2003)


COMPARISON OF DIFFERENT BENCHMARKING METHODS

The methods presented above, DEA and NPAM seem to be very different but both methods are
used in evaluating the cost-efficiency of electricity distribution companies. The methods have
totally different philosophy of benchmarking and also different directing effects.


Benchmarking philosophy

In the DEA-model companies are benchmarked against the efficient frontier, which is constructed
by efficient companies. In the other words companies are benchmarked against other companies,
which operates in similar operating environment. How similar reference companies are is
dependent on parameters of DEA. In Finland environmental parameters are length of network
lines and number of customers.

In NPAM all companies are benchmarked against a hypothetical company which has costs based
on fictive network. This is major difference between DEA and NPAM. In DEA companies are
benchmarked against other companies and parameters of benchmarking are based on existing
network. In NPAM every company is benchmarked against a hypothetical company with fictive
costs. The existing network is totally ignored in the benchmarking made with NPAM.

The costs of fictive network are strongly dependent on density of customers, which describes is
company operating in city or rural area. However, density of customers is only parameter that has
effect on costs of fictive network. The companies operating in different kind of rural areas are
assumed to have similar operating environments. In practice the companies, which have similar
density of customers, could have very different cost structure. It should be advised that density of
customer is based on fictive network and could differ significantly from actual density.


Costs of outages

The price of outage is very clear directing parameter, which is used as one cost component in
network planning. If price of outage is very high, the network is developed to be as reliable as
possible. In this case the power quality is good, but also price of electricity becomes very high. In
the other hand, if price of outage is zero, the reliability could be ignored in the development of
network.



8

---

Effects of benchmarking of electricity distribution companies in Finland has been analysed in
(Lassila et al. 2002). It is shown out that if power quality is treated as technically non-
controllable factor, it’s effect to efficiency score vary significantly from one company to other.
The dependences between power quality and efficiency score can be analysed with sensitivity
analysis. If efficiency score has direct effect to company’s return on capital, the cost of outage
can also be calculated. The results of analysis was that the price of outage varied between 0 and
600 €/customer,h depending on company as shown in fig. 5. Prices of outages varied also
between two years. It should be noticed that unit of these outage costs is [€/customer,h].
Therefore the actual price of non-delivered energy [€/kWh] is higher for smaller customers. For
example, if price of outage is 100 €/customer,h, the price of non-delivered energy is 50 €/kWh
for domestic user with average effect of 2 kW and 2,5 €/kWh for industrial user with average
effect of 40 kW.

600
500
400
300
200
Price of outage [€/customer,h] 100
0

Figure 5 Prices of outages of 94 Finnish electricity distribution companies determined with DEA
benchmarking. Price of outage is 0 €/customer,h for 26 companies. (Lassila et al. 2003)

The proposal to fix this problem in Finnish benchmarking model has been made in (Lassila et al.
2003). In the developed benchmarking model power quality was measured as interruption costs,
and operational costs and interruption costs was added together as input parameter of DEA.

In Finland power quality affects to revenue of company through efficiency benchmarking. In
Norway and Sweden power quality has direct effect to revenue, calculated by similar method in
both countries. The expected outage cost is determined for every company and if actual outage
cost is larger than expected the revenue is decreased. In Norway interruption time is also one
input parameter in the benchmarking. However, power quality has only minor effect to revenue
through efficiency benchmarking compared to direct effects to revenue.

In the Swedish model the cost of outage is function of customer density as shown in fig. 6. In
Norway cost of non-delivered energy vary between 1 and 12 €/kWh depending on customer type
as shown in fig. 7.

9

---

14
planned
12
unplanned
10
8
6
4
Cost of outage [€/kWh] .
2
0
0
50
100
150
200
Customer density (length of line [m] / customer)

Figure 6 Cost of outage in NPAM. (Larsson 2004)


14
unplanned
12
planned
10
8
6
4
2
Cost of outage [€ / kWh] .
0
Industry
Commercial &
Agriculture
Households
Public
Energy intensive
Service
consumption
& Forest
industry

Figure 7. Cost of outage in Norwegian regulation. (NVE 2004)


Effects of benchmarking on network development

When using costs of fic tive network as a reference in benchmarking, the growth of loads could
become a major problem. In NPAM the fictive network is designed to be an optimal network for
current loads. The actual electricity distribution networks are designed with 20 - 30 years time
scale and design criteria has been to minimise total costs (capital costs, operational costs, losses
and outage costs) during whole operating time of network. The growing loads have been taken
into account during the planning and developing of the network. Therefore actual network seems
to be oversized for current loads and costs needed for maintain actual network could be
significantly higher than costs provided by NPAM. Reliability of network could lower if there is
not enough incentives for maintain and develop the existing network.




10

---