In an experiment we can choose the set of events within certain bounds. Every event in such a set of events should be observable, that is we should be able to decide whether the given event has occurred during the experiment. On the other hand, the set of events should be sufficiently large so that we can mathematically formulate all questions that arise in connection with the experiment. If the sample space consists of an infinite number of elementary events, then it can cause mathematical difficulties to assign a probability function to the set of events that is in accordance with our experiences.
We require that the results of the operations performed on the events (complement, sum, product, difference) are themselves events. To put this requirement differently, the set of events should be closed with respect to the operations. Moreover, we would like the certain event and the impossible event to be indeed events. In the general case we do not demand that subsets of the sample space with exactly one element be events. Our expectations for the set of events are satisfied by the following so-called event algebra.
If Ω is the sample space, then a set A, consisting of certain subsets of Ω, is an event algebra if the following three statements hold:
(a) Ω∈ A.
(b) For all E ∈ A,E ∈ A.
(c) If for all i∈N+,Ei ∈ A, then
∞
P
i=1
Ei ∈ A.
An event algebra with the above three properties is called a sigma algebra.
Chapter 2 Probability
In this chapter first we introduce the notion of relative frequency. With the help of this, we define the intuitive notion of probability, which will be followed by its axiomatic introduction. Then we deal with probability spaces, and, finally, summarize the basic relations that are valid for probability.
2.1 Relative frequency, probability
When an experiment is performed ”independently” several times, we talk about asequence of experiments. The independent execution of the experiments intuitively means that the outcome of any execution of the experiment is not influenced by the outcome of any other execution of the experiment. For example, when flipping a coin or rolling a die, the thrown object should rotate sufficiently many times so that its earlier position does not influence noticeably the outcome of the given toss. Later we will exactly define what is meant by the independent execution of the experiments (see Section 3.6).
Let us perform the experimentn times independently, and observe how many times a given event E occurs. Let us denote this number byk (or kE), and call it thefrequency of event E.
The ratio nk shows in what proportion eventE has occurred during the n executions of the experiment. This ratio is called the relative frequency of event E.
Example 2.1. When flipping a coin, let us use the notationhif after the flip the face-up side is ”head”, and the notation t if after the flip the face-up side is ”tail”. We flip a coin 20 times one after the other ”independently”, and let the finite sequence of outcomes be
t t h t h t t h t t h t h h h t h h t t.
The frequency of the side ”head” is 9, and its relative frequency is 209 = 0.45. Denote the frequency of the side ”head” in the nth flip by kn, then its relative frequency is knn, where
n = 1,2, . . . ,20. The 20-element finite sequence of relative frequencies of the side ”head”
in the above sequence of experiment is 0 Example 2.2. The following three sequences of experiments on coin flips are also inter-esting from a historical point of view. The results of the experiments are given in Table 2.1.
name of experimenter, number of frequency relative frequency year of birth and death flips of heads of heads
Georges Buffon (1707-1788) 4040 2048 0.5080
Karl Pearson (1857-1936) 12000 6019 0.5016
Karl Pearson (1857-1936) 24000 12012 0.5005 Table 2.1: Data of three sequences of experiments on coin flips.
On the basis of our experiences obtained in the experiments theprobability of an event can intuitively be given as the number around which the relative frequencies oscillate.
In problems arising in practice, we do not usually know the probability of an event. In such a case, for want of anything better, the relative frequency obtained during a consid-erably large sequence of experiments can be considered as the (approximate) probability of the given event.
The above notion of probability is only intuitive because in it we find the undefined notion of the oscillation of relative frequencies.
LetA be an event algebra on the sample space Ω. The function P: A →R is called probability function if the following three statements are satisfied:
(a) For all events E ∈ A,P(E)≥0.
(b) P(Ω) = 1.
(c) If for all i ∈ N+, Ei ∈ A, moreover, if E1, E2, . . . form a sequence of pairwise mutually exclusive events, then P
∞
(So, in the case of a(n infinite) sequence of pairwise mutually exclusive events, the probability that one of the events occurs is equal to the sum of the (infinite) series of the probabilities of the events. This is calledsigma additivity ofP, where
’sigma’ is for ’infinite’.)
If E ∈ A is an event, then the number P(E) assigned to it is called the probability of event E.
Condition (c) can be applied also for a finite sequence of pairwise mutually exclusive events. That is to say, if with condition (c)P(En+1) = P(En+2) =. . .= 0 holds, then we get that for pairwise mutually exclusive eventsE1, . . . , En,P
n P
i=1
Ei
=
n
P
i=1
P(Ei). This is what we call the additivity of probability function. If n = 2, then for two mutually exclusive events E1 and E2 the above condition yields P(E1+E2) =P(E1) +P(E2).
If the sample space is finite, then the sequence of events in condition (c) is the impossible event with a finite number of exceptions. Therefore, in a finite sample space sigma additivity is identical to the additivity of function P.
In the definition of probability the requirements (a), (b) and (c) are called Kol-mogorov’s axioms of probability. All statements valid for probability can be derived from these axioms.