In Sec. 3.2.4 we presented how the trained HMM and HSMM models can be used for segmenting time-multiplexed archive video streams oine. In this section we describe our novel unusual event detectors, which can be used in real-time applications, and we also show example anomaly detection results detected by both HMM and HSMM-based methods. For demonstrating the performance of our detectors we use three low-quality outdoor videos. These detectors are built on the trained
3.3.1. HMM-based Detector 39
models discussed in Sec. 3.2.3. In our experiments we used more than 1000 video frames for training.
3.3.1 HMM-based Detector
For the unusual event detection, we have to dene the probability of the incoming observationOt
at timet. Therefore, we utilize the Markov property of the system, and we use the previous state Qt−1 in a Bayesian formulation. First, we introduce the unknown previous state of the system to express that we are currently in the rst time step of the process, or the previous state was anomalous. The unknown state is denoted bys−1. In case of HMM the probability of observation Otis generated by statesi given the previous stateQt−1 is
P(Ot, Qt=si|Qt−1,λ) =
πibi(Ot) ifQt−1=s−1,
ajibi(Ot) ifQt−1=sj , (3.1) where we use πi and aji as priors in the Bayes rule, and bi(Ot) is the likelihood function (see Sec. 2.3). Now we use the classication presented in Sec. 2.3 to determine thes? state, where the Eq. 3.1 posterior is maximal, i.e.
s?= argmax
si∈S
P(Ot, Qt=si|Qt−1,λ)
. (3.2)
Then for unusual event detection we dene the probability that the observationOtis usual as
P?(Ot) =P(Ot, Qt=s?|Qt−1,λ) . (3.3) Since the HMM does not contain explicit duration information, it cannot be used for alerting the unusually long or short camera duration. The HSMM-based detector will address this gap.
3.3.2 HSMM-based Detector
In case of HSMM we extend the Bayesian formulation of the HMM-based detector introduced in the previous section, and we additionally use the number of consecutive state repetitions since the last transition. Therefore, in our detector we use a counter, which stores the elapsed time since the last state change.
Let the Qϕ(t)−1, Qϕ(t), ∆t process denote that at time ϕ(t) = t−∆t+ 1 the system changed from Qϕ(t)−1 previous state into the Qϕ(t) current state and no state change occurred
3.3.2. HSMM-based Detector 40
since then, i.e. we are currently inQϕ(t)for∆tduration, and let Qϕ(t)−1=s−1, Qϕ(t), ∆tdenote the transition from an unknown state. In the following we use theε(t) =ϕ(t)−1 expression as a shorthand. Furthermore, letτˆi denote the maximal duration of statesi with non-zero duration probability,Qφ(t)i.e.
ˆ
τi= max
1≤τ≤Dmax
τ :di(τ)6= 0, (3.4)
andri(τ)the probability of the most probable duration of statesi longer thanτ, i.e.
ri(τ) = max
τ <δ≤ˆτi[di(δ)] . (3.5)
Fig. 3.4 demonstrates the variable dened in Eq. 3.4, and the Eq. 3.5 probability.
sj
ˆ τi
z }| {
si si si si si si si si si si sk
di(·)
ri(τ)
Figure 3.4: Denition of the maximum state durationτˆi and the remaining duration probability ri(τ).
Assuming an incoming observationOtat time twe use a similar Bayesian formulation and classication as in Sec. 3.3.2 to nd the currents? state which maximizes the posterior, i.e.
s?= argmax
si∈S
P Ot, Qt=si| Qε(t−1), Qϕ(t−1), ∆t−1 . (3.6) Please note that the main dierence between the above equation and Eq. 3.1 of the HMM detector is that here we additionally use the elapsed time since the last state transition, which is stored in
∆t−1. The values ofQϕ(t)−1andQϕ(t)are updated when state transition occurs, and the value of
∆t−1 is maintained at each time step by our algorithm, which is discussed in Sec. 3.3.3.
First we create the detector for the case when the last transition occurred from unknown state, i.e.Qε(t−1)=s−1, where we can dene three cases:
1. the previous state is unknown and the current state is also unknown (i.e. the process has just started);
2. we arrived from an unknown state into a valid state, then the system remains in the same
3.3.2. HSMM-based Detector 41
state (i.e. no state change occurs);
3. we arrived from an unknown state into a valid state and currently a state change is in progress.
Thus we dene the probability of observation Ot and statesi at time t, given the current state Qϕ(t−1), the unknown previous state and∆t−1=τ elapsed time as
Please note that the second case of the above equation prevents the system to remain in a given state for unusually long duration, while the remaining two cases were already dened in Eq. 3.1.
Now we dene the probabilities in the case when the previous state is known. Assuming that the current state isQϕ(t−1) =si, the elapsed time is∆t−1 =τ, the probability that the process remains in statesi is
P Ot, Qt=si| Qε(t−1)6=s−1, si, τ The rst case in the above equation limits the duration according to Eq. 3.4, and excludes unusually long durations, while the second case assumes that the probability of the state duration is the maximum from the possible durations (i.e. we assume that the systems remains in a state for the most probable duration).
Finally, we dene the probabilities for the state changes in the process, assuming a known previous state. We dene the probability that the process will change from sj current state to si6=sj as Contrary to Eq. 3.8, the rst case in the above equation excludes early state transitions, while the second case uses the Markov property of the model as in Eq. 3.1.
Similarly to the HMM-based detector we select the most probables? state of Eq. 3.6, and
3.3.3. Anomaly Detection 42
dene the probability of observationOtbeing usual at timetas
P?(Ot) =P Ot, Qt=s?| Qε(t−1), Qϕ(t−1), ∆t−1
. (3.10)
3.3.3 Anomaly Detection
In Sec. 3.3.1 and Sec. 3.3.2 we constructed two probabilistic detectors to nd anomalous events in time-multiplexed camera streams. For an incoming Ot observation at time t, these detectors produce the probabilityP?(Ot)and the estimateds?state (camera) at a time. In our experiments, anomaly detection is performed by comparingP?(Ot)to a presetTuthreshold. If the probability is low (or its negative logarithm is large), then the system indicates an anomalous event, and in this case unknown previous stateQt−1 = s−1 (HMM-based detector), and unknown previous and current stateQε(t)=Qϕ(t)=s−1 with initial ∆t= 1repetition (HSMM-based detector) was assumed in the next time step. The pseudocode of our HSMM-based detector algorithm is given below.
initialization: Qε(0)=s−1,Qϕ(0)=s−1,∆0= 1 while incomingOtdo
calculate posterior: evaluate Eq. 3.73.9 nds?: evaluate Eq. 3.6
calculateP?(Ot): evaluate Eq. 3.10 if P?(Ot)< Tuthen
alarm: unusual event
Qε(t)=s−1,Qϕ(t)=s−1, ∆t= 1 else
Qt=s?
if Qt=Qϕ(t−1)then
∆t=∆t−1+ 1 else
Qε(t)=Qϕ(t) Qϕ(t)=Qt
∆t= 1 end end end