LAS 3 RAS 3
5.7 Performance analysis
In order to investigate the effectiveness of the proposed mechanisms, I run simulations.
5.7.1 Simulations
200 mesh nodes are placed uniformly at random in a two-dimensional 10×10 unit field. The radio range is 1 unit. A node is gateway with probabilityφ, which is 0.1 in the considered scenarios. Only those scenarios are considered where each node can reach at least one gateway. If the gateways could be reached, but they are out of the View of a node, the depth of the View is enlarged which is 4 by default.
It is not necessary that there exists any route between two arbitrary mesh nodes. However, I require that any mesh node must find a route to at least one gateway. If a scenario does not fulfill this requirement I generate a new one.
The simulation is divided into rounds. In each round, a randomly chosen source builds a route to a gateway, and sends 100 data packets. After every 10th data packets, each router sends its upstream counter to the gateway. After each report, a gatewayg calculates ηi,gr(t) for each router i, and updates its table. The table at indexi stores the last 30ηi,gr(t) for eachrand t. Finally, it calculates theη(gw)i,g , and disseminates among the nodes that are in its View.
When I specified my secure routing mechanism, I stated that there are pre-established routes between access points and gateways. For the sake of simplicity, in the simulations, the access points choose from all possible routes in the subview.
I divided the whole simulations into three phases. The first phase is the bootstrap phase. The Node Trust Value of each router is initially 1, i.e. they are fully trusted, however some of them are malicious. I determined experimentally that the Node Trust Values reach their steady-state values within 2500 rounds, and therefore, I set the length of the first part of the simulation to this number of rounds. Similarly, the second phase lasted for 2500 rounds, too. In the second phase, the subset of the routers are still malicious, but here the access points have a clearer view of the network. In this phase, I collected statistics from which I investigated the properties of my mechanism (Figure5.2, 5.3, and 5.4). In the last long phase which lasts for 5000 rounds, all
the malicious nodes behave honestly. With this, I could investigate the speed of adaptivity of my mechanism.
The source of each route is an access point. Any node can play the role of the access point, but I consider only 2-hop or longer routes, otherwise none of the participants can behave maliciously due to my system and attacker model. The access points choose uniformly at random from each possible shortest path that leads to any gateways on the currently generated subview.
The route selection algorithm is based on the hop count. In each round, the access point chooses the shortest path. Note when some routers are discarded, the router select the shortest path on the subview which may increases the length of the routes. Therefore, I also investigate the path length. I utilized the Dijkstra algorithm to determine the shortest paths. The access points choose uniformly at random from each possible shortest path.
Recall that different access points may calculate different Node Trust Values as they may aggregate different Node Trust Values received from different gateways. Thus, it is difficult to plot their value. For the sake of simplicity, when I show the Node Trust Value of a router, I consider the ηi,i(ap)which is calculated by the node that the Node Trust Value is about. Note that it is the Node Trust Value where the router includes all the gateways that evaluate the router. In the following, I refer to these values simply as NTV. In Table 5.1, I summarized the fixed parameter values of the simulations.
Table 5.1: Fixed parameter values of the simulations
Field size 10 x 10
Radio range 1
Number of nodes 200
Probability of being gateway (φ) 0.1
Number of simulations 10
Simulation length 10000
Number of messages in each round 100
Size of View 4
Size of window at gateways (n) 30 Node Trust Value threshold step (ν) 0.05
Initial Node Trust Value 1
I considered different values for the probabilityς of being malicious, for the probability ϑof dropping a data packet, and for the probabilityξ of reporting the counter of the incoming data packets to the gateway. I run simulations with the values shown in Table5.2. A default scenario is described with the parameters indicated by bold text in the same table. As different scenarios do not show significant or unexpected changes, only the default scenario is analyzed in detail.
Table 5.2: Varying parameter values of the simulations Probability of being malicious (ς) 0.05 0.2 0.5 Probability of dropping a packet (ϑ) 0.2 0.5 1 Prob. of reportingcntin(ξ) 0 0.5 1
In Figure 5.1, I show a sample scenario generated with default parameters. The nodes which are neighbors are connected with a line. The gateways are denoted by large white circlesO, while the malicious nodes are denoted by large white triangles △. All other nodes are represented by small black circles.
5.7.2 Results
Recall that different access points may calculate different Node Trust Values. In the figures I show the average Node Trust Value that includes all the gateways that have evaluated the router. Recall, I refer to these values simply as NTV.
0 1 2 3 4 5 6 7 8 9 10 0
1 2 3 4 5 6 7 8 9 10
Figure 5.1: Sample scenarioGateways are denoted by O, malicious nodes are denoted by △, and regular mesh routers are denoted by.
In Figure5.2, the NTV of three different groups can be seen with the 0.95 confidence intervals.
The routers are categorized into three different groups: 1) malicious routers, 2) honest routers which are neighbors of malicious routers, and 3) other honest routers. I analyzed the latter two groups separately because the malicious routers can degrade the Node Trust Value of the neighboring nodes when the gateway evaluate the received upstream counters. At each group, four bars can be seen. The bars refer to different parameters of the malicious node detection mechanism. Theall andleast indicate the usage of Eq. (5.1) and (5.2), respectively. The NTV is aggregated using the function minimum or average when the bar is indicated withminor avg, respectively.
Honest Neighbors Malicious
0 0.2 0.4 0.6 0.8 1
Group of nodes
Average node trust value
least−avg least−min all−avg all−min
Figure 5.2: Average Node Trust Value with 0.95 confidence intervals grouped by dif-ferent node categories
As Figure5.2shows, the NTV of the honest nodes is maximal. In particular, the honest nodes are usually included in the subview which the route is selected from. In contrast, the average Node Trust Value of the malicious nodes is almost zero when the minimum function is used for the aggregation. This means that the malicious nodes are bypassed with high probability. If the Node Trust Values are aggregated by calculating the average function, the values are higher, but the difference is still significant between the average NTV of the honest and malicious nodes.
Considering the neighbors of the malicious nodes, the NTVs are relatively high, but as I expected, significantly lower than of the other honest nodes.
Note that average NTVs do not show significant differences when Eq. (5.1) or (5.2) is used.
In some scenarios (e.g.,whenς = 0.5), with the former one, the NTVs of the malicious nodes is less, but also the NTVs of the neighbors of them and the honest nodes is less. Nevertheless, the probability of a node being malicious is a priori known and exploited in Eq. (5.1), which is not a realistic assumption. The investigation of the right parameter ofqis considered as a future work.
least−avg least−min all−avg all−min no defense 9051
12439 21120
Parameters of the malicious node detection mechanism
Number of dropped messages
Figure 5.3: Average numbers of dropped data packets with 0.95 confidence intervals In Figure 5.3, the average number of dropped data packets are shown with 0.95 confidence intervals using different parameters of misbehavior node detection mechanism. These results are compared to the case when no defense mechanism is used at all. As one can see, the number of data packet drop is reduced with my mechanism considerably. It worked somewhat better with the minimum aggregation function than with the average function, which comes from the fact that the malicious nodes are excluded from the subviews with higher probability.
I also investigate the cost of avoiding malicious nodes by my mechanism. My simple QoS metric is the hop number. Thus, average length and the 0.95 confidence interval of the number of hops is shown in Figure 5.4. I indicate only above 2 hops, because it was the minimum hop number in the considered scenarios. As one can see, the length of routes does not increase significantly with my mechanism. This comes from the fact that in many cases, the access points could choose alternative routes which had the same length as the route that contained malicious routers, too.
least−avg least−min all−avg all−min no defense 2
2.5 3 3.5 4
Parameters of the malicious node detection mechanism
Average length of routes
Figure 5.4: Average lengths of the routes with 0.95 confidence intervals
In Figure5.5, the NTVs are grouped into the three group and their average value are plotted against the time. There, I investigate how fast my mechanism adapts to the case when the nodes become malicious or they are repaired. Recall that initially the routers are fully trusted and in the first part of the simulation (first 5000 rounds), some nodes are malicious, while in the last part (last 5000 rounds), the malicious nodes are repaired and do not drop any packets.
In Figure5.5(a), the process of changing of the NTV is plotted when the Node Trust Values are aggregated with average function, while in Figure5.5(b), the aggregation function is the minimum function. In both cases, the Node Trust Values are calculated with Eq. (5.2).
0 670 2500 5000 10000
0 0.2 0.4 0.6 0.8 1
Rounds
Average node trust value
Non−malicious Neighbors Malicious
(a) Average
0 25003130 5000 10000
0 0.2 0.4 0.6 0.8 1
Rounds
Average node trust value
Non−malicious Neighbors Malicious
(b) Minimum
Figure 5.5: Node Trust Value adaptation
As it is emphasized in Figure 5.5(a), the 90% of the final NTV (at 5000th round) is reached after 670 rounds. Recall that one route is evaluated in each round. In both figures, the average NTVs reach their value in round 5000 at the same speed. In contrast to this, in the second part, after the nodes are repaired, the average NTV of the misbehaving nodes return faster to 1 when the aggregation function is the average. The reasons are the following. Firstly, with the average function, the NTVs of the malicious nodes are higher, therefore, they are selected in the routes with higher probability than with minimum function. Therefore, they have more chance to increase the low NTVs. Furthermore, when a repaired routeriobtains highηi,gr(t), it falls out lowηi,gr(t)from the window maintained by gatewayg. Therefore, the router certainly increases its NTV. In contrast to this, in the case of minimum function, the lowest ηi,gr(t) may be not fallen out. Therefore, the router will have again low chance to increase its NTV.
This is always a trade-off in the reputation systems. If a node with low reputation value has many chances to correct, it has the opportunity to abuse the system. On the other hand, if a node with low reputation value is excluded from the system, it has little chance to be involved again.
I did not investigate the overhead of my mechanism in the simulator, but I think that the overhead is insignificant. In each report period each node has to send the counter value to the gateway (a node may flood the network only if its counter value did not arrive to the gateway) and the gateway floods the updated Node Trust Values in its View.