Appendix B: A small case study of differing representations

B.1 First analysis of ROV turn actions with CN2

These are rules obtained with the representation RT01, from data generated by the author. All the material in typewriter font has been output in that form by one of the programs. Many lines of output have been taken out, but what is left has not been altered. Inconsistencies are due to the different versions of the programs used as program development proceeded. The rules are presented here in order to illustrate the operation of CN2, and the general nature of the induced rules. However, these rules are not presented as plausible candidates for rules that humans might use. The figures in square brackets indicate how many of each class is covered by that rule. Thus, [15 0 0] means that all 15 of the examples falling into the category defined by the three conditions have Port_Ths_Half_Ahead as their action decision class. Similarly, [23 11 14] indicates that there were 48 examples within the scope of the two conditions, and of those, 23 had class Port_Ths_Half_Ahead, 11 had class Stbd_Ths_Half_Ahead and 14 had the class value NO_KEY. This latter rule is less accurate than the former one.

The larger the numbers, the broader the coverage, and the more the numbers are other than in the appropriate place, the less accurate is the rule.

     *------------------------*
     |  UN-ORDERED RULE LIST  |
     *------------------------*

IF    0.65 < rov_speed < 2.33
  AND rov_port_revs_demand = stop
  AND rov_stbd_revs_demand = h_ahd
THEN  class = Port_Ths_Half_Ahead  [15 0 0]

IF    rov_height > 1.80
  AND rov_port_revs_demand = stop
  AND rov_stbd_revs_demand = h_astn
THEN  class = Port_Ths_Half_Ahead  [7 0 0]

IF    rov_degrees < 31.50
  AND rov_speed > 0.31
  AND rov_target_range < 10.10
THEN  class = Port_Ths_Half_Ahead  [7 0 0]

IF    rov_target_range > 137.35
  AND sub_display = rov
THEN  class = Port_Ths_Half_Ahead  [23 11 14]

IF    rov_target_range < 56.75
  AND rov_port_revs_demand = f_ahd
THEN  class = Port_Ths_Half_Ahead  [16 12 10]

IF    rov_target_head > 135.00
  AND rov_speed > 0.22
  AND rov_r < 0.17
  AND 6.75 < rov_target_range < 7.45
  AND sub_display = rov
THEN  class = Port_Ths_Half_Ahead  [7 0 0]

IF    rov_height > 31.75
  AND rov_port_revs_demand = f_ahd
THEN  class = Port_Ths_Half_Ahead  [36 25 36]

IF    rov_target_range < 34.40
THEN  class = Port_Ths_Half_Ahead  [89 79 148]

IF    rov_height < 17.05
  AND rov_speed > 0.48
  AND rov_port_revs_demand = h_ahd
  AND rov_stbd_revs_demand = stop
THEN  class = Stbd_Ths_Half_Ahead  [0 15 0]

IF    rov_speed < 0.58
  AND 0.13 < rov_r < 0.31
  AND rov_target_range > 5.00
  AND rov_stbd_revs_demand = stop
THEN  class = Stbd_Ths_Half_Ahead  [0 12 0]

IF    rov_target_range < 59.80
  AND rov_port_revs_demand = h_ahd
  AND rov_stbd_revs_demand = f_ahd
THEN  class = Stbd_Ths_Half_Ahead  [0 7 0]

IF    rov_target_head < 243.00
  AND rov_port_revs_demand = h_astn
  AND rov_stbd_revs_demand = stop
THEN  class = Stbd_Ths_Half_Ahead  [0 6 0]

IF    rov_r > -0.01
  AND rov_target_range < 32.60
  AND rov_stbd_revs_demand = f_ahd
THEN  class = Stbd_Ths_Half_Ahead  [1 5 0]

IF    rov_degrees < 104.00
  AND rov_stbd_revs_demand = f_ahd
THEN  class = Stbd_Ths_Half_Ahead  [10 27 21]

IF    rov_target_range > 124.15
  AND sub_display = rov
THEN  class = Stbd_Ths_Half_Ahead  [38 37 36]

IF    5.45 < rov_target_range < 10.50
THEN  class = Stbd_Ths_Half_Ahead  [35 36 52]

IF    rov_target_range < 97.15
  AND rov_stbd_revs_demand = f_ahd
THEN  class = Stbd_Ths_Half_Ahead  [29 41 35]

IF    sub_display = ship
THEN  class = NO_KEY  [0 0 340]

IF    sub_display = umb
THEN  class = NO_KEY  [0 0 61]

IF    rov_speed < 0.65
  AND rov_r > -0.09
  AND rov_target_range > 4.95
  AND rov_stbd_revs_demand = h_ahd
THEN  class = NO_KEY  [0 0 21]

(DEFAULT) class = NO_KEY  [194 185 654]

And these are the results of testing those rules, first with the data from which they were derived, and second from another similar set of data. The default rule accuracy in these cases would be 654 / 1033 = 63.3% and 320 / 444 = 72.0%.

        PREDICTED 
ACTUAL     Port_Th Stbd_Th NO_KEY  Accuracy 
Port_Ths  79.00   21.00   94.00     41%
Stbd_Ths  56.00   38.00   91.00     21%
  NO_KEY  85.00   34.00  535.00     82%
Overall accuracy: 63%

        PREDICTED 
ACTUAL     Port_Th Stbd_Th NO_KEY  Accuracy 
Port_Ths  31.00    9.00   21.00     51%
Stbd_Ths  42.00    5.00   16.00      8%
  NO_KEY  60.00   16.00  244.00     76%
Overall accuracy: 63%

     *------------------------*
     |  UN-ORDERED RULE LIST  |
     *------------------------*

IF    rov_off_head > 43.00
  AND rov_height < 40.20
  AND rov_speed < 1.33
  AND rov_target_range < 133.35
  AND sub_display = rov
THEN  class = Pure_Turn_Stbd  [14 0 0]

IF    rov_off_head > 8.50
  AND 6.25 < rov_target_range < 7.60
THEN  class = Pure_Turn_Stbd  [6 0 0]

IF    rov_off_head > 29.50
  AND rov_height > 34.45
  AND rov_speed < 1.15
  AND rov_target_range > 121.75
  AND sub_display = rov
  AND rov_turn_demand = strait
THEN  class = Pure_Turn_Stbd  [12 0 0]

IF    10.85 < rov_height < 42.95
  AND rov_speed < 0.15
THEN  class = Pure_Turn_Stbd  [5 1 0]

IF    rov_off_head < -29.50
  AND rov_height > 4.20
  AND 0.09 < rov_speed < 2.85
  AND sub_display = rov
THEN  class = Pure_Turn_Port  [0 17 0]

IF    sub_display = ship
THEN  class = NO_KEY  [0 0 224]

IF    -10.50 < rov_off_head < 11.00
THEN  class = NO_KEY  [0 0 73]

(DEFAULT) class = NO_KEY  [34 23 331]

The performance of these rules looks better, but in fact this is due to the greater proportion of NO_KEY events. The overall accuracy is still very similar to the performance of the default rule by itself.

        PREDICTED 
ACTUAL     Pure_Tu Pure_Tu NO_KEY  Accuracy 
Pure_Tur  34.00    0.00    0.00    100%
Pure_Tur   6.00   17.00    0.00     74%
  NO_KEY  52.00    0.00  279.00     84%
Overall accuracy: 85%

        PREDICTED 
ACTUAL     Pure_Tu Pure_Tu NO_KEY  Accuracy 
Pure_Tur   8.00    1.00    0.00     89%
Pure_Tur   7.00    4.00    1.00     33%
  NO_KEY  17.00    2.00  182.00     91%
Overall accuracy: 87%

The accuracy of the default rule in the first case (training examples) is 85.3%. In the second case (test examples), it is 201 / (9 + 12 + 201) = 201 / 222 = 90.5%

B.2 Rework using the ordered mode in CN2

**RULE FILE**
@
Examples: RT01s09a.exs
Algorithm: ORDERED
Error_Estimate: LAPLACIAN
Threshold: 10.00
Star: 10
@

*ORDERED-RULE-LIST*

IF    sub_display = ship
THEN  class = NO_KEY  [0 0 340]
ELSE
IF    sub_display = umb
THEN  class = NO_KEY  [0 0 61]
ELSE
IF    rov_speed < 0.65
  AND rov_r > -0.09
  AND rov_target_range > 4.95
  AND rov_stbd_revs_demand = h_ahd
THEN  class = NO_KEY  [0 0 21]
ELSE
IF    rov_height > 2.00
  AND rov_r < 0.03
  AND rov_target_range < 4.85
  AND rov_stbd_revs_demand = stop
THEN  class = NO_KEY  [0 0 15]
ELSE
IF    rov_target_head > 2.50
  AND rov_speed > 6.06
  AND rov_port_revs_demand = stop
THEN  class = NO_KEY  [0 0 10]
ELSE
IF    0.35 < rov_height < 1.10
  AND rov_speed < 1.75
THEN  class = NO_KEY  [0 0 6]
ELSE
IF    rov_target_head < 333.00
  AND rov_speed < 0.09
  AND rov_port_revs_demand = stop
THEN  class = Port_Ths_Half_Ahead  [5 0 0]
ELSE
IF    rov_speed < 0.58
  AND 0.12 < rov_r < 0.31
  AND rov_target_range > 5.80
THEN  class = Stbd_Ths_Half_Ahead  [0 12 0]
ELSE
IF    rov_target_head < 187.50
  AND rov_r > 0.20
THEN  class = NO_KEY  [0 0 6]
ELSE
IF    rov_height < 33.50
  AND rov_speed > 0.49
  AND rov_port_revs_demand = h_astn
THEN  class = NO_KEY  [0 0 7]
ELSE
IF    rov_target_head < 264.00
  AND rov_speed > 0.20
  AND rov_port_revs_demand = h_astn
THEN  class = Stbd_Ths_Half_Ahead  [0 7 0]
ELSE
IF    rov_r < 0.02
  AND rov_target_range > 161.15
THEN  class = Port_Ths_Half_Ahead  [5 0 0]
ELSE
IF    rov_port_revs_demand = h_ahd
THEN  class = NO_KEY  [2 27 30]
ELSE
IF    rov_speed < 2.33
  AND rov_target_range < 134.90
  AND rov_stbd_revs_demand = h_ahd
THEN  class = Port_Ths_Half_Ahead  [18 0 0]
ELSE
IF    rov_degrees < 31.50
  AND rov_port_revs_demand = stop
THEN  class = Port_Ths_Half_Ahead  [11 0 0]
ELSE
IF    rov_port_revs_demand = stop
  AND rov_stbd_revs_demand = h_astn
THEN  class = Port_Ths_Half_Ahead  [6 0 0]
ELSE
(DEFAULT) class = NO_KEY  [147 139 158]

1033 examples!
        PREDICTED 
ACTUAL     Port_Th Stbd_Th NO_KEY  Accuracy 
Port_Ths  44.00    0.00  150.00     22.7%
Stbd_Ths   0.00   18.00  167.00      9.7%
  NO_KEY   0.00    0.00  654.00    100.0%
Overall accuracy: 69.3%

444 examples!
        PREDICTED 
ACTUAL     Port_Th Stbd_Th NO_KEY  Accuracy 
Port_Ths   7.00    1.00   53.00     11.5%
Stbd_Ths   3.00    1.00   59.00      1.6%
  NO_KEY  19.00    4.00  297.00     92.8%
Overall accuracy: 68.7%

The default rule accuracy in these cases would be 654 / 1033 = 63.3% and 320 / 444 = 72.0%. The better relative accuracy of the first set can be easily explained, as that was the training set, i.e., the set of data from which the rules were derived in the first place.

**RULE FILE**
@
Examples: RT2s09a.exs
Algorithm: ORDERED
Error_Estimate: LAPLACIAN
Threshold: 10.00
Star: 10
@

*ORDERED-RULE-LIST*

IF    sub_display = ship
THEN  class = NO_KEY  [0 0 224]
ELSE
IF    -10.50 < rov_off_head < 11.00
THEN  class = NO_KEY  [0 0 55]
ELSE
IF    rov_height < 52.30
  AND rov_speed > 1.68
  AND rov_r < 0.00
  AND rov_target_range > 49.60
THEN  class = NO_KEY  [0 0 29]
ELSE
IF    rov_off_head < -30.50
  AND rov_height > 4.20
  AND rov_speed > 0.09
  AND rov_r < 0.01
THEN  class = Pure_Turn_Port  [0 17 0]
ELSE
IF    rov_off_head > 39.50
  AND rov_height < 40.20
  AND rov_speed < 1.33
  AND rov_target_range < 133.35
THEN  class = Pure_Turn_Stbd  [14 0 0]
ELSE
IF    rov_target_range < 6.25
  AND rov_av_revs_demand = stop
THEN  class = NO_KEY  [0 0 7]
ELSE
IF    rov_off_head > 15.00
  AND 0.02 < rov_speed < 0.34
THEN  class = Pure_Turn_Stbd  [9 0 0]
ELSE
IF    rov_off_head < -12.50
  AND rov_r > -0.09
  AND rov_target_range > 6.35
THEN  class = NO_KEY  [0 0 7]
ELSE
(DEFAULT) class = Pure_Turn_Stbd  [11 6 9]

388 examples!
        PREDICTED 
ACTUAL     Pure_Tu Pure_Tu NO_KEY  Accuracy 
Pure_Tur  34.00    0.00    0.00    100.0%
Pure_Tur   6.00   17.00    0.00     73.9%
  NO_KEY  14.00   16.00  301.00     90.9%
Overall accuracy: 90.7%

222 examples!
        PREDICTED 
ACTUAL     Pure_Tu Pure_Tu NO_KEY  Accuracy 
Pure_Tur   5.00    2.00    2.00     55.6%
Pure_Tur   1.00    7.00    4.00     58.3%
  NO_KEY   5.00    5.00  191.00     95.0%
Overall accuracy: 91.4%

Again, we can easily calculate the performance of the default rule. In the first case it is 331 / 388 = 85.3% and in the second case (not training data) it is 201 / 222 = 90.5%.