4.6. Evaluation 59Figure 4.10 Schematics of the evaluation process for the overall localization frame-work performance.can be designed and combined with synthetic measurements generated from radiopropagation models.As the running of the localization algorithms is only a CPU-bound task, in contrastto the radio propagation, this is done locally by using the same components thatare also contained in the server deployment. Such a visualized localization result areshown in figure 4.9.4.6 EvaluationThe evaluation of the different localization algorithms is performed by the Evaluator component that is shown in figure 4.1. The Evaluator has access to the corpus ofposition annotated measurements that were manually collected for different Wi-Ficapable devices.The Evaluator is configured by different parameters that determine the target device,the used environment with radio propagation Model and the individual behaviouralproperties of the localization algorithms. The environment is constituted by a 3D-scene, with the relevant dimensionalities. The environment is enriched with SSMs ofall configured APs for a selected material parameter optimization run. Furthermore,the resolution of the state space in ratios of the radio propagation resolution canbe defined. A 1: 1 ratio leads to 2 · 106states with 20 cm edge size in the UMICscene. At least, the localization algorithm is defined and the corresponding defaultconfiguration values can be overridden.The evaluation is started by selecting a subset from the available evaluation corpus.These selected paths are then evaluated in parallel by running multiple Localizerinstances. The different result sequences are compared with the available correctpositions and different types of errors are computed. The HMM and the PF localizerreturn an offline, an online and an averaged online result sequence(see 4.3.1.5 and4.3.2.4). The different types of errors contain the averaged error over all typed se-quences in 2D and 3D form. Furthermore, the median error over all typed sequencesfor each dimensionality is reported. The median error is more robust to negative