Evaluation | ERC GeoDiverCity

New Paper, in JASSS

Cottineau C., Chapron P., Reuillon R., 2015, « Growing Models from the Bottom Up. An Evaluation-Based Incremental Modelling Method (EBIMM) Applied to the Simulation of Systems of Cities », Journal of Artificial Societies and Social Simulation 18 (4) 9. doi:10.18564/jasss.2828

Abstract. This paper presents an incremental method of parsimonious modelling using intensive and quantitative evaluation. It is applied to a research question in urban geography, namely how well a simple and generic model of a system of cities can reproduce the evolution of Soviet urbanisation. We compared the ability of two models with different levels of complexity to satisfy goals at two levels. The macro-goal is to simulate the evolution of the system’s hierarchical structure. The micro-goal is to simulate its micro-dynamics in a realistic way. The evaluation of the models is based on empirical data through a calibration that includes sensitivity analysis using genetic algorithms and distributed computing. We show that a simple model of spatial interactions cannot fully reproduce the observed evolution of Soviet urbanisation from 1959 to 1989. A better fit was achieved when the model’s structure was complexified with two mechanisms. Our evaluation goals were assessed through intensive sensitivity analysis. The complexified model allowed us to simulate the evolution of the Soviet urban hierarchy.

Key-words. ABM, Model-Building, System of Cities, Former Soviet Union, Evaluation, Incremental

New publication, in JASSS : Journal of Artificial Societies and Social Simulation

Reuillon R., Schmitt C., De Aldama R., Mouret J.-B., 2015, « A New Method to Evaluate Simulation Models: The Calibration Profile (CP) Algorithm », JASSS : Journal of Artificial Societies and Social Simulation, Vol. 18, Issue 1, http://jasss.soc.surrey.ac.uk/18/1/12.html

Abstract. Models of social systems generally contain free parameters that cannot be evaluated directly from data. A calibration phase is therefore necessary to assess the capacity of the model to produce the expected dynamics. However, despite the high computational cost of this calibration it doesn’t produce a global picture of the relationship between the parameter space and the behaviour space of the model. The Calibration Profile (CP) algorithm is an innovative method extending the concept of automated calibration processes. It computes a profile that depicts the effect of each single parameter on the model behaviour, independently from the others. A 2-dimensional graph is thus produced exposing the impact of the parameter under study on the capacity of the model to produce expected dynamics. The first part of this paper is devoted to the formal description of the CP algorithm. In the second part,we apply it to an agent based geographical model (SimpopLocal). The analysis of the results brings to light novel insights on the model.

Key-words. Calibration Profile, Model Evaluation