Beyond Corroboration: Strengthening Model Validation by Looking for Unexpected Patterns

diversityNew Paper, in PLoS ONE

Chérel G., Cottineau C., Reuillon R., 2015, « Beyond Corroboration: Strengthening Model Validation by Looking for Unexpected Patterns », PLoS ONE 10(9): e0138212. doi:10.1371/journal.pone.0138212

Abstract. Models of emergent phenomena are designed to provide an explanation to global-scale phenomena from local-scale processes. Model validation is commonly done by verifying that the model is able to reproduce the patterns to be explained. We argue that robust validation must not only be based on corroboration, but also on attempting to falsify the model, i.e. making sure that the model behaves soundly for any reasonable input and parameter values. We propose an open-ended evolutionary method based on Novelty Search to look for the diverse patterns a model can produce. The Pattern Space Exploration method was tested on a model of collective motion and compared to three common a priori sampling experiment designs. The method successfully discovered all known qualitatively different kinds of collective motion, and performed much better than the a priori sampling methods. The method was then applied to a case study of city system dynamics to explore the model’s predicted values of city hierarchisation and population growth. This case study showed that the method can provide insights on potential predictive scenarios as well as falsifiers of the model when the simulated dynamics are highly unrealistic.

Key-words. Simulation and Modelling, Evolutionary Algorithm, Population Growth, Space Exploration, Urbanisation, Complex Systems.

Multi-agent modeling of urban growth distribution in systems of cities

A strong regularity in urban systems has long been identified : the hierarchical distribution of city sizes. Moreover, a closer observation of the evolution of this distribution shows that in the majority of city systems, there is a trend towards a more and more unequal distribution of city sizes. Why does the majority of urban systems show those strong regularities? What are the common growth processes involved? Several dynamic growth models have been proposed but no consensus has yet been reached because of the under-determination of models by those empirical laws. In this presentation we describe a new method of agent-based parsimonious modeling that we think can contribute to the identification of the common urban growth processes. This modeling method is based  on  intensive model exploration for quantitative evaluation of implemented mechanisms. The exploration tools were first developed for the evaluation of SimpopLocal, a model of the organization of urban systems when cities first emerged. The use of those exploration tools was then generalized into a modeling method that was applied for the first time with the construction of the MARIUS family of models which aims at reproducing the evolution of Soviet urbanisation between 1959 and 1989. Those two examples show how this new modeling method can help the construction of urban theories by helping the evaluation of assumptions made on urban processes.

Download (PDF, 3.51MB)

Communication at the seminar Quanturb, ISC-PIF (Paris), November 19th.

Clara Schmitt and Paul Chapron

GeoDiverCity at AAG Annual Meeting 2014 in Tampa, Florida

Various contributions of GeoDiverCity team are scheduled for the Annual Meeting of the Association of American Geographers in Tampa, Florida :

Tuesday, 4/8/2014

> Session : 1654 European Research Council – Top European grants for brilliant minds from across the world, from 4:40 PM – 6:20 PM in Grand Salon C, Marriott, Second Floor. Organizer : Katja Meinke.

17:15-17:30    Denise Pumain, “ERC from an Advanced Grantee’s perspective.”

Wednesday, 4/9/2014

> Session : 2268 Urban systems and scaling laws: Functional diversity and urban economic trajectories, from 10:00 AM – 11:40 AM in Meeting Room 1, Marriott, Second Floor. Organizer : Céline Vacchiani-Marcuzzo.

10:00-10:20    Elfie Swerts, Céline Vacchiani-Marcuzzo, Fabien Paulus, “Scaling laws as a tool for characterising the functional evolution in urban systems”

10:20-10:40    Olivier Finance, “Transnational firms in the French system of cities and scaling laws”

> Session : 2239 Geosimulation Models 1: Methodological Advances, from 10:00 AM – 11:40 AM in Room 39, TCC, Fourth Floor. Organizers : Paul Torrens, Suzana Dragicevic, Andrew Crooks.

11:20-11:40    Mathieu Leclaire, Romain Reuillon, “Simpuzzle/Janet tools or how to build a step by step modular ABM ?”

> Session : 2539 Geosimulation Models 3 : Applications – Macro, from 2:40 PM – 4:20 PM in Room 39, TCC, Fourth Floor. Organizers : Paul Torrens, Suzana Dragicevic, Andrew Crooks.

14:00-14:20    Clémentine Cottineau, Paul Chapron, “Evaluation & Calibration for the comparison of ABMs of cities’ trajectories”

16:00-16:20  Denise Pumain, Clara Schmitt, Sébastien Rey-Coyrehourcq, Romain Reuillon, “Building and exploring an agent-based model with OpenMOLE”

From SimpopLocal to SimpopClim: Three stages of urban growth dynamics under resource accessibility constraints

  The aim of this series of models 1 is to study growth regimes of systems of cities, which are defined by the nature of the interaction between the cities. Three different stages of urbanisation are considered as resource accessibility, which plays a major role in growth dynamics, takes different forms. For each stage, a set of stylized facts characterizing the state of the system and urban growth dynamics is proposed and defines the structure of an ABM model.

The SimpopLocal model tries to characterise a stylised dynamics of urban emergence.  This first urban regime is defined by the role of local environmental constraints on growth. Central to this model is the notion of landscape carrying capacity. Settlement sizes and their growth are controlled by the amount of resources locally available.  But innovations and their diffusion in the system thanks to interactions between settlements help to overcome those limitations, eventually producing the proto-structure of urban systems.

The SimpopNet model characterise the progressive networking of urban economies. Innovations in communication and transport networks allow the trade and long distance diffusion of goods and techniques which enables cities to overcome, by importing what was lacking, the local constraints and climatic hazards that limited their growth. In this regime, the resource accessibility of each city is defined by its situation in the network. The SimpopNet model simulates the co-evolution of urban systems and transportation networks.

The SimpopClim model will represent a dynamic regime that takes into account the impact of global environmental constraints on urban dynamics.

This PhD is made possible thanks to interdisciplinary work with computer researchers participating in the GeoDiverCity programme, hired after previous collaboration with ISC-PIF (Romain Reuillon and Mathieu Leclaire, and a PhD in geomatics conducted by Sébastien Rey Coyrehourcq. The SimpopLocal and SimpopNet models are used as case studies for the development of grid exploration procedures and protocols with OpenMole ( Those explorations and validating tools are designed to meet the validation and reproducibility requirements and to be generic and adapted to the exploration of spatial simulation models.

Clara Schmitt


  1. PhD project of Clara Schmitt, under the supervision of Denise Pumain (founded by ADEME, The French Agency for Environment and Energy)