Archives par mot-clé : Validation process

Beyond Corroboration: Strengthening Model Validation by Looking for Unexpected Patterns

Publié le par

diversityNew Paper, in PLoS ONE

Chérel G., Cottineau C., Reuillon R., 2015, « Beyond Corroboration: Strengthening Model Validation by Looking for Unexpected Patterns », PLoS ONE 10(9): e0138212. doi:10.1371/journal.pone.0138212

Abstract. Models of emergent phenomena are designed to provide an explanation to global-scale phenomena from local-scale processes. Model validation is commonly done by verifying that the model is able to reproduce the patterns to be explained. We argue that robust validation must not only be based on corroboration, but also on attempting to falsify the model, i.e. making sure that the model behaves soundly for any reasonable input and parameter values. We propose an open-ended evolutionary method based on Novelty Search to look for the diverse patterns a model can produce. The Pattern Space Exploration method was tested on a model of collective motion and compared to three common a priori sampling experiment designs. The method successfully discovered all known qualitatively different kinds of collective motion, and performed much better than the a priori sampling methods. The method was then applied to a case study of city system dynamics to explore the model’s predicted values of city hierarchisation and population growth. This case study showed that the method can provide insights on potential predictive scenarios as well as falsifiers of the model when the simulated dynamics are highly unrealistic.

Key-words. Simulation and Modelling, Evolutionary Algorithm, Population Growth, Space Exploration, Urbanisation, Complex Systems.

A New Method to Evaluate Simulation Models: The Calibration Profile (CP) Algorithm

Publié le par

New publication, in JASSS : Journal of Artificial Societies and Social Simulation

Reuillon R., Schmitt C., De Aldama R., Mouret J.-B., 2015, « A New Method to Evaluate Simulation Models: The Calibration Profile (CP) Algorithm », JASSS : Journal of Artificial Societies and Social Simulation, Vol. 18, Issue 1, http://jasss.soc.surrey.ac.uk/18/1/12.html

Abstract. Models of social systems generally contain free parameters that cannot be evaluated directly from data. A calibration phase is therefore necessary to assess the capacity of the model to produce the expected dynamics. However, despite the high computational cost of this calibration it doesn’t produce a global picture of the relationship between the parameter space and the behaviour space of the model. The Calibration Profile (CP) algorithm is an innovative method extending the concept of automated calibration processes. It computes a profile that depicts the effect of each single parameter on the model behaviour, independently from the others. A 2-dimensional graph is thus produced exposing the impact of the parameter under study on the capacity of the model to produce expected dynamics. The first part of this paper is devoted to the formal description of the CP algorithm. In the second part,we apply it to an agent based geographical model (SimpopLocal). The analysis of the results brings to light novel insights on the model.

Key-words. Calibration Profile, Model Evaluation

Multi-agent modeling of urban growth distribution in systems of cities

Publié le par

A strong regularity in urban systems has long been identified : the hierarchical distribution of city sizes. Moreover, a closer observation of the evolution of this distribution shows that in the majority of city systems, there is a trend towards a more and more unequal distribution of city sizes. Why does the majority of urban systems show those strong regularities? What are the common growth processes involved? Several dynamic growth models have been proposed but no consensus has yet been reached because of the under-determination of models by those empirical laws. In this presentation we describe a new method of agent-based parsimonious modeling that we think can contribute to the identification of the common urban growth processes. This modeling method is based  on  intensive model exploration for quantitative evaluation of implemented mechanisms. The exploration tools were first developed for the evaluation of SimpopLocal, a model of the organization of urban systems when cities first emerged. The use of those exploration tools was then generalized into a modeling method that was applied for the first time with the construction of the MARIUS family of models which aims at reproducing the evolution of Soviet urbanisation between 1959 and 1989. Those two examples show how this new modeling method can help the construction of urban theories by helping the evaluation of assumptions made on urban processes.

[gview file= »http://geodivercity.parisgeo.cnrs.fr/blog/wp-content/uploads/2015/01/prez-QuantUrb.pdf »]

Communication at the seminar Quanturb, ISC-PIF (Paris), November 19th.

Clara Schmitt and Paul Chapron

GeoDiverCity at AAG Annual Meeting 2014 in Tampa, Florida

Publié le par

Various contributions of GeoDiverCity team are scheduled for the Annual Meeting of the Association of American Geographers in Tampa, Florida :

Tuesday, 4/8/2014

> Session : 1654 European Research Council – Top European grants for brilliant minds from across the world, from 4:40 PM – 6:20 PM in Grand Salon C, Marriott, Second Floor. Organizer : Katja Meinke.

17:15-17:30    Denise Pumain, « ERC from an Advanced Grantee’s perspective. »

Wednesday, 4/9/2014

> Session : 2268 Urban systems and scaling laws: Functional diversity and urban economic trajectories, from 10:00 AM – 11:40 AM in Meeting Room 1, Marriott, Second Floor. Organizer : Céline Vacchiani-Marcuzzo.

10:00-10:20    Elfie Swerts, Céline Vacchiani-Marcuzzo, Fabien Paulus, « Scaling laws as a tool for characterising the functional evolution in urban systems »

10:20-10:40    Olivier Finance, « Transnational firms in the French system of cities and scaling laws »

> Session : 2239 Geosimulation Models 1: Methodological Advances, from 10:00 AM – 11:40 AM in Room 39, TCC, Fourth Floor. Organizers : Paul Torrens, Suzana Dragicevic, Andrew Crooks.

11:20-11:40    Mathieu Leclaire, Romain Reuillon, « Simpuzzle/Janet tools or how to build a step by step modular ABM ? »

> Session : 2539 Geosimulation Models 3 : Applications – Macro, from 2:40 PM – 4:20 PM in Room 39, TCC, Fourth Floor. Organizers : Paul Torrens, Suzana Dragicevic, Andrew Crooks.

14:00-14:20    Clémentine Cottineau, Paul Chapron, « Evaluation & Calibration for the comparison of ABMs of cities’ trajectories »

16:00-16:20  Denise Pumain, Clara Schmitt, Sébastien Rey-Coyrehourcq, Romain Reuillon, « Building and exploring an agent-based model with OpenMOLE »

Complex Systems and Geography

Publié le par

GeoDiverCity Team was active at presenting their work to a diverse and large audience at the inauguration of the Complex Systems Institute in Paris.

Here are some documents we presented :

> A selection of visual results in geographical modeling

[gview file= »http://geodivercity.parisgeo.cnrs.fr/blog/wp-content/uploads/2014/03/GeoModelingok1.pdf »]

Paul Chapron, Clémentine Cottineau, Robin Cura, César Ducruet, Julie Fen-Chong, Sébastien Haule, Marion Le Texier, Nora Marei and Clara Schmitt

Picture Refs : 1. C. Cottineau, 2. D. Holten & J. W. Jarke, 3. R. Cura, 4.5. C. Ducruet, 6. M. Le Texier, 7. C. Cottineau, 8. P. Chapron, 9.10. C. Schmitt

> Netlogo models of systems of cities

  • SimpopNet

Clara Schmitt

 

  • MARIUS : Modeling of Agglomerations of Imperial Russia and the Soviet Union

Clémentine Cottineau and Paul Chapron

 

> A poster about a method to conceive, build and evaluate models at the intersection of generic systems of cities and the specific case of post-Soviet cities

Clémentine Cottineau, Paul Chapron, Denise Pumain and Romain Reuillon

 

> Accessing the European Grid computing power

Mathieu Leclaire and Romain Reuillon

 

GeoDiverCity open simulation platform

Publié le par

OpenMOLE has originally been developed in a generic way (and in particular with the cooperation of geographer modelers) to be scientific field independent. That is why dealing with geographical models within the Geodivercity program is straightforward. More specific features for geography will be added during the ERC depending on the needs of the modelers.

A first attempt is to build the Simprocess platform for the multi-level exploration of agent-based models of the Simpop type (PhD by Sébastien Rey-Coyrehourcq).

The next versions will include:

  •   new environments (remote servers through ssh, PBS clusters, cloud),
  •   a standardised serialisation format for workflows,
  •   an integration of cutting edge scientific method for model exploration (optimization, calibration, fitness landscape analysis, sensitivity analysis…),
  •   live visualization based on generic tools,
  •   many other good things!

http://www.openmole.org/

Romain Reuillon, Mathieu Leclaire, Sébastien Rey-Coyrehourcq, Hélène Mathian, Clara Schmitt and Arnaud Banos