On the initiative of pope Francis, the catholic church organised the largest consultation ever. Martijn de Jong, Professor in Marketing Research at Erasmus School of Economics, was a contributor to the consultation in the Diocese of Den Bosch and expresses his dissatisfaction in the way data was collected, analysed and reported in an article in Trouw (19 August 2023).
Underrepresentation of specific groups
On a worldwide basis, the consultation failed in a successful representation of all catholic groups, according to de Jong. For instance, Dutch youth have been heard too little, conservative groups often refused to participate and Catholics that do not visit church often were significantly underrepresented.
The Professor remarks that this underrepresentation might be due to the Vatican’s preferred consultation method, namely the ‘spiritual conversation’ in which people gather in groups to discuss matters of faith. This led to the input being largely dependent on discussion groups from parishes. This can also cause other issues. De Jong notes that dominant participants of discussion groups might drown out the more introvert voices and thereby indirectly decide the topics discussed. Additionally, large gatherings may lead to situations in which participants may have too little time to express their views.
The Vatican should refine the procedures of the group gatherings and use alternative methods, according to the Professor. He gives examples such as personal interviews and closed question surveys so that also the introvert voices will be heard. This also gives more anonymity for sensitive topics.
Analysis of data
Due to the consultation's worldwide character, all the open feedback was summarized and integrated. De Jong notes that this process of summarizing and integrating the data does not increase the objectivity of the data. He would have preferred to see a more scientific method used for this process, such as content encoding or algorithms for natural language processing.
Reporting of the data
The Professor explains that every published diocese report looked different, both visually as well as in quality. Information was missing on the background information of participants, questions asked and the manner in which data was analysed. De Jong remarks that detailed reports can yield a more faithful picture on the diversity of voices heard.
De Jong concludes by stressing that consultations are a big leap forward. However, by refining the methodologies used, every voice in the catholic church may be heard and understood better.