My Advisor, Her AI and Me: Evidence From a Field Experiment on Human-AI Collaboration and Investment Decisions

Abstract

Amid ongoing policy and managerial debates on keeping humans in the loop of AI decision-making processes, we investigate whether human involvement in AI-based service production benefits downstream consumers. Partnering with a large savings bank in Europe, we produced pure AI and human-AI collaborative investment advice, which we passed to the bank customers and investigated the degree of their advice-taking in a field experiment. On the production side, contrary to concerns that humans might inefficiently override AI output, our findings show that having a human banker in the loop of AI-based financial advisory by giving her the final say over the advice provided does not compromise the quality of the advice. More importantly, on the consumption side, we find that the bank customers are more likely to align their final investment decisions with advice from the human-AI collaboration, compared to pure AI, especially when facing more risky investments. In our setting, this increased reliance on human-AI collaborative advice leads to higher material welfare for consumers. Additional analyses from the field experiment along with an online controlled experiment indicate that the persuasive efficacy of human-AI collaborative advice cannot be attributed to consumers' belief in increased advice quality resulting from complementarities between human and AI capabilities. Instead, the consumption-side benefits of human involvement in the AI-based service largely stem from human involvement serving as a peripheral cue that enhances the affective appeal of the advice. Our findings indicate that regulations and guidelines should adopt a consumer-centric approach by fostering environments where human capabilities and AI systems can synergize effectively to benefit consumers while safeguarding consumer welfare. These nuanced insights are crucial for managers who face decisions about offering pure AI versus human-AI collaborative services and also for regulators advocating for having humans in the loop.