Editorial: Taking care of the individual: well-being in the age of algorithms
本文是2024年国际信息系统会议专题讨论的总结，四位专家探讨了AI对个体福祉的影响，包括正负效应、非用户影响及历史教训，呼吁研究者和开发者关注用户中心设计。
The rapid advancement of artificial intelligence (AI) technologies is significantly impacting individuals' lives. AI systems, from recommendation algorithms and virtual assistants to autonomous vehicles, large language models and AI agents, are reshaping the way people think, live, work and interact with the world (Liang et al., 2021). The emergence of generative AI, such as ChatGPT and other similar tools, has raised new opportunities and challenges for individuals. These technologies impact how individuals create, consume and interact with information (Henkenjohann and Trenz, 2024). They offer novel ways to not only increase creativity and productivity but also raise important ethical and societal concerns (Rahwan et al., 2019) and can impact the well-being of individuals.AI has swiftly become an integral part of information systems (IS) research, advancing our understanding of its consequences for individuals, organizations and society. This workshop focuses on the interactions between AI and the individual. With technologies' positive as well as negative impacts on individual well-being, designing IS for well-being has become increasingly important (Spiekermann et al., 2022). Therefore, this workshop aims to examine the AI world from the individual's perspective (Matt et al., 2019). It consolidates diverse perspectives on AI's influence on the individual and invites interested researchers to examine how AI shapes personal and social experiences.Research in this area is crucial for understanding the implications, opportunities and risks associated with AI-driven interactions. It can help developers create more user-centric AI technologies and services that enhance individual experiences and well-being. Additionally, it can inform companies about implementing AI tools in ways that align with user values, are ethical and promote overall well-being. To summarize, focusing on AI's influence on the individual, this workshop seeks to foster a comprehensive understanding of human interactions with AI and its consequences.To explore and address this timely and impactful topic, we organized a panel discussion with four experts. Our objective was to discuss and evolve viewpoints on AI in individuals' lives. This discussion was held as part of the Digitization of the Individual (DOTI) workshop titled “The Digitized Individual – Artificial Intelligence and the Individual” in Bangkok, Thailand, which was held in conjunction with the International Conference on Information Systems in 2024. The four panelists in alphabetical order included Alexander Benlian, Darmstadt University of Technology, Germany; Kevin Bauer, Goethe University Frankfurt, Germany; Sirkka Jarvenpaa, The University of Texas at Austin, USA, and Wai Fong Boh, Nanyang Technological University, Singapore. The panel was moderated by Hamed Qahri-Saremi from Colorado State University and Ofir Turel from the University of Melbourne.The panelists' research experiences with AI in individual lives were diverse, reflecting different motivations and areas of focus. For example, Wai Fong is currently examining how we can apply traditional team theories to understand human-AI teams and how we can use GenAI in ways that preserve individuals' critical thinking. Kevin is currently conducting extensive research on human-AI interaction, with a particular focus on the well-being of the parties affected. Sirkka views herself as both a behavioral and organizational researcher, particularly interested in emerging technology topics. Alexander's research also centers on human-AI collaboration and well-being. Their detailed conversations are summarized here:Wai Fong: We now see that AI is very ubiquitous, and it's affecting all of us. When we think about well-being from an individual perspective, we can categorize its impact based on our different roles. First of all, as consumers interacting with the technology, clearly there are positive and negative impacts. On the positive side, it's helping us to improve learning, creativity and productivity. The negative side is also obvious. As a parent, I worry about how it is impacting my daughter in terms of her well-being, the cybersecurity risks and the impact on her mental health, growth and development. We should also probably be thinking about our role as employees. There's already some work about the impact of algorithms on individual employees and how AI can be used to control and optimize their time. For example, algorithms can have serious psychological impacts for gig workers, on top of their positive impacts.The other thing we should be thinking about is the impact on the non-users. We should be thinking about the idea of the digital divide. Non-users – those who do not have access to the technologies – are impacted by their inability to use them. On the flip side, we should think about designing some of these algorithms; we all know about the biases and the possible negative impacts. Another aspect that is often neglected is the potential negative consequences if the AI itself is not designed well and ethically and does not consider human centricity. We also need to think about the implicit implications for individuals who might not be users per se, but they are impacted because of other people's use of these algorithms. These could be individuals who are not selected for jobs because of the way that the AI algorithms are designed in a biased manner or people who might be disconnected because of the technology that is designed in ways that exclude them. In addition to the direct impact on the users, we should also be considering the non-users, those people who did not choose to use the technology but are inadvertently affected by these technologies.Kevin: AI is a general-purpose technology, and we basically want it to improve our lives, making processes more efficient. When humans and AI collaborate efficiently and well, a key factor is that the individual who engages with the AI actually feels good about it and has some well-being improved by using AI; otherwise, there is hardly a purpose for that, right? If the mental health of a person who collaborates with AI is dramatically impaired, then is there actually a benefit of AI? I would seriously question that. From this perspective, there might be a very interesting relationship between people's well-being and the capability of realizing and leveraging complementarities, which can help to move societies forward and to make processes better and more efficient and, ultimately, change the way people use them. It's really in our best interest to understand how this technology affects people's well-being physically and mentally, as well as from the perspective of our capabilities to continue to learn. If we rely too much on these technologies and if we have a technology at hand that does a lot of the cognitive and effortful work for us, do we risk losing our ability to think critically and just take everything for granted? From an aggregate-level perspective, if it harms individuals, then the entire society may converge to a new equilibrium. It should always be this bottom-up approach where we look at the individual or how people work with this technology, how they leverage it, how they feel about it and whether it makes individual lives better. Thereby, we can make sure that we don't lose some of the people along the way and exacerbate the huge divide and biases that already exist in our society. Therefore, we should pay more attention to how people's well-being is affected on an individual level when they interact with this technology. We should also understand how well-being affects the way that this technology can help our societies.Sirkka: All right, we've talked about individuals and society, so I will focus on the meso level – the organizations. When I first saw the question, “Why should we care about well-being in the age of algorithms?”, my initial thought was, “Haven't we always cared? What's really different today?” That led me to reflect on my research, particularly on organizations dedicated to protecting those who are silenced. This includes political prisoners but also juveniles who are sentenced as adults, which is quite common in the USA. I would say that digitization, such as predictive algorithms, has a dramatic impact on longstanding organizations – such as Amnesty, Red Cross and Greenpeace – whose mission has been centered on the well-being of citizens, particularly vulnerable citizens. What are we seeing today? Individual well-being cannot be disconnected from these critical institutions that society needs. How are we supporting? How can technology further strengthen those critical organizations in a society?Alexander: In my view, the need to care about individual well-being when interacting with algorithmic systems stems from the very nature of these algorithms. We know they are far from neutral, often designed to serve commercial or power-driven interests rather than individual flourishing. As IS researchers, we have a responsibility to scrutinize how such systems are embedded in everyday life and how they affect people on a personal level. We've already seen that algorithms can deceive, exclude or trap individuals in filter bubbles. These developments are no longer marginal – they are penetrating core areas of human intelligence and cognition. This represents a qualitative shift and demands that we focus even more on the implications for well-being. Encouragingly, this also opens up rich opportunities for DOTI research to explore new conceptual and empirical dimensions of well-being in the age of AI.Kevin: I just wanted to add to the discussion about individuals and the potential problems that they may have with this technology. Looking at history, when we do not care about individual well-being – especially in the first waves of industrialization – it has led to huge political and economic crises and upswings. There is a historical interpretation that communism, Nazism and other political crises originate from large cultural and societal shifts, often driven by technological advancements. People needed to find new vacancies as their old jobs were just replaced by technology. Many people lost their jobs, and that led to huge societal problems because those who introduced technology did not look at the individual. They were ignored; they were left out. A big problem with AI is that we should focus more on the fact that, in the past, technological developments took a long time and gradually adapted to new technologies. But now, all of these developments are so fast. Three years ago, no one talked about ChatGPT, and now everyone is talking about it. It is a technology that is everywhere. In this rapid development, if we forget to think about individuals and leave them behind, they have even less time to adapt than before, which could lead to even greater societal crises. People need to find their places. They want to feel well, and they want to have good well-being. They want to have a stance in life. Therefore, I think the focus on the individual is not only crucial at the personal level but also a core issue we need to face from a higher societal perspective.Wai Fong: I'd like to approach this question by drawing on what we've learned from past research and theoretical perspectives that can help us study this space. One important lens is the socio-technical perspective – the idea that we must consider both social and technical elements together. Traditionally, IS researchers have focused on either the adoption or the development of technologies. But with AI, we need to integrate these two, because the technology evolves as it's used – it is trained on user data and constantly updated. So, rather than treating use and development as separate, we need to study their interplay. Another key point is the shifting roles in this ecosystem. We have developers and users, of course, but now the technology itself is taking on a more active, evolving role. Some scholars even describe AI systems as having agency – capable of learning and developing independently. That opens up new questions about how these three agents – users, developers and the AI – interact over time. Finally, there's the ecosystem perspective. AI doesn't operate in a vacuum. Like other technologies, it exists within broader social, regulatory and industrial contexts. But with AI, rules and standards are still evolving, and they influence – and are influenced by – the technology's development. So, regulators and industry actors must be considered part of this dynamic ecosystem.All of these perspectives – the socio-technical, the evolving roles, and the ecosystem view – offer opportunities for IS researchers to apply existing theories in new, nuanced ways to study AI and its implications for emotional, physical and psychological well-being.Sirkka: My view is that there is no individual without the collective. This shift is less about new knowledge and more about the transformation of identity – how we think about individuals and how we assign rights and responsibilities that were once tied solely to individuals but now potentially to other entities. This raises questions about the changing societal and organizational norms. What really strikes me is the global crisis of loneliness, especially among young people. This is happening at a time when we have incredibly powerful tools to connect in many ways. So where have we gone wrong? How is it that we're experiencing rising loneliness despite increasing connectivity? Metcalfe – you all know Metcalfe's Law – argues that connectivity has had and will continue to have a greater impact than many of the current conversations around AI. So we need to reflect on what connectivity has meant for us and how that insight can help us understand today's transformations – whether we frame them as shifts in identity or in intelligence.Alexander: I'd like to add a point – we've heard a lot about the meso-level and sociological of AI and but we forget the psychological particularly the of cognition. a when using the risk of and on these it's increasingly important to users with AI They need critical thinking and to understand how algorithmic systems This should also be in our As IS researchers, we only focus on AI development and If we're to a socio-technical we also need to address the human side – users with the and capabilities they need to with these technologies one thought on been I think the IS is particularly to address these IS has always focused on the social on the individual as part of the AI a new of in individual We now have tools that from how we use them – no in the past could A for example, doesn't change based on how you use it. But with AI, can how the – not just for but for data and information that can influence for As Alexander there's already particularly with that on these systems could lead to a of When too much data is and the human knowledge and creativity – especially in areas like – could in could the of and impact of users and So, our – how we use these systems as individuals – has on how will and benefit from them. This should not be I think we've already on but the around user well-being has clearly to more such as identity and In the past, the focus was often on less – important but not as as the questions we're with With the of creativity and productivity have more dramatically than As a research is to explore how these shifts impact well-being – not just at a but in mental well-being, and There's a qualitative change in how we're the we're now more questions about the nature of artificial human intelligence and what that for users and their I think GenAI huge opportunities but also it can help level the especially for those with But this with and risks that users need to be using GenAI tools – like ChatGPT, for – should critically reflect on the long ago, there were no or in users to reflect critically on we're seeing like and to This a I an – in the – that this as an of from a crisis of in What is What can we rely really no way around this other than to strengthen individual agency – to users the tools and responsibility to the this I rely on We need critical thinking and especially from a technical we still don't understand how these systems We or So it to the individual – they must be with tools and knowledge to this I with we need new of But agency also has to be from higher – and organizations must users with the tools and technologies that this agency and help the negative impacts of GenAI on our a good question, and I think I have to my on In my view, we've a approach to well-being in IS We've often focused just on positive or negative or life But I think we can a lot from positive For example, includes broader dimensions – such as positive and such could really help us how we well-being in our on I think we need to move or which are common in IS With more data now – such as or – we can and dynamic For example, how does well-being evolve over How do different of well-being within users as they interact with AI? So there's a to both our conceptual and in this the actually in this It's to think of that it. For example, how can you have well-being without from we move up all the way to It's a like around organizations. To well-being to the that to you at a particular in time. It's about having the that you to for Fong: I think this to what I about seeing and use as This can take but one key aspect is to on in the That some level of between users and We also the role of and standards as important who has in AI there's a or – especially because it's to human when are What one may impact This we need to the potential for and create for those in the In other we must influence ethical the development rather than the consequences only All of these to the idea of the social and the technical – which users, developers and the technology itself as These are some of the perspectives we should in when thinking about ethically AI now, there's a between the of technological development – especially in AI – and the ability of to create the AI for the as as and just as it was GenAI crisis and of to for these new without understanding the consequences of We've seen this with It a lot of – organizations sure what data they could or how to to a of regulatory now seeing similar around GenAI IS researchers, we're to understand the between technology, individuals and organizational and to consequences of In one of my on for example, we that doesn't always lead to better doesn't problem – is not always the best Our role can be to help understand user and broader so they can gradually improve I'd like to as researchers, our role does not at the We also have an to That in and – an active, role in society by our knowledge regulatory This is part of our Fong: just that technological development, we need to think more – especially about the implications for What to people whose become to AI? What do we need to that individuals can and a of identity and We must help think not just the technical impact but also broader societal How do we people whose are How do we help them adapt and find new These are key implications we From a technical perspective, once the data is in the AI then we have a problem of making it forget that From a technical perspective, generative AI models have the ability to and then we have systems that they can data I think we need some models that have of making of the of data and data to be used for the AI whose is similar to the data in that we can users can control the data in the which will be used to A in point is the even that ChatGPT can a by does not that this data was included in the there are just so many of that can be there is a that this is just not the Therefore, once the data are in the the is basically over at the we may have a by traditional with these new AI systems to the of not to data from a The to be that people are interested in their What do we have that individuals are to time their It is also very for individuals to how they want organizations or their models to their not only at but also from both the past and the In in individual data is an research in But as I one cannot address individual and the to be without considering or If I my health what are the opportunities for a or that I part we talked about research and AI Alexander that we need new of But what are we losing as we in these new of AI What critical capabilities are we humans and and as we increase our on digital technologies such as the panelists the of user well-being in AI research and development, particularly as AI technology the of individual cognitive and societal The panelists that well-being must be in drawing from such as and and using like data and They to risks such as on AI systems, cognitive and a digital all of which individuals' and even the discussion the need to consider not only direct users of AI but also and affected whose opportunities and rights may be by biased algorithms and processes of AI panelists also the critical role of researchers in the between technical development and social This includes and regulatory ethical by human in development and AI to strengthen critical with algorithmic It was that historical technological that neglected individual well-being led to societal – a for today's of AI IS scholars are to study and the between users, developers and AI itself as dynamic agents within a broader socio-technical ecosystem. they can help that AI not only but also the purpose and well-being of its users and who are or affected.