SFB 1187 ›Medien der Kooperation‹ an der Universität Siegen

B06 - Un-/desired Observation in Interaction: Smart Environments, Language, Body, and Senses in Private Households

SFB 1187-Projekt-B06-2024.JPG

Former Researchers:

Dr. Christine Hrncal

Dr. Kathrin Englert
(Associate Member)

Jacqueline Klesse, M.A.
(Associate Member)

Dr. Vincent Knopp
(Associate Member)

Prof. Dr. Wolfgang Ludwig-Mayerhofer
(Associate Member and former Principal Investigator)

Dr. habil. Oliver Schmidtke
(Associate Member)

Tamara Schwertel, M.A.
(Associate Member)

Dr. David Waldecker

 

From smart speaker to smart home: The project investigates the domestication of data-intensive sensory media in interaction. It explores how 'intelligent' living environments digitally capture households in terms of language, motor skills and sensory perception.

 


 

Executive Summary

Saving energy, eating sustainable and healthy food, managing remote work and family reproduction, care and domestic work, concerns about non-transparency in data utilization and digital surveillance, about consumer and audience manipulation: Private households serve as key examples of the daily practice of meeting contemporary challenges. These are places, moreover, where digital media are perceived and negotiated both as adding to existing problems and as resources for overcoming them. The private household traditionally plays a central role in research on the domestication (Silverstone et al. 1992) of media.

Digital, networked media are increasingly finding their way into households. This represents a correspondingly increasing challenge to the domestication perspective. As already addressed in the second funding phase from a media linguistic and sociological perspective, this is exemplified by smart speakers and the highly selective utilization they typify: Although such media are also embedded in social interaction and everyday practice, those who wish to exploit the functional potential of voice-controlled assistance systems must also adapt to technologized dialog structures and platform logics; in the process, they are forced to divulge a lot of information about themselves – information that may take on new significance beyond the household in the form of data in exploitation contexts that are opaque to users and defy attempts to influence them.

Assistance systems potentially only come into their own as central interfaces to their fullest possible extent in smart home environments. At the same time, however, the household thereby opens itself up to the outside world far more than before: Whereas in the case of conventional smart speakers, our observation was limited to the perceptual dimension of hearing or listening, camera and monitor- and sensor-based systems and their networking with a variety of stationary and mobile devices and infrastructures result, in extreme cases, to a massive expansion of registrable data. Under certain circumstances, this trend is accompanied by a further attack on of the boundaries of the private sphere. This may be perceived as a violation on the one hand (e.g. in the case of surveillance technology being used to monitor members of a household), but may also be seen as desirable (e.g. from the perspective of security) on the other. An empirical question that remains is the extent to which households are open to such technologies and what consequences, if any, arise for them or can be exploited by them. It remains to be seen how the shift from potentially acoustic to a multimodal form of surveillance will be perceived, accounted for, and shaped by the users. Following on from technical-methodological innovations explored in the second phase, this phase will continue to investigate human-technology dialogs and their integration into social interaction. However, it will expand this area to include the reconstruction of sensory orientations and bodily practices (Mondada 2021) as well as the consideration of sensor-based mechanisms and their technical interfaces in smart homes.

It can be assumed that sensory environments not only change spatial arrangements and social coexistence on the communicative level, but that the entanglement of senses and technical sensoria also generates new modes of the co-articulation of data and practices (Marres 2015). Against this background, this study asks to what extent and how users (can) evaluate and take responsibility for their situated media and data practices with regard to the discursive imperatives referred to at the beginning.

 

Smart Speaker als zentrale Smart-Home-Steuerung (© Unsplash)
Smart speaker – central smart home control unit
(© Unsplash)

On the one hand, we focus on the multimodal and perceptual dialog, including domestic environments, devices and infrastructures. On the other hand, we investigate the evaluation of digitalized household practices against the background of prevailing crisis discourses (e.g. energy saving). At the same time, we explore the (partially) opaque sensory processes of the socio-technical infrastructures - extended by camera, monitor and sensors. In this sense, the project examines both sides of the sensor and data-practical linking of households and platforms.

We visit the participants in our study over a longer period of time to make audiovisual recordings, photos and notes on various occasions. Media diaries support the recording of media practices in the household.

The analysis combines video interaction analysis with ethnographic procedures and digital methods. The videos are transcribed according to multimodal standards. We use digital methods to examine the software documentations and sensor interfaces of the smart home devices used in the households.

Software zur multimodalen Transkription der Video-Aufzeichnungen und eigene Aufnahme der Studienteilnehmenden
Software for multimodal transcription

In the first half of 2024, we will set up the research design, establish field access and explore the platforms to be investigated. We will then carry out the extensive fieldwork using AV recordings and ethnography, including data collection for the platform analyses (2024–2026). The collected data will be continuously processed, analyzed in data sessions and the procedure will be compared and critically reflected upon in interdisciplinary methodology workshops. In addition to project reports and publications, participation in several public events and conferences is planned for the final year of the project (2027).

 

➔ Find the Project Archive 2020–2023 here

 

Current vacancy

Student Assistant (SHK)

In subproject B06 "Un/erbetene Beobachtung in Interaktion: Smart Environments, Sprache, Körper und Sinne in Privathaushalten" of the DFG Collaborative Research Center 1187 “Media of Cooperation” we are looking for a student assistant (SHK) (m/f/d) as of August 1st, 2024 under the following conditions:

  • Up to 5 hours per week, with the possibility of an increase from 2025
  • Initially limited to one year, with the possibility of extension
  • Employment on the basis of the German Academic Fixed-Term Contract Act

Your tasks:

  • Preparation and transcription of audiovisual data material
  • Assistance with research and investigation (including literature research and management, planning field visits)
  • Support in the organization of events (including the planning and practical implementation of conferences and workshops)
  • Assistance in the preparation of scientific publications (e.g. proofreading, formatting work)

Your profile:

  • Ideally, knowledge in the field of conversation-analytical transcription according to GAT2 and corresponding software (Folker, Exmaralda Audacity or similar) or willingness to familiarize yourself with it
  • Enrollment in a (preferably linguistic) Bachelor’s degree program at a German university
  • Interest in working in an academic environment
  • Confident scope/independent work with MS Office
  • Very good knowledge of English
  • Structured work, enjoy teamwork, initiative and a sense of responsibility

We look forward to receiving your application by 31.05.2024.

You can find more information about the project here https://www.mediacoop.uni-siegen.de/en/projekte/b06/

Please send your application documents (cover letter, curriculum vitae in tabular form, certificates) in a single pdf file to Mr. Hector.

Your contact person:
Tim Moritz Hector, M.A.
tim.hector[æt]uni-siegen.de

full job vacancy

 

Publications

Current

„Taming Digital Practices: On the Domestication of Data-Driven Technologies“

Modern mundane life is brimming with a variety of data-driven technologies that are supposed to augment the practices they are involved in. As humans bring these technologies into their lives in a process of domestication, they tame them and are simultaneously influenced by their presence. The contributions in this issue explore the use of digitally connected media such as vacuum robots, smart speakers, drones, and kitchen appliances with reference to the domestication paradigm from interdisciplinary perspectives including media studies, sociology, anthropology, and human-computer-interaction.

Hector, Tim, David Waldecker, Niklas Strüver, und Tanja Aal, Hrsg. 2023. Thematic issue: „Taming Digital Practices – On the Domestication of Data-Driven Technologies“. Digital Culture & Society 9 (1/2023). ISBN 978-3-8376-6357-0.

 

2024

Hector, Tim Moritz, and Christine Hrncal. 2024. “Sprachassistenzsysteme in der Interaktion”. In Sprache und digitale Kommunikation, edited by Jannis Androutsopoulos and Friedemann Vogel, 309-28. Handbücher Sprachwissen 23. Berlin u.a.: de Gruyter. ISBN: 9783110744101 .

2023

Habscheid, Stephan, Tim Hector, and Christine Hrncal. 2023. “Human and non-human agency as practical accomplishment: Interactional occasions for ascription and withdrawal of (graduated) agency in the use of smart-speaker-technology”. Edited by Samira Ibnelkaïd and Iuliia Avgustis. Social Interaction. Video-Based Studies of Human Sociality. Special Issue: Situated agency in digitally artifacted social interactions 6 (1/2023). https://doi.org/10.7146/si.v6i1.137378.
Hector, Tim Moritz, David Waldecker, Niklas Strüver, and Tanja Aal, eds. 2023. Thematic issue: „Taming Digital Practices – On the Domestication of Data-Driven Technologies“. Digital Culture & Society 9 (1/2023). ISBN: 978-3-8376-6357-0.
Strüver, Niklas. 2023. “Wieso eigentlich Alexa? Konzeptualisierung eines Sprachassistenten als Infrastruktur und Plattform im soziotechnischen Ökosystem Amazons”. kommunikation@gesellschaft 24: 1-33. https://doi.org/10.15460/kommges.2023.24.1.1194 .
Strüver, Niklas. 2023. “Introduction: Forms of Context in Digital Technologies”. In The Routledge Handbook of Media and Technology Domestication, edited by Maren Hartmann, 329-30. London, New York: Routledge, Taylor & Francis Group.
Waldecker, David, and Tim Moritz Hector. 2023. “Taming digital practices: A praxeological approach towards domestication of connected devices and services. Introduction to the thematic issue”. Digital Culture & Society 9 (1/2023). https://doi.org/10.14361/dcs-2023-0102.
Waldecker, David, Tim Moritz Hector, and Dagmar Hoffmann. 2023. “Intelligent Personal Assistants in practice. Situational agencies and the multiple forms of cooperation without consensus”. In Convergence. Special Issue: Agency in a Datafied Society, edited by Stephan O. Görland, Cindy Roitsch, and Andreas Hepp. https://doi.org/10.1177/13548565231189584.
Waldecker, David, and Dagmar Hoffmann. 2023. “Inszenierung von kritischen Kompetenzen in Nischenöffentlichkeiten: Bewertungen von Smart Speakern auf YouTube”. kommunikation@gesellschaft 23 (1): 1-28. https://doi.org/10.15460/kommges.2022.23.1.1000.

2022

Englert, Kathrin, Dagmar Hoffmann, and David Waldecker. 2022. “‘Tut mir leid, ich verstehe nicht ganz.’ Smart Speaker als vermeintliche Gesprächspartner*innen”. merz – Zeitschrift für Medienpädagogik 66 (2): 24-34. https://www.merz-zeitschrift.de/alle-ausgaben/details/2022-02-sprache-und-medien/.
Habscheid, Stephan. 2022. “Socio-technical dialogue and linguistic interaction. Intelligent Personal Assistants (IPA) in the Private Home”. Sprache und Literatur 51 (2): 167-96. https://doi.org/10.30965/25890859-05002020.
Hector, Tim. 2022. “Smart Speaker in der Praxis: Methodologische Überlegungen zur medienlinguistischen Erforschung stationärer Sprachassistenzsysteme”. Sprache und Literatur 51 (2): 197-229. https://doi.org/10.30965/25890859-05002021.
Hector, Tim, Franziska Niersberger-Gueye, Franziska Petri, and Christine Hrncal. 2022. “The ‘Conditional Voice Recorder’: Data practices in the co-operative advancement and implementation of data-collection technology”. Working Paper Series Media of Cooperation 23: 1-15. http://dx.doi.org/10.25819/ubsi/10225 .
Hrncal, Christine, and Tim Hector. 2022. “Tagungsbericht: Sequence Analysis in Linguistics and Social Theory, 29.10.2021 – 30.10.2021, digital”. In H-Soz-Kult. https://www.hsozkult.de/conferencereport/id/fdkn-127824.
Waldecker, David. 2022. “Zur empirischen und theoretischen Kritik der Datensouveränität anhand der Smart-Speaker-Nutzung”. merzWissenschaft 66 (6): 147-57.
Waldecker, David, and Axel Volmar. 2022. “Die zweifache akustische Intelligenz virtueller Sprachassistenten zwischen verteilter Kooperation und Datafizierung”. In Acoustic Intellingence. Hören und Gehorchen, edited by Anna Schürmer, Maximilian Haberer, and Tomy Brautschek, 161-82. Düsseldorf: Düsseldorf University Press. https://doi.org/10.1515/9783110730791-011.

2021

Habscheid, Stephan, Tim Hector, Christine Hrncal, and David Waldecker. 2021. “Intelligente Persönliche Assistenten (IPA) mit Voice User Interfaces (VUI) als ‚Beteiligte‘ in häuslicher Alltags­interaktion. Welchen Aufschluss geben die Protokolldaten der Assistenzsysteme?”. Journal für Medienlinguistik 4: 16-53. https://doi.org/10.21248/jfml.2021.44.

2020

Hector, Tim, and Christine Hrncal. 2020. “Intelligente Persönliche Assistenten im häuslichen Umfeld. Erkenntnisse aus einer linguistischen Pilotstudie zur Erhebung audiovisueller Interaktionsdaten”. Working Paper Series Media of Cooperation 14: 1-14. https://www.mediacoop.uni-siegen.de/publikationen/working-papers-media-of-cooperation/.
Waldecker, David, Oliver Schmidtke, and Kathrin Englert. 2020. “Individuierung, Autonomie und Social Media. Überlegungen zum Strukturwandel von Öffentlichkeit und Privatheit”. Sprache und Literatur 49 (1): 171-99. https://doi.org/10.30965/25890859-04901007.

2019

Englert, Kathrin, David Waldecker, and Oliver Schmidtke. 2019. “Un/erbetene Beobachtung: Bewertung richtigen Medienhandelns in Zeiten seiner Hyper-Beobachtbarkeit”. In Digitale Bewertungspraktiken. Für eine Bewertungssoziologie des Digitalen, edited by Jonathan Kropf and Stephan Laser, 215-36. Wiesbaden: Springer VS. https://doi.org/10.1007/978-3-658-21165-3_9.
Schmidtke, Oliver, Kathrin Englert, and David Waldecker. 2019. “Vom alltäglichen Ziehen fließender Grenzen. Die Veröffentlichung von Intimität bei jugendlichen Social-Media-Nutzer/innen”. In Intimisierung des Öffentlichen. Zur multiplen Privatisierung des Öffentlichen in der digitalen Ära, edited by Patrik Ettinger, Mark Eisenegger, Marlis Prinzing, and Roger Blum, 211-26. Mediensymposium. Wiesbaden: Springer VS. https://doi.org/10.1007/978-3-658-24052-3_12.
Waldecker, David, Kathrin Englert, and Wolfgang Ludwig-Mayerhofer. 2019. “Media Ethnography and Participation in Online Practices”. Media in Action. Interdisciplinary Journal on Cooperative Media 3 (1): 9-22. https://www001.zimt.uni-siegen.de/ojs/index.php/mia/article/view/47.

2018

Kathrin, Englert, Lene Faust, Christian Heinrich-Franke, Claudia Müller, and Cornelius Schubert (Hrsg.). 2018. “Thematic Focus: Socioinformatics”. Media in Action. Interdisciplinary Journal on Cooperative Media, Sonderheft, 2 (1). https://www001.zimt.uni-siegen.de/ojs/index.php/mia/issue/view/9.

2017

Englert, Kathrin, Jacqueline Klesse, Wolfgang Ludwig-Mayerhofer, Oliver Schmidtke, and David Waldecker. 2017. “‚Das Digitale‘ und sein Modus Operandi. Bewertungen (un)erbetener Be(ob)achtung”. In Geschlossene Gesellschaften. Verhandlungen des 38. Kongresses der Deutschen Gesellschaft für Soziologie in Bamberg 2016, edited by Stephan Lessenich. Bamberg: Deutsche Gesellschaft für Soziologie. http://publikationen.soziologie.de/index.php/kongressband_2016/article/view/524.
Reißmann, Wolfgang, and Dagmar Hoffmann. 2017. “Mediatisierung und Mediensozialisation. Überlegungen zum Verhältnis zweier Forschungsfelder”. In Mediatisierung und Mediensozialisation. Prozesse – Räume – Praktiken, edited by Dagmar Hoffmann, Friedrich Krotz, and Wolfgang Reißmann, 59-78. Wiesbaden: Springer VS. https://doi.org/10.1007/978-3-658-14937-6_4.