Joshua Holstein

Joshua Holstein

Posts by Collection

publications

Towards Meaningful Anomaly Detection: The Effect of Counterfactual Explanations on the Investigation of Anomalies in Multivariate Time Series

Published in Arxiv, 2023

This study centers on improving anomaly investigation by introducing explainable anomaly detection, demonstrated through a behavioral experiment with New York City taxi data to enhance the accuracy of identifying relevant anomalies.

Recommended citation: Schemmer, M., Holstein, J., Bauer, N., Vössing, M., Kühl, N., Satzger, G. (2023) "Towards Meaningful Anomaly Detection: The Effect of Counterfactual Explanations on the Investigation of Anomalies in Multivariate Time Series." Arxiv. https://arxiv.org/pdf/2302.03302.pdf

On the Perception of Difficulty: Differences between Humans and AI

Published in AutomationXP23: Intervening, Teaming, Delegating - Creating Engaging Automation Experiences, CHI ’23, 2023

This research designs an experiment to understand and compare the perceived difficulty in human-AI interactions, aiming to improve collaboration by accurately evaluating the capabilities of both human and AI agents.

Recommended citation: Spitzer, P., Holstein, J., Vössing, M., Kühl, N. (2023). "On the Perception of Difficulty: Differences between Humans and AI." AutomationXP23: Intervening, Teaming, Delegating - Creating Engaging Automation Experiences, CHI ’23

Sanitizing data for analysis: Designing systems for data understanding

Published in Electronic Markets, 2023

This study introduces a hybrid approach combining expert knowledge and automation for metadata generation in high-dimensional datasets, enhancing data understanding and decision-making, validated by a pharmaceutical company case study.

Recommended citation: Holstein, J., Schemmer, M., Jakubik, J. et al. (2023) "Sanitizing data for analysis: Designing systems for data understanding." Electronic Markets. 33, 52.

Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information

Published in Proceedings of the ACM on Human-Computer Interaction, 2024

This research examines how providing contextual information improves human decisions to delegate tasks to AI, ultimately aiding the design of effective human-AI collaboration systems.

Recommended citation: Spitzer, P., Holstein, J., Hemmer, P., Vössing, M., Kühl, N., Martin, D., Satzger, G. (2024) "On the Effect of Contextual Information on Human Delegation Behavior in Human-AI collaboration." Proceedings of the ACM on Human-Computer Interaction. https://dl.acm.org/doi/10.1145/3710999

Understanding Data Understanding: A Framework to Navigate the Intricacies of Data Analytics

Published in European Conference on Information Systems (ECIS), 2024

This study presents a framework identifying five dimensions of data understanding to guide organizations in leveraging complex datasets for insightful analysis.

Recommended citation: Holstein, J., Spitzer, P., Hoell, M., Vössing, M., Kühl, N. (2024) "Understanding Data Understanding: A Framework to Navigate the Intricacies of Data Analytics." European Conference on Information Systems (ECIS). https://aisel.aisnet.org/ecis2024/track07_busanalytics/track07_busanalytics/5/

A Multi-Actor Benefit Classification for Inter- Organizational Data Sharing in Ecosystems

Published in 46th International Conference on Information Systems (ICIS 2025), 2025

Automatically imported publication.

Recommended citation: Fassnacht, M.; Müller, J.; Benz, C.; Holstein, J.; Satzger, G. (2025) "A Multi-Actor Benefit Classification for Inter- Organizational Data Sharing in Ecosystems". 46th International Conference on Information Systems (ICIS 2025), Association for Information Systems (AIS) https://publikationen.bibliothek.kit.edu/1000185616

Don’t be fooled: The misinformation effect of explanations in human–AI collaboration

Published in International Journal of Human–Computer Interaction, 2025

Automatically imported publication.

Recommended citation: Spitzer, P.; Holstein, J.; Morrison, K.; Holstein, K.; Satzger, G.; Kühl, N. (2025) "Don’t Be Fooled: The Misinformation Effect of Explanations in Human–AI Collaboration". International Journal of Human–Computer Interaction https://publikationen.bibliothek.kit.edu/1000187130

From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Tasks

Published in CHI 2025 Workshop: Tools for Thought: Research and Design for Understanding, 2025

Automatically imported publication.

Recommended citation: Holstein, J.; Diener, M.; Spitzer, P. (2025) "From Consumption to Collaboration: Measuring Interaction Patterns to Augment Human Cognition in Open-Ended Tasks". CHI 2025 Workshop: Tools for Thought: Research and Design for Understanding, Protecting, and Augmenting Human Cognition with Generative AI https://publikationen.bibliothek.kit.edu/1000182350

Hybrid Automation Experiences – Communication, Coordination, and Collaboration within Human-AI Teams

Published in CHI EA ’25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, 2025

Automatically imported publication.

Recommended citation: Spitzer, P.; Baldauf, M.; Palanque, P.; Roto, V.; Morrison, K.; Zipperling, D.; Holstein, J. (2025) "Hybrid Automation Experiences – Communication, Coordination, and Collaboration within Human-AI Teams". CHI EA ’25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems https://publikationen.bibliothek.kit.edu/1000183591

Designing AI Assistants for Novices: Bridging Knowledge Gaps in Onboarding

Published in Hawaii International Conference on System Sciences (HICSS), 2025

This study evaluates an AI assistant designed to support novice onboarding in pharmaceutical manufacturing through metadata filtering, graduated complexity, and sequential query generation.

Recommended citation: Holstein, J., Müller, P., Vössing, M., Fromm, H. (2025) "Designing AI Assistants for Novices: Bridging Knowledge Gaps in Onboarding." Hawaii International Conference on System Sciences (HICSS). https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/280971ed-33d2-4174-a1a3-2aae79930028/content

Don’t be fooled: The misinformation effect of explanations in human–AI collaboration

Published in International Journal of Human–Computer Interaction, 2025

This study reveals that incorrect explanations in AI systems create a misinformation effect, impairing human procedural knowledge and team performance both during and after collaboration.

Recommended citation: Spitzer, P., Holstein, J., Morrison, K., Holstein, K., Satzger, G., Kühl, N. (2025) "Don't be fooled: The misinformation effect of explanations in human–AI collaboration." International Journal of Human–Computer Interaction. https://www.tandfonline.com/doi/pdf/10.1080/10447318.2025.2574511

Data Quality Challenges in Retrieval-Augmented Generation

Published in International Conference on Information Systems (ICIS), 2025

This study develops a dimensional framework for data quality in RAG systems through semi-structured interviews, revealing new dimensions concentrated in early pipeline stages that necessitate dynamic quality management.

Recommended citation: Müller, L., Holstein, J., Bause, S., Satzger, G., Kühl, N. (2025) "Data Quality Challenges in Retrieval-Augmented Generation." International Conference on Information Systems (ICIS). https://aisel.aisnet.org/icis2025/da_bus/da_bus/9/

Development of Mental Models in Human-AI Collaboration: A Conceptual Framework

Published in International Conference on Information Systems (ICIS), 2025

This paper presents an integrated framework identifying three complementary mental models that develop through continuous interaction with AI systems and guide purposeful collaboration design.

Recommended citation: Holstein, J., Satzger, G. (2025) "Development of Mental Models in Human-AI Collaboration: A Conceptual Framework." International Conference on Information Systems (ICIS). https://aisel.aisnet.org/icis2025/hti/hti/2/

A Multi-Actor Benefit Classification for Inter-Organizational Data Sharing in Ecosystems

Published in International Conference on Information Systems (ICIS), 2025

This study develops a classification framework of 22 direct and indirect benefits for understanding value creation across multiple actors in inter-organizational data sharing ecosystems.

Recommended citation: Fassnacht, M., Müller, J., Benz, C., Holstein, J., Satzger, G. (2025) "A Multi-Actor Benefit Classification for Inter-Organizational Data Sharing in Ecosystems." ICIS 2025 Proceedings. 2. https://aisel.aisnet.org/icis2025/digitstrategy/digitstrategy/2/

Balancing the Unknown: Exploring Human Reliance on AI Advice under Aleatoric and Epistemic Uncertainty

Published in ACM Transactions on Computer-Human Interaction, 2025

This work presents two behavioral experiments examining how different types of uncertainty (aleatoric and epistemic) affect human reliance on AI advice in decision-making.

Recommended citation: Holstein, J., Böcking, L., Spitzer, P., Kühl, N., Vössing, M., Satzger, G. (2025) "Balancing the Unknown: Exploring Human Reliance on AI Advice under Aleatoric and Epistemic Uncertainty." ACM Transactions on Computer-Human Interaction. 32(6): 1-34. https://dl.acm.org/doi/full/10.1145/3762813

Data Understanding for Data-Centric AI – Framework Development and Review of Current Methods

Published in Business & Information Systems Engineering. doi:10.1007/s12599-026-00987-1, 2026

Automatically imported publication.

Recommended citation: Holstein, J.; Spitzer, P.; Gensch, S.; Hoell, M.; Vössing, M.; Kühl, N. (2026) "Data Understanding for Data-Centric AI – Framework Development and Review of Current Methods". Business & Information Systems Engineering. doi:10.1007/s12599-026-00987-1 https://publikationen.bibliothek.kit.edu/1000190373

When Thinking Pays Off: Incentive Alignment for Human-AI Collaboration

Published in FAccT 2026, 2026

This study proposes an alternative incentive mechanism to address systematic overreliance in human-AI collaboration, demonstrating through behavioral experiments that appropriately designed incentives can enhance collaboration quality.

Recommended citation: Holstein, J., Hemmer, P., Satzger, G., Sun, W. (2026) "When Thinking Pays Off: Incentive Alignment for Human-AI Collaboration." FAccT 2026. https://arxiv.org/pdf/2511.09612

talks