Research
Journal Publications
[ Top ]
1. When Automated Assessment Meets Automated Content Generation: Examining Text Quality in the Era of GPTs
Bevilacqua M., Oketch K., Qin R., Stamey W., Zhang X., Gan Y., Yang K., Abbasi A. (2025)
ACM Transactions on Information Systems, 43(2), 2025, pp. 1-36
(The all-time second most downloaded ACM TOIS paper, over 30K downloads)
Categories: Generative AI,Text Analytics
[ Abstract ]
The use of machine learning (ML) models to assess and score textual data has become increasingly pervasive in an array of contexts including natural language processing, information retrieval, search and recommendation, and credibility assessment of online content. A significant disruption at the intersection of ML and text are text-generating large-language models (LLMs) such as generative pre-trained transformers (GPTs). We empirically assess the differences in how ML-based scoring models trained on human content assess the quality of content generated by humans versus GPTs. To do so, we propose an analysis framework that encompasses essay scoring ML models, human- and ML-generated essays, and a statistical model that parsimoniously considers the impact of type of respondent, prompt genre, and the ML model used for assessment model. A rich testbed is utilized that encompasses 18,460 human-generated and GPT-based essays. Results of our benchmark analysis reveal that LLMs and transformer pretrained language models (PLMs) more accurately score human essay quality as compared to CNN/RNN and feature-based ML methods. Interestingly, we find that LLMs and transformer PLMs tend to score GPT-generated text 10–20% higher on average, relative to human-authored documents. Conversely, traditional deep learning and feature-based ML models score human text considerably higher. Further analysis reveals that even though the LLMs and transformer PLMs are exclusively fine-tuned on human text, they more prominently attend to certain tokens appearing only in GPT-generated text, possibly (in part) due to familiarity/overlap in pre-training. Our framework and results have implications for text classification settings where automated scoring of text is likely to be disrupted by generative AI.
Conference Papers
[ Top ]
1. Dash-M5H: An Interactive Dashboard for Multi-Modal, Multi-Model Mental Health Assessment
Alavo R., Zhang X., Ademaj G., Cai J., Kwon H., Cotes R., Clifford G., Abbasi A. (2026)
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (System Demonstrations), San Diego, July 2-7, 2026
Categories: Multimodal AI,Mental Health
[ Abstract ] [ Demo ]
We present Dash-M5H, an interactive dashboard for multi-modal, multi-model mental health assessment that helps clinicians and researchers jointly inspect multimodal behavioral data with multi-model signal outputs of recorded clinical interviews. Guided by signal detection and integrated sensemaking theories, Dash-M5H synchronizes transcript text, audio, and facial behavior (action units and gaze) to support overview-to-detail evidence tracing; and it integrates extracted signals (e.g., sentiment and facial activity) with a clinically grounded VLM prediction pipeline that produces DSM-5-aligned depression predictions. Dash-M5H is implemented in a lightweight, browser-based stack (Quarto + Observable JS + D3), supports local data import and time-synced clinical annotation with export. We demonstrate Dash-M5H through a depression screening scenario, evaluate its note-taking and screening capabilities through a user experiment, and release a live demo and code to facilitate reproducible evaluation.
Under Review
[ Top ]
1. Designing for Digital Traces: Towards Holistically Contextualized Tensor-based Representations
Zhang X., Cai J., Li J., Abbasi A.
Preparing for third round review at INFORMS Information Systems Research
Categories: Digital Trace Data,Tensor Decomposition
[ Abstract ]
The increasing availability of digital trace data presents opportunities for representational richness in modeling user behavioral patterns with downstream implications across an array of socio-technical contexts. However, the multidimensionality, longitudinality, user heterogeneity, and sparsity of digital trace data necessitates computational design artifacts capable of deriving holistically contextualized representations that disentangle meaningful digital patterns from digital exhaust. In this paper, we propose a Bayesian tensor-based framework that uses a three-dimensional tensor to represent the interplay between heterogeneous user groups, platform/sensor-based channels, and time. As part of our framework, we introduce a novel contextual empirical Bayes tensor decomposition method to estimate the usage intensity across channels and time as well as their interactions. Collectively, our framework and method capture user behaviors via low-dimensional latent factors using a family of covariate-assisted sparsity-induced priors, thereby extracting behavioral patterns with enhanced explanatory and predictive power. We evaluate our framework through simulations, two case studies, a user study, and an expert study. The simulations show that our algorithm outperforms competing methods with less estimation bias and better explanatory and predictive performance in synthetic settings. We further demonstrate the downstream value of our framework for explanation and prediction through two real world case studies, each involving millions of raw data points: understanding online customer journeys with omnichannel clickstreams, and examining patient behavior in a telehealth setting with multimodal data for remote mental health assessment. Moreover, the user study demonstrates our framework yields theory-aligned representations, and the expert user study involving healthcare professionals provides evidence of the practical and societal value for decision-making through more relevant, aligned, and useful digital trace patterns.
Working Papers
[ Top ]
1. Sensemaking in Multimodal, Multi-model Environments: Designing Support for Remote Mental Health Assessments
Ademaj G., Zhang X., Abbasi A., Sarker S., Sarker S.
Preparing for submission to Management Science, August 2026
Categories:
[ Abstract ]
Remote mental health assessments routinely generate rich multimodal data in the form of video, audio, and text. This creates new opportunities to support psychiatrists who are overloaded with patients, face increasing documentation demands, and risk missing subtle behaviors or emotional cues. These multimodal data are processed by different machine learning models, creating the need to support sensemaking in complex, multimodal and multi-model environments. This research presents a theory-driven approach to designing sensemaking tools for remote mental health assessments. Drawing on Integrative Sensemaking Theory and Signal Detection Theory, the study derives a set of design requirements and instantiates them in a dashboard prototype to support sensemaking during mental health assessments. The evaluation employs a set of design validity measures to assess the artifact's support of sensemaking, tracing clinician attention patterns, integrative sensemaking framings, and diagnostic outcomes.
2. CoSparTA: An R Package for Covariate-Aware Sparsity-Adaptive Tensor Analysis
Zhang X., Cai J., Li J., Abbasi A.
Preparing for submission to Journal of Machine Learning Research, August 2026
Categories: Tensor Decomposition
[ Abstract ] [ Code ]
Tensors are generalized forms of matrices that extend to higher dimensions. Count-based tensors arise naturally in many application domains where data record event counts across multiple modes. Effective decomposition of such count tensors demands methods that can handle structural sparsity, incorporate observation-level heterogeneity, and support downstream statistical and practical inference. There remains a need for software implementations of algorithms that effectively combine these capabilities. We present CoSparTA, an open-source R package implementation of a context-aware sparsity-adaptive empirical Bayes tensor decomposition algorithm tailored for sparse count tensors. CoSparTA imposes sparsity-adaptive spike-and-slab priors that allow each latent factor to independently adapt to its own sparsity level, and incorporates observation-level covariates directly into the decomposition process rather than as a post-hoc adjustment. The package supports posterior uncertainty quantification, covariate coefficient inference, missing data handling, and visualization through a unified pipeline. Simulation experiments demonstrate consistently improved pattern and tensor recovery over competing methods, with the advantage most pronounced under extreme sparsity and high rank, and reliable recovery of observation-level covariate effects. A real-data demonstration on a large-scale clickstream dataset illustrates the pipeline in practice. CoSparTA is available at https://github.com/nd-hal/CoSparTA
3. Behavioral Grounded Digital Twins in Large-Scale E-commerce
Zhang X., Cai J., Yoo Y., Abbasi A.
Model development.
Categories:
[ Abstract ]
4. Volunteer Behavior in Online Communities
Zhang X., Yoo E., Pedraza-Martinez A.
Model development.
Categories:
[ Abstract ]
5. Theory-guided Multi-Source In-Context Learning Framework for LLM-based Health Assessments
Zhang X., Cai J., Mousavi R., Kitchens B., Abbasi A.
Model development.
Categories:
[ Abstract ]