Yiyan Zhai (zhái yì yán)

Yiyan Zhai (zhái yì yán)

she/her

Undergrad researcher @ CMU

Posts by Collection

portfolio

publications

Token Prediction as Implicit Classification to Identify LLM-Generated Text

Published in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023

This paper introduces a novel approach for identifying the possible large language models (LLMs) involved in text generation. Instead of adding an additional classification layer to a base LM, we reframe the classification task as a next-token prediction task and directly fine-tune the base LM to perform it. We utilize the Text-to-Text Transfer Transformer (T5) model as the backbone for our experiments. We compared our approach to the more direct approach of utilizing hidden states for classification. Evaluation shows the exceptional performance of our method in the text classification task, highlighting its simplicity and efficiency. Furthermore, interpretability studies on the features extracted by our model reveal its ability to differentiate distinctive writing styles among various LLMs even in the absence of an explicit classifier. We also collected a dataset named OpenLLMText, containing approximately 340k text samples from human and LLMs, including GPT3.5, PaLM, LLaMA, and GPT2.

Download Paper

talks

teaching

CMU 21-241 Teaching Assistant

Undergraduate level course, Carnegie Mellon University, Department of Mathematical Sciences, 2023

Matrices and Linear Transformations

CMU 10-605 Teaching Assistant

Graduate level course, Carnegie Mellon University, Machine Learning Department, 2024

Machine Learning with Large Datasets