Robin Jia

Profile picture

Email: robinjia at usc dot edu

News: Starting in the Fall of 2021, I will be an assistant professor in the Department of Computer Science at the University of Southern California!

I am currently a visiting researcher at Facebook AI Research, working with Luke Zettlemoyer and Douwe Kiela. I recently graduated with a Ph.D. in Computer Science from Stanford University, where I was advised by Percy Liang. I am interested broadly in natural language processing and machine learning, with a particular focus on building NLP systems that are robust to distribution shift at test time.

Publications

The statistical advantage of automatic NLG metrics at the system level.
Johnny Tian-Zheng Wei and Robin Jia.
Association for Computational Linguistics (ACL), 2021.
(github)

Evaluation Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?
Pedro Rodriguez, Joe Barrow, Alexander Hoyle, John P. Lalor, Robin Jia, and Jordan Boyd-Graber.
Association for Computational Linguistics (ACL), 2021.
(website)

Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA.
Ana Valeria Gonzalez, Gagan Bansal, Angela Fan, Robin Jia, Yashar Mehdad, and Srinivasan Iyer.
Findings of ACL, 2021.

Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality.
Mina Lee*, Chris Donahue*, Robin Jia, Alexander Iyabor, and Percy Liang.
North American Association for Computational Linguistics (NAACL), 2021.
(github) (codalab)

Dynabench: Rethinking Benchmarking in NLP
Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams.
North American Association for Computational Linguistics (NAACL), 2021.
(website)

N-ary relation prediction over text spans.
Hoifung Poon, Cliff Wong, and Robin Jia.
US Patent, 2021.

On the Importance of Adaptive Data Collection for Extremely Imbalanced Pairwise Tasks.
Stephen Mussmann*, Robin Jia*, and Percy Liang.
Findings of EMNLP, 2020.
(bib) (codalab) (github)

With Little Power Comes Great Responsibility.
Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, and Dan Jurafsky.
Empirical Methods in Natural Language Processing (EMNLP), 2020.
(bib) (github)

Building Robust Natural Language Processing Systems.
Robin Jia.
Ph.D. Dissertation, 2020.

Selective Question Answering under Domain Shift.
Amita Kamath, Robin Jia, and Percy Liang.
Association for Computational Linguistics (ACL), 2020.
(bib) (codalab)

Robust Encodings: A Framework for Combating Adversarial Typos.
Erik Jones, Robin Jia*, Aditi Raghunathan*, and Percy Liang.
Association for Computational Linguistics (ACL), 2020.
(bib) (codalab) (github)

Certified Robustness to Adversarial Word Substitutions.
Robin Jia, Aditi Raghunathan, Kerem Göksel, Percy Liang.
Empirical Methods in Natural Language Processing (EMNLP), 2019.
(bib) (codalab) (github)

Document-Level N-ary Relation Extraction with Multiscale Representation Learning.
Robin Jia, Cliff Wong, and Hoifung Poon.
North American Association for Computational Linguistics (NAACL), 2019.
(bib) (code and data)

Know What You Don’t Know: Unanswerable Questions for SQuAD.
Pranav Rajpurkar*, Robin Jia*, and Percy Liang.
Best Short Paper Award.
Association for Computational Linguistics (ACL), 2018.
(bib) (website) (codalab) (pptx slides) (pdf slides)

Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer.
Juncen Li, Robin Jia, He He, and Percy Liang.
North American Association for Computational Linguistics (NAACL), 2018.
(bib) (codalab) (pptx slides) (pdf slides)

Adversarial Examples for Evaluating Reading Comprehension Systems.
Robin Jia and Percy Liang.
Empirical Methods in Natural Language Processing (EMNLP), 2017.
Outstanding Paper Award.
(bib) (codalab) (pptx slides) (pdf slides)

Learning Concepts through Conversations in Spoken Dialogue Systems.
Robin Jia, Larry Heck, Dilek Hakkani-Tür, and Georgi Nikolov.
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2017.
(bib) (data)

Data Recombination for Neural Semantic Parsing.
Robin Jia and Percy Liang.
Association for Computational Linguistics (ACL), 2016.
(bib) (codalab) (pptx slides) (pdf slides)

“Reverse genomics” predicts function of human conserved noncoding elements.
Amir Marcovitz, Robin Jia, and Gill Bejerano.
Molecular Biology and Evolution, 2016.

Mx1 and Mx2 key antiviral proteins are surprisingly lost in toothed whales.
Benjamin A. Braun, Amir Marcovitz, J. Gray Camp, Robin Jia, and Gill Bejerano.
Proceedings of the National Academy of Sciences (PNAS), 2015.

* denotes equal contribution

Preprints

Question Answering Infused Pre-training of General-Purpose Contextualized Representations.
Robin Jia, Mike Lewis, and Luke Zettlemoyer.
arXiv, 2021.

Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking.
Zhiyi Ma*, Kawin Ethayarajh*, Tristan Thrush*, Somya Jain, Ledell Wu, Robin Jia, Christopher Potts, Adina Williams, and Douwe Kiela.
arXiv, 2021.

Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation.
Max Bartolo, Tristan Thrush, Robin Jia, Sebastian Riedel, Pontus Stenetorp, and Douwe Kiela.
arXiv, 2021.

Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little.
Koustuv Sinha, Robin Jia, Dieuwke Hupkes, Joelle Pineau, Adina Williams, and Douwe Kiela.
arXiv, 2021.

Can Small and Synthetic Benchmarks Drive Modeling Innovation? A Retrospective Study of Question Answering Modeling Approaches
Nelson F. Liu, Tony Lee, Robin Jia, and Percy Liang.
arXiv, 2021.

To what extent do human explanations of model behavior align with actual model behavior?
Grusha Prasad, Yixin Nie, Mohit Bansal, Robin Jia, Douwe Kiela, and Adina Williams.
arXiv, 2020.

Teaching

Professional Service

Other Work

Industry Internships

Undergraduate Research

Music

I have had the great pleasure of studying piano performance with Angela Wright and Laura Dahl. At various points, I have also studied solo piano with George Barth, duo piano with Kumaran Arul, and chamber music with Stephen Harrison.

Here are some of my recordings:

Piano duo concert, June 2017

Lisa Wang and I gave a piano duo concert on June 4, 2017.

* Probably not actually by Haydn

Piano Quintet Recital, May 2016

Ricky Wedeen, Brad Girardeau, Lee Fan, Andrew Guo, and I gave a concert on May 18, 2016, at which we performed the Schumann Piano Quintet, Op. 44 (mp3).

Senior Recital, April 2014

I gave my undergraduate senior recital on April 12, 2014. Here are the live audio recordings.

Other