15th Workshop on Innovative Use of NLP for Building Educational Applications


Quick Info
Co-located with ACL 2020
Location Seattle, Washington, USA
Deadline Monday, April 6, 2020 11:59pm EST
Monday, April 13, 2020 11:59pm EST (extended)
Date Friday, July 10, 2020
Organizers Jill Burstein, Ekaterina Kochmar, Claudia Leacock, Nitin Madnani, Ildikó Pilán, Helen Yannakoudakis, and Torsten Zesch
Contact bea.nlp.workshop@gmail.com

Workshop Description

The BEA Workshop is a leading venue for NLP innovation in the context of educational applications. It is one of the largest one-day workshops in the ACL community with over 80 attendees in the past several years. The growing interest in educational applications and a diverse community of researchers involved resulted in the creation of the Special Interest Group in Educational Applications (SIGEDU) in 2017, which currently has 191 members.

The workshop’s continuing growth highlights the alignment between societal needs and technological advances. NLP capabilities can now support an array of learning domains, including writing, speaking, reading, science, and mathematics, as well as the related intra-personal (e.g., self-confidence) and inter-personal (e.g., peer collaboration) skills. Within these areas, the community continues to develop and deploy innovative NLP approaches for use in educational settings. In the writing and speech domains, automated writing evaluation (AWE) and speech scoring applications, respectively, are commercially deployed in high-stakes assessment and in instructional contexts (e.g., Massive Open Online Courses (MOOCs) and K-12 classrooms). Commercially-deployed plagiarism detection is also commonly used in both K-12 and higher education settings. For writing, the focus is on innovations that support writing tasks requiring source use, argumentative discourse, and factual content accuracy. For speech, there is an interest in advancing automated scoring to include the evaluation of discourse and content features in responses to spoken assessments. General advances in speech technology have promoted a renewed interest in spoken dialog and multimodal systems for instruction and assessment, for instance, for workplace interviews and simulated teaching environments. The explosive growth of mobile applications for game-based and simulation-based applications is another area where NLP has begun to play a large role, especially for language learning.

NLP for educational applications has gained visibility outside of the NLP community. First, the Hewlett Foundation reached out to the public and private sectors and sponsored two competitions: one for automated essay scoring, and the other for scoring of short response items. The motivation driving these competitions was to engage the larger scientific community in this enterprise. Learning @ Scale is a relatively new venue for NLP research in education. MOOCs now incorporate AWE systems to manage several thousand assignments that may be received during a single MOOC course. MOOCs for Refugees have more recently emerged in response to the current social situations. Courses include language learning, and we can imagine that AWE and other NLP capabilities could support coursework. Another breakthrough for educational applications within the CL community is the presence of a number of shared-task competitions over the past several years — including four shared tasks on grammatical error detection and correction. NLP/Education shared tasks have seen new areas of research, such as the Automated Evaluation of Scientific Writing at BEA 11, Native Language Identification at BEA 12, Second Language Acquisition Modelling and Complex Word Identification both at BEA 13, and Grammatical Error Correction at BEA 14. These competitions increased the visibility of, and interest in, our field.

The 15th BEA workshop will have virtual presentation and poster sessions in order to maximize the amount of original work presented. We expect that the workshop will continue to highlight novel technologies and opportunities for educational NLP in English as well as other languages. The workshop will solicit both full papers and short papers for either oral or poster presentation. We will solicit papers that incorporate NLP methods, including, but not limited to: automated scoring of open-ended textual and spoken responses; game-based instruction and assessment; educational data mining; intelligent tutoring; peer review; grammatical error detection and correction; learner cognition; spoken dialog; multimodal applications; tools for teachers and test developers; and use of corpora. Specific topics include:

Automated scoring/evaluation for written student responses (across multiple genres)

  • Content analysis for scoring/assessment
  • Detection and correction of grammatical and other types of errors
  • Argumentation, discourse, sentiment, stylistic analysis, & non-literal language

Intelligent Tutoring (IT), Collaborative Learning Environments

  • Educational Data Mining: Collection of user log data from educational applications
  • Game-based learning
  • Multimodal communication (including dialog systems) between students and computers

Learner cognition

  • Assessment of learners’ language and cognitive skill levels
  • Systems that detect and adapt to learners’ cognitive or emotional states
  • Tools for learners with special needs

Use of corpora in educational tools

  • Data mining of learner and other corpora for tool building
  • Annotation standards and schemas / annotator agreement

Tools and applications for classroom teachers and/or test developers

  • NLP tools for second and foreign language learners
  • Semantic-based access to instructional materials to identify appropriate texts
  • Tools that automatically generate test questions

Important Dates

Note: these dates are still preliminary and may change.

  • Submission Deadline: Monday, April 6, 2020, 11:59pm EST Monday, April 13, 2020, 11:59pm EST (extended)
  • Notification of Acceptance: Monday, May 4, 2020 Monday, May 11, 2020
  • Camera-ready Papers Due: Monday, May 18, 2020 Monday, May 25, 2020
  • Workshop: Thursday/Friday, July 10, 2020


  1. Note that BEA 2020 will be fully virtual just like ACL 2020.
  2. All talks will be pre-recorded and played in dedicated Zoom rooms (one per paper & poster session) during the times show below.
  3. Oral presentations pre-recorded talks are 15 minutes and poster pre-recorded talks are 5 minutes.
  4. All authors should already have received instructions on how to pre-record their talks using the SlidesLive service.
  5. Each paper will be followed by a Live 5-minute Q&A.
  6. At least one of the authors must be available in their respective Zoom room to start their presentation and for the Q&A at the end.
  7. Zoom links will be provided closer to the workshop date.
  8. All times are 24 hours PDT.
July 10, 2020 (All times are in PDT)
6:00-6:10 Opening Remarks
6:10–7:30 Paper Session 1
6:10–6:30 Linguistic Features for Readability Assessment + Live Q&A. Tovly Deutsch, Masoud Jasbi and Stuart Shieber.
6:30–6:50 Using PRMSE to Evaluate Automated Scoring Systems in the Presence of Label Noise + Live Q&A. Anastassia Loukina, Nitin Madnani, Aoife Cahill, Lili Yao, Matthew S. Johnson, Brian Riordan and Daniel F. McCaffrey.
6:50–7:10 Multiple Instance Learning for Content Feedback Localization without Annotation + Live Q&A. Scott Hellman, William Murray, Adam Wiemerslage, Mark Rosenstein, Peter Foltz, Lee Becker and Marcia Derr.
7:10–7:30 Complementary Systems for Off-topic Spoken Response Detection + Live Q&A. Vatsal Raina, Mark Gales and Kate Knill.
7:30–8:00 Break
8:00–9:10 Poster Session 1
8:00–8:10 CIMA: A Large Open Access Dialogue Dataset for Tutoring + Live Q&A. Katherine Stasaski, Kimberly Kao and Marti A. Hearst.
8:10–8:20 Becoming Linguistically Mature: Modeling English and German Children’s Writing Development Across School Grades + Live Q&A. Elma Kerz, Yu Qiao, Daniel Wiechmann and Marcus Ströbel.
8:20–8:30 Annotation and Classification of Evidence and Reasoning Revisions in Argumentative Writing + Live Q&A. Tazin Afrin, Elaine Lin Wang, Diane Litman, Lindsay Clare Matsumura and Richard Correnti.
8:30–8:40 Can Neural Networks Automatically Score Essay Traits? + Live Q&A. Sandeep Mathias and Pushpak Bhattacharyya.
8:40–8:50 Tracking the Evolution of Written Language Competence in L2 Spanish Learners + Live Q&A. Alessio Miaschi, Sam Davidson, Dominique Brunato, Felice Dell’Orletta, Kenji Sagae, Claudia Helena Sanchez-Gutierrez and Giulia Venturi.
8:50–9:00 Distractor Analysis and Selection for Multiple-Choice Cloze Questions for Second-Language Learners + Live Q&A. Lingyu Gao, Kevin Gimpel and Arnar Jensson.
9:00–9:10 Assisting Undergraduate Students in Writing Spanish Methodology Sections + Live Q&A. Samuel González-López, Steven Bethard and Aurelio Lopez-Lopez.
9:10–10:10 Break
10:10–11:30 Paper Session 2
10:10–10:30 Applications of Natural Language Processing in Bilingual Language Teaching: An Indonesian-English Case Study + Live Q&A. Zara Maxwelll-Smith, Simón González Ochoa, Ben Foley and Hanna Suominen.
10:30–10:50 An Empirical Investigation of Neural Methods for Content Scoring of Science Explanations + Live Q&A. Brian Riordan, Sarah Bichler, Allison Bradford, Jennifer King Chen, Korah Wiley, Libby Gerard and Marcia C. Linn.
10:50–11:10 An Exploratory Study of Argumentative Writing by Young Students: A transformer-based Approach + Live Q&A. Debanjan Ghosh, Beata Beigman Klebanov and Yi Song.
11:10–11:30 Should You Fine-Tune BERT for Automated Essay Scoring? + Live Q&A. Elijah Mayfield and Alan W Black.
11:30–12:00 Break
12:00–13:00 Poster Session 2
12:00–12:10 GECToR – Grammatical Error Correction: Tag, Not Rewrite + Live Q&A. Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub and Oleksandr Skurzhanskyi.
12:10–12:20 Interpreting Neural CWI Classifiers’ Weights as Vocabulary Size + Live Q&A. Yo Ehara.
12:20–12:30 Automated Scoring of Clinical Expressive Language Evaluation Tasks + Live Q&A. Yiyi Wang, Emily Prud’hommeaux, Meysam Asgari and Jill Dolata.
12:30–12:40 Context-based Automated Scoring of Complex Mathematical Responses + Live Q&A. Aoife Cahill, James H Fife, Brian Riordan, Avijit Vajpayee and Dmytro Galochkin.
12:40–12:50 Predicting the Difficulty and Response Time of Multiple Choice Questions Using Transfer Learning + Live Q&A. Kang Xue, Victoria Yaneva, Christopher Runyon and Peter Baldwin.
12:50–13:00 A Comparative Study of Synthetic Data Generation Methods for Grammatical Error Correction + Live Q&A. Max White and Alla Rozovskaya.
13:00–13:30 Ambassador Paper & Closing
13:00–13:20 Modeling Individual Differences in Second-Language Reading Skill using Language Experience, Executive Attention, and Cross-Linguistic Interactions + Live Q&A. Brianna L. Yamasaki.
13:20–13:30 Closing Remarks

Attending Virtually

  • The opening remarks, the closing remarks, and the ambassador paper presentation are the only live (not pre-recorded) presentations on the schedule. All other talks are pre-recorded.
  • The liveStream window on the virtual workshop website will show everything: pre-recorded talks, the live zoom Q&A, and any live presentations.
  • Zoom will only be used for for asking & answering questions. There is no need to join Zoom unless you are an author or want to ask a question.
  • The Zoom meeting that will be used for questions is linked at the top of the virtual workshop with the name “Join Zoom Meeting”.
  • After every talk on the livestream, there will be a 1 minute break to give time to authors and folks with questions to join Zoom.
  • Everyone who joins the Zoom meeting will have their video on, but their mics muted. They cannot unmute themselves. If you have a question, raise your hand virtually via the raise hand Zoom functionality. The session chair will be able to see who has raised their hand in the participants tab, automatically ordered from first to last. The Session Chair can unmute folks so that they can ask their question.
  • Once the 5-minute Q&A is over, the Session Chair will move on to the next talk in their session.
  • Questions may also be asked on the #workshop-13-bea ACL 2020 rocketchat channel. However, we recommend that authors focus on answering the live Q&A questions on Zoom first and answer the chat questions later when they have time (think of such questions as “taking things offline” in regular conferences).

This is our first virtual BEA so we request everyone to be patient if things don’t work exactly as advertised!

Submission Information

We will be using the ACL Submission Guidelines for the BEA Workshop this year. Authors are invited to submit a full paper of up to eight (8) pages of content, plus unlimited references; final versions of long papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account. We also invite short papers of up to of up to four (4) pages of content, plus unlimited references. Upon acceptance, short papers will be given five (5) content pages in the proceedings. Authors are encouraged to use this additional page to address reviewers’ comments in their final versions.

Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either “full paper + demo” or “short paper + demo” under “Submission Category” in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …”, should be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …”.

We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.

We will be using the START conference system to manage submissions: https://www.softconf.com/acl2020/bea/

Double Submission Policy

We will follow the official ACL double-submission policy. Specificially:

Papers being submitted both to BEA and another conference or workshop must:

  • Note on the title page the other conference or workshop to which they are being submitted.
  • State on the title page that if the authors choose to present their paper at BEA (assuming it was accepted), then the paper will be withdrawn from other conferences and workshops.




Organizing Committee

Program Committee

  • Tazin Afrin, University of Pittsburgh
  • David Alfter, University of Gothenburg
  • Dimitris Alikaniotis, Grammarly
  • Fernando Alva-Manchego, University of Sheffield
  • Rajendra Banjade, Audible (Amazon)
  • Timo Baumann, Universität Hamburg
  • Lee Becker, Pearson
  • Beata Beigman Klebanov, Educational Testing Service
  • Lisa Beinbron, University of Amsterdam
  • Maria Berger, German Research Center for Artificial Intelligence
  • Kay Berkling, DHBW Cooperative State University Karlsruhe
  • Delphine Bernhard, Université de Strasbourg, France
  • Sameer Bhatnagar, Polytechnique Montreal
  • Serge Bibauw, KU Leuven; UCLouvain; Universidad Central del Ecuador
  • Joachim Bingel, University of Copenhagen
  • Kristy Boyer, University of Florida
  • Chris Brew, Facebook AI
  • Ted Briscoe, University of Cambridge
  • Chris Brockett, Microsoft Research AI
  • Julian Brooke, University of British Columbia
  • Christopher Bryant, University of Cambridge
  • Jill Burstein, Educational Testing Service
  • Aoife Cahill, Educational Testing Service
  • Andrew Caines, University of Cambridge
  • Guanliang Chen, Monash University
  • Mei-Hua Chen, Department of Foreign Languages and Literature
  • Martin Chodorow, City University of New York
  • Leshem Choshen, Hebrew University of Jerusalem
  • Mark Core, University of Southern California
  • Luis Fernando D’Haro, Universidad Politécnica de Madrid
  • Vidas Daudaravicius, UAB VTeX
  • Orphée De Clercq, LT3, Ghent University
  • Kordula De Kuthy, Tübingen University
  • Iria del Río Gayo, University of Lisbon
  • Carrie Demmans Epp, University of Alberta
  • Ann Devitt, Trinity College, Dublin
  • Yo Ehara, Shizuoka Institute of Science and Technology
  • Noureddine Elouazizi, Faculty of Science
  • Keelan Evanini, Educational Testing Service
  • Youmna Farag, University of Cambridge
  • Mariano Felice, University of Cambridge
  • Michael Flor Educational Testing Service
  • Thomas François, Université catholique de Louvain
  • Jennifer-Carmen Frey, Eurac Research
  • Michael Gamon, Microsoft Research
  • Dipesh Gautam, University of Memphis
  • Sian Gooding, University of Cambridge
  • Cyril Goutte, National Research Council Canada
  • Roman Grundkiewicz, University of Edinburgh
  • Masato Hagiwara, Octanove Labs LLC
  • Jiangang Hao, Educational Testing Service
  • Polina Harik, NBME
  • Homa Hashemi, Microsoft
  • Trude Heift, Simon Fraser University
  • Heiko Holz, LEAD Graduate School & Research Network
  • Andrea Horbach, University Duisburg-Essen
  • Renfen Hu, Beijing Normal University
  • Chung-Chi, Huang Frostburg State University
  • Yi-Ting Huang, Academia Sinica
  • Radu Tudor Ionescu, University of Bucharest
  • Lifeng Jin, Ohio State University
  • Marcin Junczys-Dowmunt, Microsoft
  • Tomoyuki Kajiwara, Osaka University
  • Elma Kerz, RWTH Aachen University
  • Fazel Keshtkar, St. John’s University
  • Mamoru Komachi, Tokyo Metropolitan University
  • Lun-Wei Ku, Academia Sinica
  • Kristopher Kyle, University of Oregon
  • Ji-Ung Lee, UKP Lab, TU Darmstadt
  • Lung-Hao Lee, National Central University
  • John Lee, City University of Hong Kong
  • Chee Wee (Ben) Leong, Educational Testing Service
  • Chen Liang, Facebook
  • Diane Litman, University of Pittsburgh
  • Zitao Liu, TAL Education Group
  • Peter Ljunglöf, University of Gothenburg; Chalmers University of Technology
  • Anastassia Loukina, Educational Testing Service
  • Lieve Macken, Ghent University
  • Nabin Maharjan, Audible (Amazon)
  • Montse Maritxalar, University of the Basque Country
  • James Martin, University of Colorado Boulder
  • Irina Maslowski
  • Ditty Mathew, Accenture
  • Sandeep Mathias, IIT Bombay
  • Noboru Matsuda, North Carolina State University
  • Julie Medero, Harvey Mudd College
  • Detmar Meurers, University of Tübingen
  • Michael Mohler, Language Computer Corporation
  • Natawut Monaikul, University of Illinois at Chicago
  • William Murray, Pearson
  • Farah Nadeem, University of Wahington
  • Courtney Napoles, Grammarly
  • Diane Napolitano, Refinitiv
  • Hwee Tou Ng, National University of Singapore
  • Huy Nguyen, LingoChamp
  • Neasa Ní Chiaráin, Trinity College, Dublin
  • Rodney Nielsen, University of North Texas
  • Yoo Rhee Oh, Electronics and Telecommunications Research Institute (ETRI)
  • Robert Östling, Department of linguistics, Stockholm university
  • Ulrike Pado, HFT Stuttgart
  • Patti Price, PPRICE Speech and Language Technology
  • Long Qin, Singsound Inc
  • Mengyang Qiu, University at Buffalo
  • Martí Quixal, Universität Tübingen
  • Vipul Raheja, Grammarly
  • Zahra Rahimi Pandora Media
  • Taraka Rama, University of North Texas
  • Vikram Ramanarayanan, Educational Testing Service; University of California, San Francisco
  • Hanumant Redkar, IIT Bombay
  • Marek Rei, University of Cambridge
  • Robert Reynolds, Brigham Young University
  • Brian Riordan, Educational Testing Service
  • Andrew Rosenberg, Google
  • Alla Rozovskaya, City University of New York
  • C. Anton Rytting, University of Maryland
  • Keisuke Sakaguchi, Allen Institute for Artificial Intelligence
  • Katira Soleymanzadeh, EGE University
  • Swapna Somasundaran, Educational Testing Service
  • Helmer Strik, Radboud University Nijmegen
  • Jan Švec, University of West Bohemia
  • Anaïs Tack, UCLouvain & KU Leuven
  • Alexandra Uitdenbogerd, RMIT University
  • Shalaka Vaidya, Research assistant
  • Sowmya Vajjala, National Research Council, Canada
  • Piper Vasicek, Brigham Young University
  • Giulia Venturi, Institute for Computational Linguistics
  • Tatiana Vodolazova, University of Alicante
  • Elena Volodina, University of Gothenburg, Sweden
  • Yiyi Wang, UIUC; Boston College
  • Shuting Wang, Facebook
  • Zarah Weiss, University of Tübingen
  • Michael White, The Ohio State University; Facebook AI
  • Alistair Willis, Open University, UK
  • Wei Xu, Ohio State University
  • Yiqiao Xu, North Carolina State University
  • Kevin Yancey, Duolingo
  • Victoria Yaneva, NBME; University of Wolverhampton
  • Seid Muhie Yimam, University of Hamburg
  • Marcos Zampieri, Rochester Institute of Technology
  • Klaus Zechner, Educational Testing Service
  • Fabian Zehner, DIPF, Leibniz Institute for Research and Information in Education
  • Haoran Zhang, University of Pittsburgh