BEA Team

BEA Team

SIGEDU/BEA Newsletter #3

16 minute read

Hi Everyone,

with the BEA14 deadline fast approaching, we hope you are planning to submit your papers to the workshop! Besides the final call for papers for BEA14, we have several other updates for you:

  • BEA14 Final Call for Papers
  • Results of the GEC Shared Task 2019
  • Update on BEA14 Sponsorship
  • Upcoming Conferences on Educational Applications
  • Job Advertisements

You can directly navigate to any of the sections by using the table of contents to the right.

As always, if you know of any corpora, resources, tools, conferences, job postings, etc. that would be good to include in the newsletter, please let us know and we’ll be sure to include them in the next version.

Best, the SIGEDU team

BEA14 Final Call for Papers

  • The 14th Workshop on Innovative Use of NLP for Building Educational Applications (BEA14)
  • Location: Florence, Italy; August 2, 2019 (co-located with ACL)
  • Website: https://sig-edu.org/bea/2019
  • Submission Deadline: Friday, April 26, 2019, 11:59pm EST

Workshop Description

The BEA Workshop is a leading venue for NLP innovation in the context of educational applications. It is one of the largest one-day workshops in the ACL community with over 80 attendees in the past several years. The growing interest in educational applications and a diverse community of researchers involved resulted in the creation of the Special Interest Group in Educational Applications (SIGEDU) in 2017 which currently has 159 members.

The workshop’s continuing growth highlights the alignment between societal needs and technological advances. NLP capabilities can now support an array of learning domains, including writing, speaking, reading, science, and mathematics, as well as the related intra-personal (e.g., self-confidence) and inter-personal (e.g., peer collaboration) skills. Within these areas, the community continues to develop and deploy innovative NLP approaches for use in educational settings. In the writing and speech domains, automated writing evaluation (AWE) and speech scoring applications, respectively, are commercially deployed in high-stakes assessment and in instructional contexts (e.g., Massive Open Online Courses (MOOCs) and K-12 classrooms). Commercially-deployed plagiarism detection is also commonly used in both K-12 and higher education settings. For writing, the focus is on innovations that support writing tasks requiring source use, argumentative discourse, and factual content accuracy. For speech, there is an interest in advancing automated scoring to include the evaluation of discourse and content features in responses to spoken assessments. General advances in speech technology have promoted a renewed interest in spoken dialog and multimodal systems for instruction and assessment, for instance, for workplace interviews and simulated teaching environments. The explosive growth of mobile applications for game-based and simulation-based applications is another area where NLP has begun to play a large role, especially for language learning.

NLP for educational applications has gained visibility outside of the NLP community. First, the Hewlett Foundation reached out to the public and private sectors and sponsored two competitions: one for automated essay scoring, and the other for scoring of short response items. The motivation driving these competitions was to engage the larger scientific community in this enterprise. Learning @ Scale is a relatively new venue for NLP research in education. MOOCs now incorporate AWE systems to manage several thousand assignments that may be received during a single MOOC course. MOOCs for Refugees have more recently emerged in response to the current social situations. Courses include language learning, and we can imagine that AWE and other NLP capabilities could support coursework. Another breakthrough for educational applications within the CL community is the presence of a number of shared-task competitions over the past several years – including three shared tasks on grammatical error detection and correction. NLP/Education shared tasks have seen new areas of research, such as the Automated Evaluation of Scientific Writing at BEA 11, Native Language Identification at BEA 12, Second Language Acquisition Modelling at BEA 13, and Complex Word Identification at BEA 13.  These competitions increased the visibility of, and interest in, our field.

The 14th BEA workshop will have oral presentation sessions and a large poster session in order to maximize the amount of original work presented. We expect that the workshop will continue to highlight novel technologies and opportunities for educational NLP in English as well as other languages. The workshop will solicit both full papers and short papers for either oral or poster presentation. We will solicit papers that incorporate NLP methods, including, but not limited to: automated scoring of open-ended textual and spoken responses; game-based instruction and assessment; educational data mining; intelligent tutoring; peer review; grammatical error detection and correction; learner cognition; spoken dialog; multimodal applications; tools for teachers and test developers; and use of corpora. Specific topics include:

Automated scoring/evaluation for written student responses (across multiple genres)

  • Content analysis for scoring/assessment
  • Detection and correction of grammatical and other types of errors
  • Argumentation, discourse, sentiment, stylistic analysis, & non-literal language

Intelligent Tutoring (IT), Collaborative Learning Environments

  • Educational Data Mining: Collection of user log data from educational applications
  • Game-based learning
  • Multimodal communication (including dialog systems) between students and computers

Learner cognition

  • Assessment of learners’ language and cognitive skill levels
  • Systems that detect and adapt to learners’ cognitive or emotional states
  • Tools for learners with special needs

Use of corpora in educational tools

  • Data mining of learner and other corpora for tool building
  • Annotation standards and schemas / annotator agreement

Tools and applications for classroom teachers and/or test developers

  • NLP tools for second and foreign language learners
  • Semantic-based access to instructional materials to identify appropriate texts
  • Tools that automatically generate test questions

Shared Task: Grammatical Error Correction

  The BEA14 workshop is also hosting a shared task on Grammatical Error Correction.

More information can be found on the task webpage: https://www.cl.cam.ac.uk/research/nl/bea2019st/

Submission Information

  We will be using the ACL Submission Guidelines for the BEA14 Workshop this year. Long papers may consist of up to eight (8) pages of content, plus unlimited references; final versions of long papers will be given one additional page of content (up to 9 pages) so that reviewers’ comments can be taken into account. Short papers may consist of up to four (4) pages of content, plus unlimited references. Upon acceptance, short papers will be given five (5) content pages in the proceedings. Authors are encouraged to use this additional page to address reviewers’ comments in their final versions.

Papers which describe systems are also invited to give a demo of their system. If you would like to present a demo in addition to presenting the paper, please make sure to select either “full paper + demo” or “short paper + demo” under “Submission Category” in the START submission page.

Previously published papers cannot be accepted. The submissions will be reviewed by the program committee. As reviewing will be blind, please ensure that papers are anonymous. Self-references that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …”, should be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …”.

We have also included conflict of interest in the submission form. You should mark all potential reviewers who have been authors on the paper, are from the same research group or institution, or who have seen versions of this paper or discussed it with you.

We will be using the START conference system to manage submissions: https://www.softconf.com/acl2019/bea/.

Important Dates

  • Submission Deadline: Friday, April 26, 2019, 11:59pm  EST
  • Notification of Acceptance: Friday, May 24, 2019
  • Camera-ready Papers Due: Monday, June 3, 2019
  • Workshop: Friday, August 2, 2019

Organizing Committee

  • Helen Yannakoudakis, University of Cambridge (primary contact)
  • Ekaterina Kochmar, University of Cambridge
  • Claudia Leacock, Grammarly
  • Nitin Madnani, Educational Testing Service
  • Ildikó Pilán, Develop Diverse
  • Torsten Zesch, University of Duisburg-Essen

If you have any questions, please contact us at the official workshop emil address: bea.nlp.workshop@gmail.com

Program Committee

  • Tazin Afrin, University of Pittsburgh
  • David Alfter, University of Gothenburg
  • Dimitrios Alikaniotis, Grammarly
  • Rajendra Banjade, Audible Inc. (an Amazon company)
  • Timo Baumann, Carnegie Mellon University
  • Lee Becker, Pearson
  • Beata Beigman Klebanov, ETS
  • Kay Berkling, Cooperation State university Karlsruhe Germany
  • Suma Bhat, University of Illinois at Urbana-Champaign
  • Sameer Bhatnagar, Polytechnique Montreal
  • Joachim Bingel, University of Copenhagen
  • Karim Bouzoubaa, Mohammed V University in Rabat
  • Chris Brew, Facebook
  • Ted Briscoe, University of Cambridge
  • Julian Brooke, University of British Columbia
  • Dominique Brunato, Institute for Computational Linguistics, ILC-CNR, Pisa, Italy
  • Christopher Bryant, University of Cambridge
  • Paula Buttery, University of Cambridge
  • Andrew Caines, University of Cambridge
  • Mei-Hua Chen, Department of Foreign Languages and Literature, Tunghai University
  • Martin Chodorow, ETS & City University of New York
  • Shamil Chollampatt, National University of Singapore
  • Mark Core, University of Southern California
  • Vidas Daudaravicius, UAB VTEX
  • Kordula De Kuthy, University of Tübingen
  • Carrie Demmans Epp, University of Alberta
  • Yo Ehara, Faculty of Informatics, Shizuoka Institute of Science and Technology
  • Keelan Evanini, Educational Testing Service
  • Mariano Felice, University of Cambridge
  • Michael Flor, Educational Testing Service
  • Thomas François, Université catholique de Louvain
  • Michael Gamon, Microsoft Research
  • Dipesh Gautam, The University of Memphis
  • Sian Gooding, University of Cambridge
  • Jonathan Gordon, Vassar College
  • Cyril Goutte, National Research Council Canada
  • Iryna Gurevych, UKP Lab, TU Darmstadt
  • Na-Rae Han, University of Pittsburgh
  • Jiangang Hao, Educational Testing Service
  • Homa Hashemi, Microsoft
  • Trude Heift, Simon Fraser University
  • Derrick Higgins, American Family Insurance
  • Heiko Holz, LEAD Graduate School & Research Network at the University of Tuebingen
  • Andrea Horbach, University Duisburg-Essen
  • Chung-Chi Huang, Frostburg State University
  • Yi-Ting Huang, Academia Sinica
  • Radu Tudor Ionescu, University of Bucharest
  • Lifeng Jin, The Ohio State University
  • Pamela Jordan, University of Pittsburgh
  • Taraka Kasicheyanula, University of Oslo
  • Elma Kerz, RWTH Aachen
  • Fazel Keshtkar, St. John's University
  • Mamoru Komachi, Tokyo Metropolitan University
  • Lun-Wei Ku, Academia Sinica
  • Ji-Ung Lee, UKP Lab, Technische Universität Darmstadt
  • John Lee, City University of Hong Kong
  • Lung-Hao Lee, National Central University
  • Ben Leong, Educational Testing Service
  • James Lester, North Carolina State University
  • Chen Liang, Facebook
  • Diane Litman, University of pittsburgh
  • Yang Liu, Laix
  • Peter Ljunglöf, University of Gothenburg
  • Anastassia Loukina, ETS
  • Xiaofei Lu, Pennsylvania State University
  • Luca Lugini, University of Pittsburgh
  • Nabin Maharjan, University of Memphis
  • Jean Maillard, University of Cambridge
  • Shervin Malmasi, Harvard Medical School
  • Montse Maritxalar, University of the Basque Country
  • Ditty Mathew, IIT Madras
  • Julie Medero, Harvey Mudd College
  • Beata Megyesi, Uppsala University
  • Detmar Meurers, Universität Tübingen
  • Elham Mohammadi, CLaC Laboratory, Concordia University
  • Maria Moritz, University of Goettingen
  • William Murray, Pearson
  • Courtney Napoles, Grammarly
  • Diane Napolitano, Educational Testing Service
  • Hwee Tou Ng, National University of Singapore
  • Huy Nguyen, LingoChamp
  • Rodney Nielsen, University of North Texas
  • Nobal Niraula, Boeing Research and Technology
  • Yoo Rhee Oh, Electronics and Telecommunications Research Institute (ETRI)
  • Constantin Orasan, University of Wolverhampton
  • Ulrike Pado, HFT Stuttgart
  • Alexis Palmer, University of North Texas
  • Martí Quixal, Universitat Oberta de Catalunya
  • Zahra Rahimi, University of Pittsburgh
  • Lakshmi Ramachandran, Amazon Search
  • Hanumant Redkar, Indian Institute of Technology Bombay
  • Marek Rei, University of Cambridge
  • Robert Reynolds, Brigham Young University
  • Brian Riordan, ETS
  • Kat Robb, University of Leeds
  • Andrew Rosenberg, Google
  • Mark Rosenstein, Pearson
  • Alla Rozovskaya, City University of New York
  • C. Anton Rytting, University of Maryland
  • Keisuke Sakaguchi, Allen Institute for Artificial Intelligence
  • Allen Schmaltz, Harvard University
  • Mat Schulze, San Diego State University
  • Burr Settles, Duolingo
  • Serge Sharoff, University of Leeds
  • Swapna Somasundaran, ETS
  • Richard Sproat, Google
  • Helmer Strik, Centre for Language and Speech Technology (CLST), Centre for Language Studies (CLS), Radboud University Nijmegen
  • Jan Švec, NTIS, University of West Bohemia
  • Anaïs Tack, UCLouvain & KU Leuven
  • Yuen-Hsien Tseng, National Taiwan Normal University
  • Giulia Venturi, Institute for Computational Linguistics "A. Zampolli", Italy
  • Aline Villavicencio, Federal University of Rio Grande do Sul (Brazil) and University of Essex (UK)
  • Carl Vogel, Trinity College Dublin
  • Elena Volodina, University of Gothenburg
  • Shuting Wang, Facebook Inc
  • Michael White, The Ohio State University
  • Michael Wojatzki, LTL, University of Duisburg-Essen
  • Magdalena Wolska, Eberhard Karls Universität Tübingen
  • Huichao Xue, LinkedIn
  • Victoria Yaneva, National Board of Medical Examiners / University of Wolverhampton
  • Zheng Yuan, University of Cambridge
  • Marcos Zampieri, University of Wolverhampton (UK)
  • Klaus Zechner, ETS
  • Fan Zhang, Google
  • Haoran Zhang, University of Pittsburgh
  • Ramon Ziai, University of Tübingen

GEC Shared Task 2019 Results

We are pleased to report that the recently completed BEA 19 Shared Task on Grammatical Error Correction (GEC) was a huge success and received a total of 37 submissions across all 3 tracks. Of these, 21 submissions were also made to the main Restricted Track, which is the largest number of submissions to a single GEC shared task to date. A further 9 teams also submitted to the Low Resource track, which aimed to encourage research into low-resource GEC where large quantities of annotated data may not be available. More information and shared task results can be found on the following page: https://www.cl.cam.ac.uk/research/nl/bea2019st/

Participants are currently writing papers about their submissions, and these will all be published and presented at the next BEA workshop in Florence. Many thanks to Christopher Bryant, Mariano Felice, Øistein Andersen and Ted Briscoe for organising the shared task, and we also look forward to a Shared Task overview paper in August!

Update on BEA14 Sponsorship

We are extremely grateful to our sponsors for BEA14! This year BEA workshop will be sponsored by Duolingo, iLexIR, NBME, Grammarly, Educational Testing Service, and Newsela. If your organization would like to subsidize the workshop costs and become a Gold ($500), Silver ($250) or Bronze ($100) sponsor for BEA14, please get in touch! This year, we want to continue helping students to attend the workshop, including the accommodation of the student post-workshop dinner. Perks of sponsorship include logos on the workshop website and proceedings.

Upcoming Conferences on Educational Applications

SLaTE 2019

The 8th ISCA Workshop on Speech and Language Technology in Education (SLaTE 2019) will be held in conjunction with Interspeech 2019 in Graz, Austria, on September 20 - 21, 2019.   The workshop will welcome papers covering all topics related to speech and language technology in education. This includes, but is not limited to, the following topics:

  • speech technology for first and second language learning
  • natural language processing for education
  • spoken dialog systems for education
  • applications using speech and/or natural language processing for education
  • intelligent tutoring systems using speech and natural language
  • robot-assisted learning
  • development of language resources for SLaTE applications
  • assessment and user studies of SLaTE methods and applications
  • theoretical and methodological issues in SLaTE
  • use of speech synthesis for language learning
  • serious games in SLaTE
  • SLaTE for communicative disorders and pathology

In addition, following the first edition of the Spoken CALL Shared Task at SLaTE 2017 and the second edition at Interspeech 2018, we invite submissions to the third edition as a special session at SLaTE 2019. This shared task will use the same training data and resources as the second edition, but the results will be evaluated on new test data. Given the strong results reported in the second edition, we are however making an important change: the third edition will use a new and more challenging scoring metric. Full details are available from the Shared Task 3 site.

Important Dates:

  • Paper submission deadline: June 3, 2019
  • Notification of paper acceptance: July 8, 2019
  • Camera-ready paper submission deadline: July 22, 2019
  • Early registration deadline: July 22, 2019
  • SLaTE 2019 workshop at Interspeech: September 20 - 21, 2019

Paper submission site: https://easychair.org/my/conference.cgi?conf=slate20190

8th workshop on NLP4CALL

Co-located with NoDaLiDa 2019, Turku, Finland

Workshop description

The workshop series on Natural Language Processing (NLP) for Computer-Assisted Language Learning (NLP4CALL) is a meeting place for researchers working on the integration of Natural Language Processing and Speech Technologies in CALL systems and exploring the theoretical and methodological issues arising in this connection. The latter includes, among others, insights from Second Language Acquisition (SLA) research, on the one hand, and promote development of “Computational SLA” through setting up Second Language research infrastructure(s), on the other.

The intersection of Natural Language Processing (or Language Technology / Computational Linguistics) and Speech Technology with Computer-Assisted Language Learning (CALL) brings “understanding” of language to CALL tools, thus making CALL intelligent. This fact has given the name for this area of research – Intelligent CALL, ICALL. As the definition suggests, apart from having excellent knowledge of Natural Language Processing and/or Speech Technology, ICALL researchers need good insights into second language acquisition theories and practices, as well as knowledge of second language pedagogy and didactics. This workshop invites therefore a wide range of ICALL-relevant research, including studies where NLP-enriched tools are used for testing SLA and pedagogical theories, and vice versa, where SLA theories, pedagogical practices or empirical data are modeled in ICALL tools.

The NLP4CALL workshop series is aimed at bringing together competences from these areas for sharing experiences and brainstorming around the future of the field.

We welcome papers:

  • that describe research directly aimed at ICALL;
  • that demonstrate actual or discuss the potential use of existing Language and Speech Technologies or resources for language learning;
  • that describe the ongoing development of resources and tools with potential usage in ICALL, either directly in interactive applications, or indirectly in materials, application or curriculum development, e.g.   learning   material   generation, assessment of learner texts and responses, individualized learning solutions, provision of feedback;
  • that discuss challenges and/or research agenda for ICALL
  • that describe empirical studies on language learner data.

A special focus is given to the established and upcoming infrastructures aimed at SLA and learner corpus research, covering questions such as data collection, legal issues, reliability of annotation, annotation tool development, search environments for SLA-relevant data, etc. We encourage paper presentations and software demonstrations describing the above-mentioned themes primarily, but not exclusively, for the Nordic languages.

Invited speakers

This year we have the pleasure to welcome two invited speakers:

  1. Thomas François, UCLouvain

    Thomas François is Assistant Professor in Applied Linguistics and Natural Language Processing at UCLouvain (Cental). His work focuses on automatic assessment of text readability, automatic text simplification, complex word identification, efficient communication in business, and the use of French as a professional language. He has been an invited researcher at IRCS (University of Pennsylvania) as a Fulbright and BAEF fellow and, later, has been a FNRS post-doctoral researcher. He has led research projects such as CEFRLex (http://cental.uclouvain.be/cefrlex/), a CEFR-graded lexicon for foreign language learning or AMesure (http://cental.uclouvain.be/amesure/), a platform to support simple writing. His work on readability for French as a foreign language has been awarded the best thesis Award by the ATALA in 2012 and the best paper in the TALN2016 conference.

    Title: TBD

  2. Egon Stemle, Eurac

    More information soon.

    Title: TBD

Submission information

Authors are invited to submit long papers (8-12 pages) alternatively short papers (4-7 pages), page count not including references. We will be using the NoDaLiDa 2019 template for the workshop this year. The author kit, including LaTeX and Word template files can be accessed here.

Submissions will be managed through the electronic conference management system EasyChair. Papers must be submitted digitally through the conference management system, in PDF format. Final camera-ready versions of accepted papers will be given an additional page to address reviewer comments.

Papers should describe original unpublished work or work-in-progress. Papers will be peer reviewed by at least two members of the program committee in a double-blind fashion. All accepted papers will be collected into a proceedings volume to be submitted for publication in the NEALT Proceeding Series (Linköping Electronic Conference Proceedings) and, additionally, double-published through the ACL anthology, following experiences from the previous NLP4CALL editions (e.g. the 7th NLP4CALL).

Important dates

  • June 30: paper submission deadline
  • 18 August: notification of acceptance
  • 6 September: camera-ready papers for publication
  • 30 September: workshop date

Organizers

David Alfter1, Elena Volodina1, Ildikó Pilán, Herbert Lange2, Lars Borin2

  1. Språkbanken, University of Gothenburg, Sweden
  2. Department of Computer Science and Engineering, University of Gothenburg and Chalmers University of Technology, Sweden

Contact

For further information, see the workshop website. For any questions, please contact David Alfter. Follow us on Twitter @NLP4CALL.

Job Advertisements

Research Scientist and Research Engineer at ETS

ETS’s Research & Development division has openings for a Research Scientist and a Research Engineer in the NLP & Speech research group. The projects in this research group focus on the application of NLP, speech, dialog, and multimodal processing algorithms in automated scoring capabilities for assessment and learning tasks involving constructed responses (such as essays and spoken responses). This is an excellent opportunity to be part of a world-renowned research and development team and have a significant impact on existing and next-generation NLP, speech, dialog, and multimodal systems and their use in educational applications.  Follow the links for further details about the Research Scientist position and the Research Engineer position.