Ling7800/CSCI 7000: Computational Lexical Semantics
Fall 2018
Instructors:
Martha Palmer
Time and Location: Fridays, 2:00 - 4:30, Fleming 178
Assessment: Four homeworks, one Paper presentation, and a term project.
Office Hours: Martha Palmer, Thursday 2-3pm, Friday 4:30-5:30pm, Fleming 289
Textbooks:
Semantic Role Labeling (eBook),
Martha Palmer, Daniel Gildea, Nianwen Xue,
Synthesis Lectures on Human
Language Technologies ,
ed., Graeme Hirst, Morgan & Claypool, 2010. ISBN: 9781598298321
available on line on campus through Chinook
Representation and Inference for Natural Language.
A First Course in Computational Semantics.
Patrick Blackburn and Johan Bos, 2005,
CSLI Publications. ISBN: 1-57586-496-7
selected chapters, available from the CU bookstore and D2L
Theme
Lexical semantics is becoming an increasingly
important part of Natural Language Processing (NLP), as the field is
beginning to address semantics at a large scale. This graduate seminar
covers key issues in computational lexical
semantics. We start with an introduction to theoretical models of
lexical semantics and events, considering both their adequacy as
linguistic models and their place in NLP. We focus particularly
on computational lexical resources such as PropBank, VerbNet, FrameNet and the
Generative Lexicon, and examine their strengths and limitations with
respect to NLP applications. We will introduce apporoaches to
developing automatic classifiers that are intended to make use of
these resources and to offer richer representations of sentences in
context. These techniques can be fully supervised (requiring
hand-labeled training data), semi-supervised, or unsupervised
(learning lexical information from unlabeled text).
We will also discuss the impact of Word Embeddings as an
approximmation of semantic similarity and the resulting implications for future research directions.
Suggested Schedule and Readings
Introduction and Module 1: the Lexical Semantics of Verbs - Chap 1
- Aug 31 Course Overview and Natural Language Processing, the Pundit case study
Palmer, Martha, Carl Weir, Rebecca Passonneau, and Tim Finin.
"The Kernel Text Understanding System."
Artificial Intelligence 63: 17-68: Special Issue on Text Understanding.
October, 1993.
- Aug 31 Thematic Roles in Linguistics,
Assignment 1: Exercises 1, 2 and 3, p. 19, SRL book, Due Sep 14
- Background reading for Assignment:
Fillmore, C. J. 1968 "The Case for Case" in E. Bach and R.T. Harms, eds.
Universals in Linguistic Theory, 1-88. New York: Holt, Rinehart and Winston. Section 3.
Paper
Jackendoff, R.S. 1976 Towards an Explanatory Semantic Representation,
Linguistic Inquiry, 7:1, pp. 89-150.
Paper
Dowty D.R 1991 Thematic Proto-Roles and Argument Selection.
Language 67: 547-619 sections 1-7 Paper
Levin, B. English Verb Classes: A Preliminary Classification Introduction,
MIT Press, pp. 1-23, 1990., Paper
Module 2: Available Computational Lexicons - Chap 2
- Sep 7 Word Senses, WordNet and the OntoNotes Groupings
Palmer, M., Dang, H. and Fellbaum, C, 2007,
Making Fine-grained and Coarse-grained sense distinctions,both manually and automatically,
Journal of Natural Language Engineering,13:2, 137-163.
George A. Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller, 1993,
Introduction to WordNet: An On-line Lexical Database, 5 Papers on WordNet availalbe from the WordNet web site.
Background Reading for Sense Distinctions:
Edmonds, P. and Hirst, G., Near-Synonymy and Lexical Choice,
Computational Linguistics June, 2002, Vol. 28, No. 2, Pages 105-144,
Paper
Atkins, S., Fillmore, C. J., Johnson, C. R.,
Lexicographic Relevance: Selecting Information from Corpus Evidence,
International Journal of Lexicography, Vol. 16 No. 3, Oxford University Press, 2003,
Paper
- Sep 7 WSD as a Machine Learning Task
Dmitriy Dligach and Martha Palmer.
Good Seed Makes a Good Crop: Accelerating Active Learning Using Language Modeling. In ACL '11: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. June 19 - 24, 2011, Portland, OR.
11)Danilo Croce; Alessandro Moschitti; Roberto Basili; Martha Palmer, Verb Classification using Distributional Similarity in Syntactic and Semantic Structures, In the Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, ACL-2012, Jeju Island, Korea, July, 2012.
Daisuke Kawahara, Daniel W. Peterson and Martha Palmer. (2014) A Step-wise Usage-based Method for Inducing Polysemy-aware Verb Classes, In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL2014), Baltimore, MD.
IP Klapaftis, S Manandhar, (2008)
Word Sense Induction Using Graphs of Collocations,
ECAI, 298-302
- Sep 14 ICS Talk, Rebecca Knowles (JHU), Human in the Loop: Interactive and Adaptive Neural Machine Translation
12:00-1:30 pm, Muen D430, Refreshments Served
- Relevant Readings for in class discussion:
Rebecca's Web Page
Chris Hokamp, Qun Liu, 2017,
Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search ACL 2017.
- Sep 14 The Generative Lexicon/Review Ass 1
Pustejovsky, James, 1991, The Generative Lexicon, ComputationaI Linguistics, Volume 17, Number 4, December.
Hanks, P. and Pustejovsky, J., A Pattern Dictionary for Natual Language Processing, Revue francaise de linguistique appliquie
2005/2 (Vol. X), CAIRN, INFO, 2005.
- Sep 14 PropBank,
Assignment 2: Exercise 2 p. 29, SRL book, Hint: start with VerbNet and use it to find the right frames in PB and FN, and don't forget to provide the unique sense tag: VN class, FN frame, PB roleset Oct 5
Martha Palmer, Dan Gildea, Paul Kingsbury, 2005,
The Proposition Bank: An Annotated Corpus of Semantic Roles,
Computational Linguistics, 31:1 , pp. 71-105.
- Sep 21 VerbNet
Kipper, Karin, Anna Korhonen, Neville Ryant, Martha Palmer. "A Large-scale Classification of English Verbs." Language Resources and Evaluation Journal,42(1). Springer Netherland: 2008. pp. 21-40.
- Sep 21 FrameNet
Fillmore et al 2001
"Building a large lexical databank which provides deep
semantics",
Proceedings of the 15th Pacific Asia Conference on Language, Information and Computation. Eds. Benjamin Tsou, and Olivia Kwong. Hong Kong 2001.
Fillmore, Charles J., Christopher R. Johnson, and Miriam R.L. Petruck. 2002.
Background to FrameNet.
International Journal of Lexicography, 1'6(3):2435
- Access to Computational Lexicons:
WordNet
FrameNet
PropBank
VerbNet
SemLink
VerbCorner
- Sep 28 Preposition SuperSenses
Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Meredith Green, Abhijit Suresh, Kathryn Conger, Tim O Gorman, and Martha Palmer (2016).
A corpus of preposition supersenses. LAW, at ACL 2016, Berlin, Germany.
- Sep 28 Abstract Meaning Representations (AMRs) and After AMRs, QAMR
Laura Banarescu; Claire Bonial; Shu Cai; Madalina Georgescu; Kira Griffitt; Ulf Hermjakob; Kevin Knight; Philipp Koehn; Martha Palmer; Nathan Schneider (2013)
Abstract Meaning Representation for Sembanking Linguistics Annotation Workshop and Interoperability with Discourse, held with ACL 2013, Sofia, Bulgaria.
Nianwen Xue, Ondrej Bojar, Jan Hajic, Martha Palmer, Zdenka Uresova, Xiuhong Zhang
Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech LREC 2014.
Julian Michael, Gabriel Stanovsky, Luheng He, Ido Dagan, Luke Zettlemoyer
*Crowdsourcing Question-Answer Meaning Representations arXiv, Nov, 2017.
- Oct 5 Semantic Proto-Roles and Universal Decomposition
Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. 2015.
*Semantic Proto-roles In Transactions of the Association of Computational Linguistics.
Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira,
Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme.
2016.
Universal Decompositional Semantics on Universal Dependencies.
In Empirical Methods in Natural Language Processing (EMNLP 2016), Austin, TX.
Sheng Zhang; Rachel Rudinger; Benjamin Van Durme
*An Evaluation of PredPatt and Open IE via Stage 1 Semantic Role Labeling IWCS 2017.
Module 3: Drawing Inferences
- Oct 5 Review Assignment 2 and Term Project Proposal Discussion,
- Oct 5 Predicate Logic,
B&B Chap 1 and 2 and
A good references is:
Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig, Pearson Education, 2003,
ISrBN:0137903952, Chap 14 and 15
(see Canvas for Nils Nilsson, Chaps 13 and 15a, Principles of Artificial Intelligence)
Symbolic Logic: A First Course, Gary Hardegree, UMASS,
- Oct 12 Predicate Logic, Event Variables Assignment 3: Predicate Logic, due Oct 26
Davidson D. 1967. "The Logical Form of Action Sentences,"
Reprinted in Davidson, D: Essays on Actions and Events, Oxford University Press(1980)
Events, Stanford Encyclopedia of Philosophy
Parsons T. 1990 Events in Semantics of English . MIT Press, Boston
Casati, R., and Varzi, A., editors. Events, Dartmouth, Aldershot, 1996.
the introduction
Interlude
- Oct 19 No class, two talks instead, LING 4pm Monday and the Boulderado ICMI conference, 9am, Wednesday, Thursday or Friday
Monday, October 15,
4:00-5:30, Hellems 237,
Claire Bonial Computational and Information Sciences Directorate, Army Research Laboratory (ARL)
Event semantics in text constructions, vision, and human-robot dialogue
The 20th ACM International Conference on Multimodal Interaction
- TBD Deep Learning approaches to WSD and SRL James Gung
- Oct 26 Topic Models Michael Paul
- Oct 26 Modal Logic Facundo Carreiro
- Nov 2 More on Vector Representations Chelsea Chandler and Annebeth Buis
Background Reading:
Manaal Faruqui, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard Hovy, Noah A. Smith. Carnegie Mellon University
*Retrofitting Word Vectors to Semantic Lexicons, NAACL 2015, Best Student Paper Award.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner,
Christopher Clark, Kenton Lee, Luke Zettlemoyer.
Deep contextualized word representations NAACL 2018, New Orleans, LA, June, 2018.
Sascha Rothe; Hinrich Schuetze, (2017),
AutoExtend: Combining Word Embeddings with Semantic Resources
Computational Linguistics, Volume 43, Issue 3 - September 2017
Back to Drawing Inferences
- Nov 2 Term Project Proposals discussionn - due Nov 2, schedule student presentations
- Nov 2 Assignment 4: due Nov 16 Healing the SICK Inferences
- Nov 2 Natural Language Inference
AL Kalouli, L Real, V de Paiva, 2017,
Textual Inference: getting logic from humans,
12th International Conference on Computational Semantics, IWCS 2017, Short Paper.
Aikaterini-Lida Kalouli, Livy Real, Valeria de Paiva. 2017,
Correcting Contradictions. Proceedings of the Computing Natural Language Inference (CONLI) Workshop, 19 September 2017. Held in Montpellier, France.
Aikaterini-Lida Kalouli, Livy Real, Valeria de Paiva. 2018, WordNet for " Easy " Textual Inferences. Proceedings of the Globalex Workshop, associated with LREC 2018, 08 May 2018. Miyazaki, Japan.
- Background Reading
L.K. Schubert,
Semantic representation, 29th AAAI Conference (AAAI15), Jan. 25-30, 2015, Austin, TX.
Omri Abend; Ari Rappoport,
The State of the Art in Semantic Representation, In the Proceedings of ACL 2017, Vancouver, BC, August, 2017
>/li>
- Nov 9 Conbinatory Categorial Grammar and Boxer, the Groningen Meaning Bank
OpenCCG: The OpenNLP CCG Library and the new github site
Steedman, Mark and Jason Baldridge,
Combinatory Categorial Grammar (2011), in R. Borsley and K. Borjars (eds.) Non-Transformational Syntax , 181-224, Blackwell.
Groningen Meaning Bank
Johannes Bjerva; Johan Bos; Hessel Haagsma, (2016),
The Meaning Factory at SemEval-2016 Task 8: Producing AMRs with Boxer In the Proceedings of SemEval 2016, pp. 1179-1184.
- Nov 9 Metaphor Kevin Stowe
- Nov 16 Different Approaches to Inference (VerbNet + Generative Lexicon subevents; RED; Universal Decomposition)
James Pustejovsky, The Syntax of Event Structure, Cognition,
Volume 41, Issues 1-3, December 1991, Pages 47-81
Rei Ikuta, William F. Styler IV, Mariah Hamang, Tim O'Gorman, and Martha Palmer (2014)
Challenges of Adding Causation to Richer Event Descriptions, In proceedings of the 2nd Events Workshop, held in conjunction with ACL 2014, Baltimore, MD.
Tim O'Gorman, Kristin Wright-Bettner, Martha Palmer, (2016)
Richer Event Description: Integrating event coreference with temporal, causal and briding annotation, Computing News Storylines Workshop at EMNLP 2016, pp. 47-56, Austin, TX, October, 2016.
Qiang Ning and Zhili Feng and Hao Wu and Dan Roth, (2018),
Joint Reasoning for Temporal and Causal Relations, ACL 2018
Qiang Ning and Hao Wu and Dan Roth, (2018)
A Multi-Axis Annotation Scheme for Event Temporal Relation,s ACL 2018
Qiang Ning and Hao Wu and Haoruo Peng and Dan Roth, (2018),
Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource NAACL 2018
- Nov 16 Discussion of Assignment 4, Natural Language Inference
Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Ben Van Durme, Chris Callison-Burch
*Adding Semantics to Data-driven Paraphrasing In the Proceedings of ACL 2015, pp. 1512-1522
Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van
Durme. 2017.
Inference is Everything: Recasting Semantic Resources
into a Unified Evaluation Framework. In The Proceedings of the 8th
International Conference on Natural Language Processing (IJCNLP).
Module 5: Future Directions
- Nov 23 Thanksgiving Break
- Nov 30, Dec 7 Student Presentations
Possible Papers:
- Marco Baroni; Roberto Zamparelli
Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space In the Proceedings of EMNLP 2010, pp. 1183-1193
An interesting related paper. Pay special attention to Section 4 on the background to existing compositional methods.
- Laura Rimell; Jean Maillard; Tamara Polajnar; Stephen Clark
RELPRON: A Relative Clause Evaluation Data Set for Compositional Distributional Semantics,
Computational Linguistics, Volume 42, Issue 4 - December 2016
Lifu Huang, Heng Ji, Kyunghyun Cho, Clare R. Voss,
*Zero-Shot Transfer Learning for Event Extraction
arXiv, July, 2017.
Lanbo She; Joyce Chai (2017)
*Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication
ACL 2017 Vancouver, BC, August, 2017.
- Dec 14 No Class, Reading Day
Possbile Additional Papers on Events:
The NAACL and
ACL Events Workshops
James Pustejovsky; Marc Verhagen, 2009, SemEval-2010
Task 13: Evaluating Events, Time Expressions, and Temporal Relations
(TempEval-2) In the Proceedings of the Workshop on Semantic
Evaluations: Recent Achievements and Future Directions (SEW-2009)
held with NAACL-2009, Boulder, CO.
Rei Ikuta and Martha Palmer, (2014)
Challenges of Adding Causation to Richer Event Descriptions,
In the
Proceedings of the 2nd Events Workshop, held in conjunction with ACL 2014, Baltimore, MD.
McClosky, D., Surdeanu, M., & Manning, C. D. (2011).
Event extraction as dependency parsing.
In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:
Human Language Technologies-Volume 1 (pp. 1626-1635).
Advanced topics: possible term projects and/or post-class readings for interest:
Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2017.
Pocket Knowledge Base Population. In The Proceedings of the Annual Meeting of
the Association for Computational Linguistics (ACL 2017), Vancouver, BC.
Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017.
Ordinal Common-sense Inference. Transactions of the Association for
Computational Linguistics, 5:379$(G!9(B395.
Ellie Pavlick and Chris Callison-Burch.
Most babies are little and most problems are huge: Compositional Entailment in Adjective Nouns
ACL 2016, Berlin, Germany, August, 2016.
Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2013.
Concreteness ratings for 40 thousand
generally known English word lemmas. Behavior research methods, pages 1-8.
Felix Hill and Anna Korhonen. 2014.
Concreteness and subjectivity as dimensions of lexical meaning.
In the Proceedings of ACL 2014
David R. Dowty, 1986,
The effects of aspectual class on the temporal structure of discourse: semantics or pragmatics? Linguistics and Philosophy,
February 1986, Volume 9, Issue 1, pp 37-61
Machine Learning Background
Machine Learning links:
Deep Learning Summer School, Montreal 2016
32nd International Conference on Machine Learning, Lille, 2015
Christopher Manning's Videos
Language Vectors
Deep Learning
Yoav Goldberg, primer and tutorial
Primer
T1: Practical Neural Networks for NLP: From Theory to Code
Machine Learning Papers
Background in Ontologies
SUMO
CYC
Description Logic, including
CLASSIC and
OWL