Ling7800/CSCI 7000: Computational Lexical Semantics
Time and Location: Tue/Thur, 11:00 - 12:15, ECCR 150
Assessment: Four homeworks, one Paper presentation, and a term project.
Office Hours: Martha Palmer, Monday/Tuesday 2-3, Hellems 295
Semantic Role Labeling (eBook),
Martha Palmer, Daniel Gildea, Nianwen Xue,
Synthesis Lectures on Human
Language Technologies ,
ed., Graeme Hirst, Morgan & Claypool, 2010. ISBN: 9781598298321
available on line on campus through Chinook
Representation and Inference for Natural Language.
A First Course in Computational Semantics.
Patrick Blackburn and Johan Bos, 2005,
CSLI Publications. ISBN: 1-57586-496-7
selected chapters, available from the CU bookstore and D2L
Lexical semantics is becoming an increasingly
important part of Natural Language Processing (NLP), as the field is
beginning to address semantics at a large scale. This graduate seminar
covers key issues in computational lexical
semantics. We start with an introduction to theoretical models of
lexical semantics and events, considering both their adequacy as
linguistic models and their place in NLP. We focus particularly
on computational lexical resources such as PropBank, VerbNet, FrameNet and the
Generative Lexicon, and examine their strengths and limitations with
respect to NLP applications. We will introduce apporoaches to
developing automatic classifiers that are intended to make use of
these resources and to offer richer representations of sentences in
context. These techniques can be fully supervised (requiring
hand-labeled training data), semi-supervised, or unsupervised
(learning lexical information from unlabeled text).j
We will also discuss the impact of Word Embeddings as an
approximmation of semantic similarity and the resulting implications for future research directions.
Suggested Schedule and Readings
Introduction and Module 1: the Lexical Semantics of Verbs - Chap 1
- Jan 16 Course Overview and Natural Language Processing, the Pundit case study
Palmer, Martha, Carl Weir, Rebecca Passonneau, and Tim Finin.
"The Kernel Text Understanding System."
Artificial Intelligence 63: 17-68: Special Issue on Text Understanding.
- Jan 18 Thematic Roles in Linguistics,
Assignment 1: Exercises 1, 2 and 3, p. 19, SRL book, Due Jan 30
- Background reading for Assignment:
Fillmore, C. J. 1968 "The Case for Case" in E. Bach and R.T. Harms, eds.
Universals in Linguistic Theory, 1-88. New York: Holt, Rinehart and Winston. Section 3.
Jackendoff, R.S. 1976 Towards an Explanatory Semantic Representation,
Linguistic Inquiry, 7:1, pp. 89-150.
Dowty D.R 1991 Thematic Proto-Roles and Argument Selection.
Language 67: 547-619 sections 1-7 Paper
Levin, B. English Verb Classes: A Preliminary Classification Introduction,
MIT Press, pp. 1-23, 1990., Paper
Module 2: Available Computational Lexicons - Chap 2
- Jan 23 Word Senses, WordNet and the OntoNotes Groupings
Palmer, M., Dang, H. and Fellbaum, C, 2007, Making Fine-grained and Coarse-grained sense distinctions,both manually and automatically,
Journal of Natural Language Engineering,13:2, 137-163.
George A. Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller, 1993,
Introduction to WordNet: An On-line Lexical Database, 5 Papers on WordNet availalbe from the WordNet web site.
Background Reading for Sense Distinctions:
Edmonds, P. and Hirst, G., Near-Synonymy and Lexical Choice,
Computational Linguistics June, 2002, Vol. 28, No. 2, Pages 105-144,
Atkins, S., Fillmore, C. J., Johnson, C. R.,
Lexicographic Relevance: Selecting Information from Corpus Evidence,
International Journal of Lexicography, Vol. 16 No. 3, Oxford University Press, 2003,
Hanks, P. and Pustejovsky, J., A Pattern Dictionary for Natual Language Processing, Revue francaise de linguistique appliquie
2005/2 (Vol. X), CAIRN, INFO, 2005. Paper
- Jan 25 WSD as a Machine Learning Task/Preposition SuperSenses/The Generative Lexicon
Nathan Schneider, Jena D. Hwang, Vivek Srikumar, Meredith Green, Abhijit Suresh, Kathryn Conger, Tim O$A!/(BGorman, and Martha Palmer (2016).
A corpus of preposition supersenses. LAW, at ACL 2016, Berlin, Germany.
Pustejovsky, James, 1991, The Generative Lexicon, ComputationaI Linguistics, Volume 17, Number 4, December. Paper
- Jan 30 Wrap up The Generative Lexicon/Review Ass 1
- Feb 1 PropBank,
Assignment 2: Exercises 2,3,4 p. 29, SRL book, Due Feb 20
Martha Palmer, Dan Gildea, Paul Kingsbury, 2005,
The Proposition Bank: An Annotated Corpus of Semantic Roles,
Computational Linguistics, 31:1 , pp. 71-105.
- Feb 6 VerbNet
Kipper, Karin, Anna Korhonen, Neville Ryant, Martha Palmer. "A Large-scale Classification of English Verbs." Language Resources and Evaluation Journal,42(1). Springer Netherland: 2008. pp. 21-40.
- Feb 8 FrameNet
Fillmore et al 2001
"Building a large lexical databank which provides deep
Proceedings of the 15th Pacific Asia Conference on Language, Information and Computation. Eds. Benjamin Tsou, and Olivia Kwong. Hong Kong 2001.
Fillmore, Charles J., Christopher R. Johnson, and Miriam R.L. Petruck. 2002.
Background to FrameNet.
International Journal of Lexicography, 1'6(3):2435
- Feb 13 Semantic Proto-Roles
Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. 2015.
*Semantic Proto-roles In Transactions of the Association of Computational Linguistics.
- Access to Computational Lexicons:
- Feb 15 Abstract Meaning Representations (AMRs) and After AMRs, QAMR
Laura Banarescu; Claire Bonial; Shu Cai; Madalina Georgescu; Kira Griffitt; Ulf Hermjakob; Kevin Knight; Philipp Koehn; Martha Palmer; Nathan Schneider (2013)
Abstract Meaning Representation for Sembanking Linguistics Annotation Workshop and Interoperability with Discourse, held with ACL 2013, Sofia, Bulgaria.
Nianwen Xue, Ondrej Bojar, Jan Hajic, Martha Palmer, Zdenka Uresova, Xiuhong Zhang
Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech LREC 2014.
Julian Michael, Gabriel Stanovsky, Luheng He, Ido Dagan, Luke Zettlemoyer
*Crowdsourcing Question-Answer Meaning Representations arXiv, Nov, 2017.
Module 3: Beyond shallow semantics
- Feb 20, 22 Predicate Logic, Susan Brown
B&B Chap 1 and 2 and
Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig, Pearson Education, 2003,
ISrBN:0137903952, Chap 14 and 15
Symbolic Logic: A First Course, Gary Hardegree, UMASS,
- March 6 Wrap up Predicate Logic, Event Variables Assignment 3: Predicate Logic, due March 22
Davidson D. 1967. "The Logical Form of Action Sentences,"
Reprinted in Davidson, D: Essays on Actions and Events, Oxford University Press(1980)
Events, Stanford Encyclopedia of Philosophy
Parsons T. 1990 Events in Semantics of English . MIT Press, Boston
Casati, R., and Varzi, A., editors. Events, Dartmouth, Aldershot, 1996.
Module 4: More Empirical Approaches
- March 1 Deep Learning approaches to WSD and SRL James Gung
- March 8 Word Embeddings, Term Project Proposals are due Scott Denning
Manaal Faruqui, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard Hovy, Noah A. Smith. Carnegie Mellon University
*Retrofitting Word Vectors to Semantic Lexicons, NAACL 2015, Best Student Paper Award.
- March 13 Event Force Dynamics Bill Croft
- March 15 Student Presentations
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner,
Christopher Clark, Kenton Lee, Luke Zettlemoyer.
Deep contextualized word representations NAACL 2018, New Orleans, LA, June, 2018.
Sascha Rothe; Hinrich Schuetze, (2017),
AutoExtend: Combining Word Embeddings with Semantic Resources
Computational Linguistics, Volume 43, Issue 3 - September 2017
- March 20 VerbNet and Generative Lexicon Event Structure Susan Brown
James Pustejovsky, The Syntax of Event Structure, Cognition,
Volume 41, Issues 1-3, December 1991, Pages 47-81
- March 22 Ontologies and Event Ontologies Susan Brown
Assignment 4: due Apr 10 Compare and contrast these two papers. Critique one of them.
Semantic representation, 29th AAAI Conference (AAAI15), Jan. 25-30, 2015, Austin, TX.
Omri Abend; Ari Rappoport,
The State of the Art in Semantic Representation, In the Proceedings of ACL 2017, Vancouver, BC, August, 2017
Module 5: Future Directions
- March 27, 29 Spring Break
- April 3 PredPatt compared to Automatic Semantic Role Labeling, Chapter 3
Sheng Zhang; Rachel Rudinger; Benjamin Van Durme
*An Evaluation of PredPatt and Open IE via Stage 1 Semantic Role Labeling IWCS 2017.
- April 5 Student Presentations
Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Ben Van Durme, Chris Callison-Burch
*Adding Semantics to Data-driven Paraphrasing In the Proceedings of ACL 2015, pp. 1512-1522
- Marco Baroni; Roberto Zamparelli
Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space In the Proceedings of EMNLP 2010, pp. 1183-1193
An interesting related paper. Pay special attention to Section 4 on the background to existing compositional methods.
- Laura Rimell; Jean Maillard; Tamara Polajnar; Stephen Clark
RELPRON: A Relative Clause Evaluation Data Set for Compositional Distributional Semantics,
Computational Linguistics, Volume 42, Issue 4 - December 2016
- April 10 Student Presentation and Automatic Event Extraction, THYME/RED,
Lifu Huang, Heng Ji, Kyunghyun Cho, Clare R. Voss,
*Zero-Shot Transfer Learning for Event Extraction
arXiv, July, 2017.
Danilo Croce; Alessandro Moschitti; Roberto Basili; Martha Palmer
Verb Classification using Distributional Similarity in Syntactic and Semantic Structures , In the Proceedings of ACL 2012, pp. 263-272
- April 12 Recovery of Implicit Arguments Tim O'Gorman
- April 17 Grounded Verb Semantics
Lanbo She; Joyce Chai (2017)
*Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication
ACL 2017 Vancouver, BC, August, 2017.
- April 19 Comparing Semantic Representations - Discussion
- April 24 Combinatory Categorial Grammar
Steedman, Mark and Jason Baldridge,
Combinatory Categorial Grammar (2011), in R. Borsley and K. Borjars (eds.) Non-Transformational Syntax , 181-224, Blackwell.
- April 26 Logic plus distributional models
Islam Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk,and Raymond J. Mooney.
*Representing meaning with a combination of logical and distributional models.
Computational Linguistics 42(4), special issue on formal distributional semantics.
Module 6: Term Project Paper Presentations
- May 1 Student Presentations related to VerbNet
- May 3 Student Presentations related to PPDB and ELMO
Possbile Additional Papers on Events:
The NAACL and
ACL Events Workshops
James Pustejovsky; Marc Verhagen, 2009, SemEval-2010
Task 13: Evaluating Events, Time Expressions, and Temporal Relations
(TempEval-2) In the Proceedings of the Workshop on Semantic
Evaluations: Recent Achievements and Future Directions (SEW-2009)
held with NAACL-2009, Boulder, CO.
Rei Ikuta and Martha Palmer, (2014)
Challenges of Adding Causation to Richer Event Descriptions,
Proceedings of the 2nd Events Workshop, held in conjunction with ACL 2014, Baltimore, MD.
McClosky, D., Surdeanu, M., & Manning, C. D. (2011).
Event extraction as dependency parsing.
In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:
Human Language Technologies-Volume 1 (pp. 1626-1635).
Advanced topics: possible term projects and/or post-class readings for interest:
Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira,
Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme.
Universal Decompositional Semantics on Universal Dependencies.
In Empirical Methods in Natural Language Processing (EMNLP 2016), Austin, TX.
Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2017.
Pocket Knowledge Base Population. In The Proceedings of the Annual Meeting of
the Association for Computational Linguistics (ACL 2017), Vancouver, BC.
Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van
Inference is Everything: Recasting Semantic Resources
into a Unified Evaluation Framework. In The Proceedings of the 8th
International Conference on Natural Language Processing (IJCNLP).
Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017.
Ordinal Common-sense Inference. Transactions of the Association for
Computational Linguistics, 5:379$(G!9(B395.
Ellie Pavlick and Chris Callison-Burch.
Most babies are little and most problems are huge: Compositional Entailment in Adjective Nouns
ACL 2016, Berlin, Germany, August, 2016.
Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2013.
Concreteness ratings for 40 thousand
generally known English word lemmas. Behavior research methods, pages 1-8.
Felix Hill and Anna Korhonen. 2014.
Concreteness and subjectivity as dimensions of lexical meaning.
In the Proceedings of ACL 2014
David R. Dowty, 1986,
The effects of aspectual class on the temporal structure of discourse: semantics or pragmatics? Linguistics and Philosophy,
February 1986, Volume 9, Issue 1, pp 37-61
Machine Learning Background
Machine Learning links:
Deep Learning Summer School, Montreal 2016
32nd International Conference on Machine Learning, Lille, 2015
Christopher Manning's Videos
Yoav Goldberg, primer and tutorial
T1: Practical Neural Networks for NLP: From Theory to Code
Machine Learning Papers
Background in Ontologies
Description Logic, including