Ling7800/CSCI 7000: Computational Lexical Semantics
Time and Location: Tue/Thur, 11:00 - 12:15, ECCR 150
Assessment: Four homeworks, one Paper presentation, and a term project.
Office Hours: Martha Palmer, Monday/Tuesday 2-3, Hellems 295
Semantic Role Labeling (eBook),
Martha Palmer, Daniel Gildea, Nianwen Xue,
Synthesis Lectures on Human
Language Technologies ,
ed., Graeme Hirst, Morgan & Claypool, 2010. ISBN: 9781598298321
available on line on campus through Chinook
Representation and Inference for Natural Language.
A First Course in Computational Semantics.
Patrick Blackburn and Johan Bos, 2005,
CSLI Publications. ISBN: 1-57586-496-7
selected chapters, available from the CU bookstore and D2L
Lexical semantics is becoming an increasingly
important part of Natural Language Processing (NLP), as the field is
beginning to address semantics at a large scale. This graduate seminar
covers key issues in computational lexical
semantics. We start with an introduction to theoretical models of
lexical semantics and events, considering both their adequacy as
linguistic models and their place in NLP. We focus particularly
on computational lexical resources such as PropBank, VerbNet, FrameNet and the
Generative Lexicon, and examine their strengths and limitations with
respect to NLP applications. We will introduce apporoaches to
developing automatic classifiers that are intended to make use of
these resources and to offer richer representations of sentences in
context. These techniques can be fully supervised (requiring
hand-labeled training data), semi-supervised, or unsupervised
(learning lexical information from unlabeled text).j We will also discuss the impact of Word Embeddings as an approximmation of semantic similarity and the resulting implications for future research directions.
Suggested Schedule and Readings
Introduction and Module 1: the Lexical Semantics of Verbs - Chap 1
- Jan 16 Course Overview and Natural Language Processing, the Pundit case study
Palmer, Martha, Carl Weir, Rebecca Passonneau, and Tim Finin.
"The Kernel Text Understanding System."
Artificial Intelligence 63: 17-68: Special Issue on Text Understanding.
- Jan 18 Thematic Roles in Linguistics,
Assignment 1: Exercises 1, 2 and 3, p. 19, SRL book, Due Jan 30
- Background reading for Assignment:
Fillmore, C. J. 1968 "The Case for Case" in E. Bach and R.T. Harms, eds.
Universals in Linguistic Theory, 1-88. New York: Holt, Rinehart and Winston. Section 3.
Jackendoff, R.S. 1976 Towards an Explanatory Semantic Representation,
Linguistic Inquiry, 7:1, pp. 89-150.
Dowty D.R 1991 Thematic Proto-Roles and Argument Selection.
Language 67: 547-619 sections 1-7 Paper
Levin, B. English Verb Classes: A Preliminary Classification Introduction,
MIT Press, pp. 1-23, 1990., Paper
Module 2: Available Computational Lexicons - Chap 2
- Jan 23 Word Senses, WordNet and the OntoNotes Groupings
Palmer, M., Dang, H. and Fellbaum, C, 2007, Making Fine-grained and Coarse-grained sense distinctions,both manually and automatically,
Journal of Natural Language Engineering,13:2, 137-163.
George A. Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller, 1993,
Introduction to WordNet: An On-line Lexical Database, 5 Papers on WordNet availalbe from the WordNet web site.
Background Reading for Sense Distinctions:
Edmonds, P. and Hirst, G., Near-Synonymy and Lexical Choice,
Computational Linguistics June, 2002, Vol. 28, No. 2, Pages 105-144,
Atkins, S., Fillmore, C. J., Johnson, C. R.,
Lexicographic Relevance: Selecting Information from Corpus Evidence,
International Journal of Lexicography, Vol. 16 No. 3, Oxford University Press, 2003,
Hanks, P. and Pustejovsky, J., A Pattern Dictionary for Natual Language Processing, Revue francaise de linguistique appliquie
2005/2 (Vol. X), CAIRN, INFO, 2005. Paper
- Jan 25 The Generative Lexicon
Pustejovsky, James, 1991, The Generative Lexicon, ComputationaI Linguistics, Volume 17, Number 4, December. Paper
- Jan 30 Review Ass 1
- Feb 1 PropBank,
Assignment 2: Exercises 2,3,4 p. 29, SRL book, Due Feb 20
Martha Palmer, Dan Gildea, Paul Kingsbury, 2005,
The Proposition Bank: An Annotated Corpus of Semantic Roles,
Computational Linguistics, 31:1 , pp. 71-105.
- Feb 6 VerbNet
Kipper, Karin, Anna Korhonen, Neville Ryant, Martha Palmer. "A Large-scale Classification of English Verbs." Language Resources and Evaluation Journal,42(1). Springer Netherland: 2008. pp. 21-40.
- Feb 8 FrameNet
Fillmore et al 2001
"Building a large lexical databank which provides deep
Proceedings of the 15th Pacific Asia Conference on Language, Information and Computation. Eds. Benjamin Tsou, and Olivia Kwong. Hong Kong 2001.
Fillmore, Charles J., Christopher R. Johnson, and Miriam R.L. Petruck. 2002.
Background to FrameNet.
International Journal of Lexicography, 16(3):2435
- Feb 13 Semantic Proto-Roles
Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. 2015.
Semantic Proto-roles. In Transactions of the Association of Computational Linguistics.
- Access to Computational Lexicons:
- Feb 15 Abstract Meaning Representations (AMRs) and After AMRs
Laura Banarescu; Claire Bonial; Shu Cai; Madalina Georgescu; Kira Griffitt; Ulf Hermjakob; Kevin Knight; Philipp Koehn; Martha Palmer; Nathan Schneider (2013)
Abstract Meaning Representation for Sembanking Linguistics Annotation Workshop and Interoperability with Discourse, held with ACL 2013, Sofia, Bulgaria.
- Feb 20 Review Assignment 2, Discuss Term Projects
Module 3: Beyond shallow semantics
- Feb 22, 27 Predicate Logic, Assignment 3: Predicate Logic due March 13
Parsons T. 1990 Events in Semantics of English . MIT Press, Boston
B&B Chap 1 and 2 and
Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig, Pearson Education, 2003,
ISrBN:0137903952, Chap 14 and 15
Symbolic Logic: A First Course, Gary Hardegree, UMASS,
Davidson D. 1967. "The Logical Form of Action Sentences,"
Reprinted in Davidson, D: Essays on Actions and Events, Oxford University Press
Events, Stanford Encyclopedia of Philosophy
Casati, R., and Varzi, A., editors. Events . Dartmouth, Aldershot, 1996.
- March 6 VerbNet and Generative Lexicon Event Structure Susan Brown
James Pustejovsky, The Syntax of Event Structure, Cognition,
Volume 41, Issues 1-3, December 1991, Pages 47-81
- March 8 Ontologies and Event Ontologies Susan Brown, Term Project Proposals are due
- March 13 Event Extraction, Assignment 3 is due
- March 15 THYME/RED Tim O'Gorman Assignment 4, due Apr 3 Compare and contrast these two papers. Critique one of them.
Semantic representation, 29th AAAI Conference (AAAI15), Jan. 25-30, 2015, Austin, TX.
Omri Abend; Ari Rappoport,
The State of the Art in Semantic Representation, ACL 2017, Vancouver, BC, August, 2017
Module 4: Machine Learning
- March 20 Automatic Word Sense Disambiguation
- March 22 Automatic Semantic Role Labeling, Chapter 3
- March 27, 29 Spring Break
- April 3 Word Embeddings
- April 5 Deep Learning approaches to WSD and SRL James Gung
- April 10 Grounded Verb Semantics
Lanbo She; Joyce Chai (2017)
Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication
ACL 2017 Vancouver, BC, August, 2017.
- April 12 Logic plus distributional models
Islam Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk,and Raymond J. Mooney.
Representing meaning with a combination of logical and distributional models.
Computational Linguistics 42(4), special issue on formal distributional semantics.
Deep Learning Summer School, Montreal 2016
32nd International Conference on Machine Learning, Lille, 2015
Christopher Manning's Videos
Yoav Goldberg, primer and tutorial
T1: Practical Neural Networks for NLP: From Theory to Code
Module 5: Term Project Paper Presentations
Possbile Additional Papers on Events:
- April 17, 19, 24, 26 Student Presentations
- May 1, 3 Student Presentations
The NAACL and
ACL Events Workshops
James Pustejovsky; Marc Verhagen, 2009, SemEval-2010
Task 13: Evaluating Events, Time Expressions, and Temporal Relations
(TempEval-2) In the Proceedings of the Workshop on Semantic
Evaluations: Recent Achievements and Future Directions (SEW-2009)
held with NAACL-2009, Boulder, CO.
Rei Ikuta and Martha Palmer, (2014)
Challenges of Adding Causation to Richer Event Descriptions,
Proceedings of the 2nd Events Workshop, held in conjunction with ACL 2014, Baltimore, MD.
McClosky, D., Surdeanu, M., & Manning, C. D. (2011).
Event extraction as dependency parsing.
In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:
Human Language Technologies-Volume 1 (pp. 1626-1635).
Advanced topics: possible term projects and/or post-class readings for interest:
Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira,
Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme.
Universal Decompositional Semantics on Universal Dependencies.
In Empirical Methods in Natural Language Processing (EMNLP 2016), Austin, TX.
Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2017.
Pocket Knowledge Base Population. In The Proceedings of the Annual Meeting of
the Association for Computational Linguistics (ACL 2017), Vancouver, BC.
Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van
Inference is Everything: Recasting Semantic Resources
into a Unified Evaluation Framework. In The Proceedings of the 8th
International Conference on Natural Language Processing (IJCNLP).
Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017.
Ordinal Common-sense Inference. Transactions of the Association for
Computational Linguistics, 5:379$(G!9(B395.
Ellie Pavlick and Chris Callison-Burch.
Most babies are little and most problems are huge: Compositional Entailment in Adjective Nouns
ACL 2016, Berlin, Germany, August, 2016.
Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2013.
Concreteness ratings for 40 thousand
generally known English word lemmas. Behavior research methods, pages 1-8.
Felix Hill and Anna Korhonen. 2014.
Concreteness and subjectivity as dimensions of lexical meaning.
In the Proceedings of ACL 2014
David R. Dowty, 1986,
The effects of aspectual class on the temporal structure of discourse: semantics or pragmatics? Linguistics and Philosophy,
February 1986, Volume 9, Issue 1, pp 37-61
Machine Learning Background
Tom M. Mitchell, 2006, Machine Learning Department technical report CMU-ML-06-108, Carnegie Mellon University,
The Discipline of Machine Learning
Machine Learning Resources/Links
Dan Klein's Machine
Learning for Natural Language Processing: New Developments and Challenges
(slides and video)
Michael Collins tutorial on NLP
Introduction to Machine Learning, S V N Vishwanathan
Weka, a collection of machine learning algorithms for data mining tasks.
Orange, Open source data visualization and analysis for novice and experts. Data mining through visual programming or Python scripting.
Videos of Andrew Ng's Stanford ML course
Noah Smith's course titled Language and Statistics,
Background in Ontologies
Description Logic, including