Ling7800/CSCI 7000: Computational Lexical Semantics
Time and Location: Tuesdays/Thursdays, 11 - 12:15, ECCR 1B51
Assessment: Four homeworks, one Paper presentation, and a term project.
Office Hours: Martha Palmer, Tuessday 1-2pm, Friday 4:00-5:pm, Fleming 289
Semantic Role Labeling (eBook),
Martha Palmer, Daniel Gildea, Nianwen Xue,
Synthesis Lectures on Human
Language Technologies ,
ed., Graeme Hirst, Morgan & Claypool, 2010. ISBN: 9781598298321
available on line on campus through Chinook
Representation and Inference for Natural Language.
A First Course in Computational Semantics.
Patrick Blackburn and Johan Bos, 2005,
CSLI Publications. ISBN: 1-57586-496-7
selected chapters, available from the CU bookstore and D2L
Lexical semantics is becoming an increasingly
important part of Natural Language Processing (NLP), as the field is
beginning to address semantics at a large scale. This graduate seminar
covers key issues in computational lexical
semantics. We start with an introduction to theoretical models of
lexical semantics and events, considering both their adequacy as
linguistic models and their place in NLP. We focus particularly
on computational lexical resources such as PropBank, VerbNet, FrameNet and the
Generative Lexicon, and examine their strengths and limitations with
respect to NLP applications. We will introduce apporoaches to
developing automatic classifiers that are intended to make use of
these resources and to offer richer representations of sentences in
context. These techniques can be fully supervised (requiring
hand-labeled training data), semi-supervised, or unsupervised
(learning lexical information from unlabeled text).
We will also discuss the impact of Word Embeddings as an
approximmation of semantic similarity and the resulting implications for future research directions.
Suggested Schedule and Readings
Introduction and Module 1: the Lexical Semantics of Verbs - Chap 1
- Aug 27 Course Overview and Natural Language Processing, the Pundit case study
Palmer, Martha, Carl Weir, Rebecca Passonneau, and Tim Finin.
"The Kernel Text Understanding System."
Artificial Intelligence 63: 17-68: Special Issue on Text Understanding.
- Aug 29 Thematic Roles in Linguistics,
Assignment 1: Exercises 1, 2 and 3, p. 19, SRL book, Due Sep 12
- Background reading for Assignment:
Fillmore, C. J. 1968 "The Case for Case" in E. Bach and R.T. Harms, eds.
Universals in Linguistic Theory, 1-88. New York: Holt, Rinehart and Winston. Section 3.
Jackendoff, R.S. 1976 Towards an Explanatory Semantic Representation,
Linguistic Inquiry, 7:1, pp. 89-150.
Dowty D.R 1991 Thematic Proto-Roles and Argument Selection.
Language 67: 547-619 sections 1-7 Paper
Levin, B. English Verb Classes: A Preliminary Classification Introduction,
MIT Press, pp. 1-23, 1990., Paper
Module 2: Available Computational Lexicons - Chap 2
- Sep 3 PropBank,
Martha Palmer, Dan Gildea, Paul Kingsbury, 2005,
The Proposition Bank: An Annotated Corpus of Semantic Roles,
Computational Linguistics, 31:1 , pp. 71-105.
- Sep 5 Automatic Semantic Role Labeling/Assignment 1 discussion
- Sep 10 State of the art (SOA) Neural Net Approaches to SRL - Skatje Myers,
Luheng He, Kenton Lee, Mike Lewis, Luke Zettlemoyer, Deep Semantic Role Labeling: What Works and What is Next ACL 2017.
Luheng He, Kenton Lee, Omer Levy, Luke Zettlemoyer, Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling ACL 2018.
- Sep 12 VerbNet
Assignment 2: Exercises 2, 3, and 4, p. 29, SRL book, Hint: start with VerbNet and use it to find the right frames in PB and FN, and don't forget to provide the unique sense tag: VN class, FN frame, PB roleset Due Sep 26
Kipper, Karin, Anna Korhonen, Neville Ryant, Martha Palmer. "A Large-scale Classification of English Verbs." Language Resources and Evaluation Journal,42(1). Springer Netherland: 2008. pp. 21-40.
- Sep 17 FrameNet
Fillmore et al 2001
"Building a large lexical databank which provides deep
Proceedings of the 15th Pacific Asia Conference on Language, Information and Computation. Eds. Benjamin Tsou, and Olivia Kwong. Hong Kong 2001.
Fillmore, Charles J., Christopher R. Johnson, and Miriam R.L. Petruck. 2002.
Background to FrameNet.
International Journal of Lexicography, 1'6(3):2435
- Sep 19 The Generative Lexicon
Pustejovsky, James, 1991, The Generative Lexicon, ComputationaI Linguistics, Volume 17, Number 4, December.
Hanks, P. and Pustejovsky, J., A Pattern Dictionary for Natual Language Processing, Revue francaise de linguistique appliquie
2005/2 (Vol. X), CAIRN, INFO, 2005.
- Sep 24 Word Senses, WordNet and the OntoNotes Groupings
Palmer, M., Dang, H. and Fellbaum, C, 2007,
Making Fine-grained and Coarse-grained sense distinctions,both manually and automatically,
Journal of Natural Language Engineering,13:2, 137-163.
George A. Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller, 1993,
Introduction to WordNet: An On-line Lexical Database, 5 Papers on WordNet availalbe from the WordNet web site.
Background Reading for Sense Distinctions:
Edmonds, P. and Hirst, G., Near-Synonymy and Lexical Choice,
Computational Linguistics June, 2002, Vol. 28, No. 2, Pages 105-144,
Atkins, S., Fillmore, C. J., Johnson, C. R.,
Lexicographic Relevance: Selecting Information from Corpus Evidence,
International Journal of Lexicography, Vol. 16 No. 3, Oxford University Press, 2003,
- Access to Computational Lexicons:
- Sep 26 WSD as a Machine Learning Task
Dmitriy Dligach and Martha Palmer.
Good Seed Makes a Good Crop: Accelerating Active Learning Using Language Modeling. In ACL '11: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. June 19 - 24, 2011, Portland, OR.
Danilo Croce; Alessandro Moschitti; Roberto Basili; Martha Palmer, Verb Classification using Distributional Similarity in Syntactic and Semantic Structures, In the Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, ACL-2012, Jeju Island, Korea, July, 2012.
Daisuke Kawahara, Daniel W. Peterson and Martha Palmer. (2014) A Step-wise Usage-based Method for Inducing Polysemy-aware Verb Classes, In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL2014), Baltimore, MD.
IP Klapaftis, S Manandhar, (2008)
Word Sense Induction Using Graphs of Collocations,
Kumar, Sawan Jat, Sharmistha, Saxena, Karan, Talukdar, Partha,
Zero-shot Word Sense Disambiguation using Sense Definition Embeddings
Module 3: Drawing Inferences
- Oct 1 Review Assignment 2 and Term Project Proposal Discussion / Predicate Logic,
Assignment 2 is due, Assignment 3: Predicate Logic, due Oct 10
B&B Chap 1 and 2 and
A good references is:
Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig, Pearson Education, 2003,
ISrBN:0137903952, Chap 14 and 15
(see Canvas for Nils Nilsson, Chaps 13 and 15a, Principles of Artificial Intelligence)
Symbolic Logic: A First Course, Gary Hardegree, UMASS,
- Oct 3 Predicate Logic, Event Variables
Davidson D. 1967. "The Logical Form of Action Sentences,"
Reprinted in Davidson, D: Essays on Actions and Events, Oxford University Press(1980)
Events, Stanford Encyclopedia of Philosophy
Parsons T. 1990 Events in Semantics of English . MIT Press, Boston
Casati, R., and Varzi, A., editors. Events, Dartmouth, Aldershot, 1996.
- Oct 8 Integration of Force Dynamic Representations with VerbNet - Michael Regan, Chris Peverada; University of New Mexico
- Oct 10 Force Dynamic Analysis of Events - Pavlina Peskova Kalm, Meagan Vigus, Sook-kyung Lee; University of New Mexico
- Oct 15 Abstract Meaning Representations (AMRs) and After AMRs
Laura Banarescu; Claire Bonial; Shu Cai; Madalina Georgescu; Kira Griffitt; Ulf Hermjakob; Kevin Knight; Philipp Koehn; Martha Palmer; Nathan Schneider (2013)
Abstract Meaning Representation for Sembanking Linguistics Annotation Workshop and Interoperability with Discourse, held with ACL 2013, Sofia, Bulgaria.
Nianwen Xue, Ondrej Bojar, Jan Hajic, Martha Palmer, Zdenka Uresova, Xiuhong Zhang
Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech LREC 2014.
Assignment 3 is due, Project Proposals are due Oct 24
- Oct 17 State of the art for AMR Parsing - Jon Cai
Chunchuan Lyu, Ivan Titov (2017)
AMR Parsing as Graph Prediction with Latent Alignment ACL 2018.
Sheng Zhang, Xutai Ma, Kevin Duh, Benjamin Van Durme, (2018)
AMR Parsing as Sequence-to-Graph Transduction ACL 2019.
- Oct 22 Natural Language Inference
Assignment 4: Entailments, due Nov 19
AL Kalouli, L Real, V de Paiva, 2017,
Textual Inference: getting logic from humans,
12th International Conference on Computational Semantics, IWCS 2017, Short Paper.
Aikaterini-Lida Kalouli, Livy Real, Valeria de Paiva. 2017,
Correcting Contradictions. Proceedings of the Computing Natural Language Inference (CONLI) Workshop, 19 September 2017. Held in Montpellier, France.
Aikaterini-Lida Kalouli, Livy Real, Valeria de Paiva. 2018, WordNet for " Easy " Textual Inferences. Proceedings of the Globalex Workshop, associated with LREC 2018, 08 May 2018. Miyazaki, Japan.
Aikaterini-Lida Kalouli, Annebeth Buis, Livy Real, Martha Palmer, Valeria dePaiva, (2019)
Explaining Simple Natural Language Inference In the Proceedings of the Linguistic Annotation Workshop, held with ACL, Florence, Italy, August, 2019.
Semantic representation, 29th AAAI Conference (AAAI15), Jan. 25-30, 2015, Austin, TX.
Omri Abend; Ari Rappoport,
The State of the Art in Semantic Representation, In the Proceedings of ACL 2017, Vancouver, BC, August, 2017
Ellie Pavlick, Johan Bos, Malvina Nissim, Charley Beller, Ben Van Durme, Chris Callison-Burch
*Adding Semantics to Data-driven Paraphrasing In the Proceedings of ACL 2015, pp. 1512-1522
Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van
Inference is Everything: Recasting Semantic Resources
into a Unified Evaluation Framework. In The Proceedings of the 8th
International Conference on Natural Language Processing (IJCNLP).
Module 4: And the Rest
- Oct 24 Metaphor
- Oct 29 Elmo and Auto Extend Chelsea Chandler and Annebeth Buis
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner,
Christopher Clark, Kenton Lee, Luke Zettlemoyer.
Deep contextualized word representations NAACL 2018, New Orleans, LA, June, 2018.
Sascha Rothe; Hinrich Schuetze, (2017),
AutoExtend: Combining Word Embeddings with Semantic Resources
Computational Linguistics, Volume 43, Issue 3 - September 2017
Manaal Faruqui, Jesse Dodge, Sujay Kumar Jauhar, Chris Dyer, Eduard Hovy, Noah A. Smith. Carnegie Mellon University
*Retrofitting Word Vectors to Semantic Lexicons, NAACL 2015, Best Student Paper Award.
- Oct 31 Bert and XLNet - Abhidip Bhattacharyya
- Nov 5 More on AMR parsing and MT with AMR's - Dan Gildea
Linfeng Song, Daniel Gildea,
SemBleu: A Robust Metric for AMR Parsing Evaluation ACL 2019.
Linfeng Song, Daniel Gildea, Yue Zhang, Zhiguo Wang, Jinsong Su, Semantic Neural Machine Translation Using AMR , TACL 2019 (presented at ACL)
- Nov 7 Richer Event Descriptions - Michael Regan, UNM
Rei Ikuta, William F. Styler IV, Mariah Hamang, Tim O'Gorman, and Martha Palmer (2014)
Challenges of Adding Causation to Richer Event Descriptions, In proceedings of the 2nd Events Workshop, held in conjunction with ACL 2014, Baltimore, MD.
Tim O'Gorman, Kristin Wright-Bettner, Martha Palmer, (2016)
Richer Event Description: Integrating event coreference with temporal, causal and briding annotation, Computing News Storylines Workshop at EMNLP 2016, pp. 47-56, Austin, TX, October, 2016.
Qiang Ning and Zhili Feng and Hao Wu and Dan Roth, (2018),
Joint Reasoning for Temporal and Causal Relations, ACL 2018
Qiang Ning and Hao Wu and Dan Roth, (2018)
A Multi-Axis Annotation Scheme for Event Temporal Relation,s ACL 2018
Qiang Ning and Hao Wu and Haoruo Peng and Dan Roth, (2018),
Improving Temporal Relation Extraction with a Globally Acquired Statistical Resource NAACL 2018
- Nov 14 Expanding AMR: Grounded Spatial Annotation of a Minecraft Corpus, Julia Bonn
and Intro to CCG, Ghazaleh Kazeminejad
Module 5: Student Presentations
- Nov 19 Student Presentations, CK, JY
Barhom, S., Shwartz, V., Eirew, A., Bugert, M., Reimers, N., & Dagan, I. (2019).
Revisiting joint modeling of cross-document entity and event coreference resolution. ACL 2019.
Haoyue Shi, Jiayuan Mao, Kevin Gimpel, Karen Livescu, (2019), Visually Grounded Neural Syntax Acquisition In ACL 2019.
- Nov 21 Student Presentations, LS, PJ
Xinjie Zhou, Xiaojun Wan, Jianguo Xiao, (2016),
Attention-based LSTM Network for Cross-Lingual Sentiment Classification , In EMNLP 2016.
Joshua Coates, Danushka Bollegala, (2018),
Frustratingly Easy Meta-Embedding: Computing Meta-Embeddings by Averaging Source Word Embeddings Short Paper, In NAACL 2018.
- Nov 26, 28 Thanksgiving Break
- Dec 3 AMR Wrapup and Student Presentation, DR
Nianwen Xue, Ondrej Bojar, Jan Hajic, Martha Palmer, Zdenka Uresova, Xiuhong Zhang, (2014), Not an Interlingua, But Close: Comparison of English AMRs to Chinese and Czech , LREC 2014.
Link to Reification page
- Dec 5 Student Presentations, PD, HL
Aminian, M., Mohammad S. R., and Mona D., (2017),
Transferring Semantic Roles using Translation and Syntactic Information In IJCNLP 2017.
Yoshinari Fujinuma, Jordan Boyd-Graber, Michael J. Paul, (2019),
A Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity, In ACL 2019, Pages:4952 to 4962, Florence, Italy.
- Dec 10 Grounded Verb Semantics for Human-Robot Communication
Lanbo She; Joyce Chai (2017)
*Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication
ACL 2017 Vancouver, BC, August, 2017.
- Dec 12 Semantic Proto-roles, Universal Decompositional Semantics and PredPatt
Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme, (2015).
*Semantic Proto-roles In Transactions of the Association of Computational Linguistics.
Steven White, Drew Reisinger, Keisuke Sakaguchi, Tim Vieira,
Sheng Zhang, Rachel Rudinger, Kyle Rawlins, and Benjamin Van Durme,
Universal Decompositional Semantics on Universal Dependencies.
In Empirical Methods in Natural Language Processing (EMNLP 2016), Austin, TX.
Sheng Zhang; Rachel Rudinger; Benjamin Van Durme
*An Evaluation of PredPatt and Open IE via Stage 1 Semantic Role Labeling IWCS 2017.
- Dec 17 Final Project Presentations, 1:30pm - 4:00pm
- Julian Michael, Gabriel Stanovsky, Luheng He, Ido Dagan, Luke Zettlemoyer
*Crowdsourcing Question-Answer Meaning Representations arXiv, Nov, 2017.
- James Pustejovsky, The Syntax of Event Structure, Cognition,
Volume 41, Issues 1-3, December 1991, Pages 47-81
- Marco Baroni; Roberto Zamparelli
Nouns are Vectors, Adjectives are Matrices: Representing Adjective-Noun Constructions in Semantic Space In the Proceedings of EMNLP 2010, pp. 1183-1193
An interesting related paper. Pay special attention to Section 4 on the background to existing compositional methods.
- Laura Rimell; Jean Maillard; Tamara Polajnar; Stephen Clark
RELPRON: A Relative Clause Evaluation Data Set for Compositional Distributional Semantics,
Computational Linguistics, Volume 42, Issue 4 - December 2016
Lifu Huang, Heng Ji, Kyunghyun Cho, Clare R. Voss,
*Zero-Shot Transfer Learning for Event Extraction
arXiv, July, 2017.
Advanced topics: possible term projects and/or post-class readings for interest:
Travis Wolfe, Mark Dredze, and Benjamin Van Durme. 2017.
Pocket Knowledge Base Population. In The Proceedings of the Annual Meeting of
the Association for Computational Linguistics (ACL 2017), Vancouver, BC.
Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017.
Ordinal Common-sense Inference. Transactions of the Association for
Computational Linguistics, 5:379–395.
Ellie Pavlick and Chris Callison-Burch.
Most babies are little and most problems are huge: Compositional Entailment in Adjective Nouns
ACL 2016, Berlin, Germany, August, 2016.
Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2013.
Concreteness ratings for 40 thousand
generally known English word lemmas. Behavior research methods, pages 1-8.
Felix Hill and Anna Korhonen. 2014.
Concreteness and subjectivity as dimensions of lexical meaning.
In the Proceedings of ACL 2014
David R. Dowty, 1986,
The effects of aspectual class on the temporal structure of discourse: semantics or pragmatics? Linguistics and Philosophy,
February 1986, Volume 9, Issue 1, pp 37-61
Machine Learning Background
Machine Learning links:
Deep Learning Summer School, Montreal 2016
32nd International Conference on Machine Learning, Lille, 2015
Christopher Manning's Videos
Yoav Goldberg, primer and tutorial
T1: Practical Neural Networks for NLP: From Theory to Code
Machine Learning Papers
Background in Ontologies
Description Logic, including