Ling7800/CSCI 7000: Computational Lexical Semantics
Fall 2014
Instructors:
Martha Palmer and occasionally James Pustejovsky
Time and Location: Tue/Thur, 11:00 - 12:15, Ketchum 301, and some Fridays, 10:30-12, Hellems 285
Assessment: Five homeworks, one Paper presentation, and a term project.
Office Hours: Martha Palmer, Hellems 295, Tuesday 5-6pm, Thursday 1-2pm
Textbooks:
Semantic Role Labeling (eBook),
Martha Palmer, Daniel Gildea, Nianwen Xue,
Synthesis Lectures on Human
Language Technologies ,
ed., Graeme Hirst, Morgan & Claypool, 2010. ISBN: 9781598298321
available on line on campus through Chinook
Representation and Inference for Natural Language.
A First Course in Computational Semantics.
Patrick Blackburn and Johan Bos, 2005,
CSLI Publications. ISBN: 1-57586-496-7
available from the CU bookstore
Theme
Lexical semantics is becoming an increasingly
important part of Natural Language Processing (NLP), as the field is
beginning to address semantics at a large scale. This introductory
lecture course will cover key issues in computational lexical
semantics. We will start with an introduction to theoretical models of
lexical semantics and events, considering both their adequacy as
linguistic models and their place in NLP. We will focus particularly
on computational lexical resources such as PropBank, VerbNet and the
Generative Lexicon, and examine their strengths and limitations with
respect to NLP applications. We will introduce apporoaches to
developing automatic classifiers that are intended to make use of
these resources and to offer richer representations of sentences in
context. These techniques can be fully supervised (requiring
hand-labeled training data), semi-supervised, or unsupervised
(learning lexical information from unlabeled text).
Several lectures will be shared with a parallel CS course on Computational
Semantics being run by
Professor James Pustejovsky at Brandeis University, who will avail us of
his expertise in semantics, especially with respect to the Generative
Lexicon and Event Representations.
Suggested Schedule and Readings
Introduction and Module 1: the Lexical Semantics of Verbs - Chap 1
- Aug 26 Course Overview and Natural Language Processing, the Pundit case study
Pundit Overview and
Pundit/Kernel Slides
Palmer, Martha, Carl Weir, Rebecca Passonneau, and Tim Finin.
"The Kernel Text Understanding System."
Artificial Intelligence 63: 17-68: Special Issue on Text Understanding.
October, 1993.
- Aug 28, Sep 2, Thematic Roles in Linguistics,
Aug 28 slides,
Sep 2 slides
Assignment 1: Exercises 1, 2 and 3, p. 19, SRL book, Due Sep 11
Background reading for Assignment:
- Fillmore, C. J. 1968 "The Case for Case" in E. Bach and R.T. Harms, eds.
Universals in Linguistic Theory, 1-88. New York: Holt, Rinehart and Winston. Section 3.
Paper
- Jackendoff, R.S. 1976 Towards an Explanatory Semantic Representation,
Linguistic Inquiry, 7:1, pp. 89-150.
Paper
- Dowty D.R 1991 Thematic Proto-Roles and Argument Selection.
Language 67: 547-619 sections 1-7 Paper
- Levin, B. English Verb Classes: A Preliminary Classification Introduction,
MIT Press, pp. 1-23, 1990., Paper
Module 2: Available Computational Lexicons - Chap 2
- Sep 4 Word Senses, WordNet and the OntoNotes Groupings
Sep 4 slides
- Palmer, M., Dang, H. and Fellbaum, C, 2007, Making Fine-grained and Coarse-grained sense distinctions,both manually and automatically,
Journal of Natural Language Engineering,13:2, 137-163.
Paper
- George A. Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine Miller, 1993,
Introduction to WordNet: An On-line Lexical Database, 5 Papers on WordNet availalbe from the WordNet web site.
- Sep 9, 12 TUESDAY, FRIDAY: The Generative Lexicon James Pustejovsky
Sep 9, 12 slides
- Pustejovsky, James, 1991, The Generative Lexicon, ComputationaI Linguistics, Volume 17, Number 4, December. Paper
-
Background Reading for Sense Distinctions:
Edmonds, P. and Hirst, G., Near-Synonymy and Lexical Choice,
Computational Linguistics June, 2002, Vol. 28, No. 2, Pages 105-144,
Paper
- Atkins, S., Fillmore, C. J., Johnson, C. R.,
Lexicographic Relevance: Selecting Information from Corpus Evidence,
International Journal of Lexicography, Vol. 16 No. 3, Oxford University Press, 2003,
Paper
- Hanks, P. and Pustejovsky, J., A Pattern Dictionary for Natual Language Processing, Revue francaise de linguistique appliquie
2005/2 (Vol. X), CAIRN, INFO, 2005. Paper
- Sep 16 VerbNet and PropBank
Slides
Assignment 2: Exercises 1, 2, p. 29, SRL book, Due Sep 25
- Kipper, Karin, Anna Korhonen, Neville Ryant, Martha Palmer. "A Large-scale Classification of English Verbs." Language Resources and Evaluation Journal,42(1). Springer Netherland: 2008. pp. 21-40.
- Martha Palmer, Dan Gildea, Paul Kingsbury, 2005,
The Proposition Bank: An Annotated Corpus of Semantic Roles,
Computational Linguistics, 31:1 , pp. 71-105.
- Sep 18 Natural Language Processing: What we can and can't do
- Sep 23 FrameNet Slides-FrameNet
- Fillmore et al 2001
"Building a large lexical databank which provides deep
semantics",
Proceedings of the 15th Pacific Asia Conference on Language, Information and Computation. Eds. Benjamin Tsou, and Olivia Kwong. Hong Kong 2001.
- Fillmore, Charles J., Christopher R. Johnson, and Miriam R.L. Petruck. 2002.
Background to FrameNet.
International Journal of Lexicography, 16(3):2435
- Access to Computational Lexicons:
WordNet
FrameNet
PropBank
VerbNet
VerbCorner
- Sep 25 Automatic Word Sense Disambiguation,
Slides
Term Project Discussion, review Assignments 1 and 2
Module 3: Semantic representations
- Sep 30, Oct 2 Predicate Logic
September 30 Slides,
October 2 Slides
- Assignment 3, Due Oct 9
B&B Chap 1, and
Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig, Pearson Education, 2003, ISrBN:0137903952, Chap 14 and 15
- Oct 7 Mapping from the Generative Lexicon to VerbNet Joint with Brandeis
Assignment 4 Mapping GL-VN Due Oct 21
- Oct 9 Combinatory Categorial Grammar and Lambda Calculus
B&B, Chap 2
- Oct 14 CCG/Boxer examples Jared Desjardins
Papers on Boxer:
J. Bos (2008) Wide-Coverage Semantic Analysis with Boxer. In, R. Delmonte (eds): Semantics in Text Processing. STEP 2008 Conference Proceedings, pp. 277-286, Research in Computational Semantics, College Publications.
J. Bos (2008): Formal Semantics in the Real World. In
Advances in natural language processing [electronic resource] :
6th International conference, GoTAL 2008, Gothenburg, Sweden, August
25-27, 2008 : proceedings / Bengt Nordstrom, Aarne Ranta, (eds.)
paper
J. Bos(2011):A Survey of Computational Semantics:
Representation, Inference and Knowledge in Wide-Coverage Text
Understanding. Language and Linguistics Compass 5(6): 336366.
paper
- Oct 16 Caused-Motion Constructions in VerbNet Jena Hwang
- Oct 21 Deeper Representations
Rappaport M. and B.Levin 1998 "Building Verb Meanings" in Butt,
Geuder, eds. The Projection of Arguments: Lexical and Compositional
Factors, CSLI Publications Paper
Slides
- Oct 23 Abstract Meaning Representations (AMRs)
Slides
Assignment 5 Comparing Boxer to AMR, Due Oct 30
Module 4: Representations of Events
- Oct 28, 30 Event Semantics
Slides
Davidson D. 1967. "The Logical Form of Action Sentences,"
Reprinted in Davidson, D: Essays on Actions and Events, Oxford University Press
(1980) Paper
Events, Stanford Encyclopedia of Philosophy
Parsons T. 1990 Events in Semantics of English . MIT Press, Boston
Paper
Casati, R., and Varzi, A., editors. Events . Dartmouth, Aldershot, 1996.
the introduction
Possbile Additional Papers:
- The NAACL and
ACL Events Workshops
- James Pustejovsky; Marc Verhagen, 2009, SemEval-2010
Task 13: Evaluating Events, Time Expressions, and Temporal Relations
(TempEval-2) In the Proceedings of the Workshop on Semantic
Evaluations: Recent Achievements and Future Directions (SEW-2009)
held with NAACL-2009, Boulder, CO.
- Nov 4 Richer Event Descriptions Tim O'Gorman
- Nov 6 Student Presentations: Events
James Pustejovsky, The Syntax of Event Structure, Cognition,
Volume 41, Issues 1-3, December 1991, Pages 47-81 MM Discussant: SM
Rei Ikuta and Martha Palmer, (2014)
Challenges of Adding Causation to Richer Event Descriptions,
In the
Proceedings of the 2nd Events Workshop, held in conjunction with ACL 2014, Baltimore, MD. ALaP , Discussant: SA
- Nov 11 Event Extraction Jinying Chen
Also discussion of AMR-Boxer homework
Hogenboom, F., Frasincar, F., Kaymak, U., & de Jong, F. (2011).
An overview of event extraction from text.
In Workshop on Detection, Representation, and Exploitation of Events in the Semantic Web (DeRiVE 2011)
at ISWC 2011 (Vol. 779, pp. 48-57)
McClosky, D., Surdeanu, M., & Manning, C. D. (2011).
Event extraction as dependency parsing.
In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:
Human Language Technologies-Volume 1 (pp. 1626-1635).
Okamoto, M., & Kikuchi, M. (2009). Discovering Volatile Events in Your Neighborhood: Local-Area Topic Extraction from Blog Entries. In Information Retrieval Technology (pp. 181-192).
Hung, S. H., Lin, C. H., & Hong, J. S. (2010). Web mining for event-based commonsense knowledge using lexico-syntactic pattern matching and semantic role labeling. Expert Systems with Applications, 37(1), 341-347.
Advanced topics (post-class readings for interest):
Li, Qi, Heng Ji, and Liang Huang.
Joint Event Extraction via Structured Prediction with Global Features. In ACL (1), pp. 73-82. 2013.
Mirza, Paramita.
Extracting Temporal and Causal Relations between Events. In ACL 2014
- Nov 14 FRIDAY: Events, going forward James Pustejovsky, joint with Brandeis
Also discussion of last homework, Due Dec 2
- Nov 19, 20 Ontologies
Background in Ontologies
SUMO
CYC
Description Logic, including
CLASSIC and
OWL
- Nov 25, 27 Fall Break
- Dec 2 Student Presentations
- Joint Project: Binary Features, RH and DB
Marc Brysbaert, Amy Beth Warriner, and Victor Kuperman. 2013.
Concreteness ratings for 40 thousand
generally known English word lemmas. Behavior research methods, pages 1-8. RH Discussant: MA
Felix Hill and Anna Korhonen. 2014.
Concreteness and subjectivity as dimensions of lexical meaning.
In the Proceedings of ACL 2014 DB Disccusant: EG
- POS tags and SRL for Kotiria, an Amazonian language, MH
Stenzel, Kristine. A Reference Grammar of Kotiria (Wanano). 2013. University of Nebraska Press. Discussant: YB
- Dec 4 Student Presentations
Joint Project: Tense and Aspect, SA and SM
Ogihara, T. (1990). The Semantics of the Progressive and the Perfect in English. Tense and Aspect in English, 338. The paper can be found here. SA, Discussant: ALP
David R. Dowty, 1986,
The effects of aspectual class on the temporal structure of discourse: semantics or pragmatics? Linguistics and Philosophy,
February 1986, Volume 9, Issue 1, pp 37-61
Ye, Y., & Zhang, Z. (2005).
Tense tagging for verbs in cross-lingual context: A case study. In Natural Language ProcessingIJCNLP 2005 (pp. 885-895). Springer Berlin Heidelberg. SM, Discussant: NR
van der Plas, Lonneke, Marianna Apidianaki, and Chenhua Chen.
"Global methods for cross-lingual semantic role and predicate labelling." Proceedings of COLING. 2014. GK, Discussant: DB
- Dec 9 Student Presentations
Lee Becker, Wayne Ward, Sarel van Vuuren, and Martha Palmer.
DISCUSS: A dialogue move taxonomy layered over semantic representations. In IWCS 2011: The 9th International Conference on Computational Semantics, Oxford, England, January 2011. NR, Discussant: MM
Yoav Artzi, Dipanjan Das and Slav Petrov, 2014,
Learning Compact Lexicons for CCG Semantic Parsing.
In the Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Qatar.
EG, Discussant: RH
- Dec 11 Student Presentations
Abdelati Hawwari, Jena D. Hwang, Aous Mansouri and Martha Palmer. 2011.
Classification and Deterministic PropBank Annotation of Predicative Adjectives in Arabic. In Proceedings of the Sixth Joint ISO - ACL SIGSEM Workshop on Interoperable Semantic Annotation. Oxford, UK. January, 2011.
Zaghouani, Wajdi, Mona Diab, Aous Mansouri, Sameer Pradhan and
Martha Palmer.
The Revised Arabic PropBank. Poster in the Proceedings
of the Linguistic Annotation Workshop, held in conjunction
with ACL-2010.. July 15-16, 2010, Uppsala,
Sweden. YB, Discussant: MH
Shumin Wu, Jinho D. Choi, Martha Palmer,
Detecting Cross-lingual Semantic Similarity Using Parallel PropBanks,
Proceedings of the 9th Conference of the Association for Machine Translation in the Americas (AMTA'10), Denver, CO, 2010. (poster) MA, Discussant: GK
Final Exam - Student Project Presentations, Tuesday, Dec 16, 4:00-7:30pm MUEN D 430, ICS Large conference room
Machine Learning Background
Tom M. Mitchell, 2006, Machine Learning Department technical report CMU-ML-06-108, Carnegie Mellon University,
The Discipline of Machine Learning
Machine Learning Resources/Links
Dan Klein's Machine
Learning for Natural Language Processing: New Developments and Challenges
(slides and video)
Michael Collins tutorial on NLP
Introduction to Machine Learning, S V N Vishwanathan
Weka, a collection of machine learning algorithms for data mining tasks.
Orange, Open source data visualization and analysis for novice and experts. Data mining through visual programming or Python scripting.
Videos of Andrew Ng's Stanford ML course
Noah Smith's course titled Language and Statistics,
at CMU