Cs224d:deep Learning for Natural Language Process

Course Description

Teaching assistants

Peng Qi

Course Notes (updated each week) detailed syllabus

Class Time and location

Spring Quarter (March-june, 2015).
Lecture:monday, Wednesday 11:00-12:15
Location:tbd

Office Hours

Richard:wed 12:45-2:00, Location:tbd
(For and project discussions)
Tas:tbd

Grading Policy

Assignment #1:15%
Assignment #2:15%
Assignment #3:15%
midterm:15%
Final project:40%

Course Discussions

Stanford Students: Piazza (for Stanford students)
Online discussions: Reddit Group (for Non-stanford students)
Our Twitter account: @CS224d

Assignment Details

See the assignment Page (coming soon) for more details about how to hand in your assignments.

Course Project Details

See the Project Page (coming soon) for more details on the course project.

Prerequisites

• Proficiency in Python
  All class assignments'll is in Python (with use numpy). There is a tutorial here for those who aren ' t as familiar with Python. If you had a lot of programming experience but in a different language (e.g. c/c++/matlab/javascript) you'll probably B E fine.

• College calculus, Linear Algebra (e.g. math, math 51)
  You should is comfortable taking derivatives and understanding matrix vector operations and notation.

• Basic Probability and Statistics (e.g. CS 109 or other stats course)
  You should know basics of probabilities, Gaussian distributions, mean, standard deviation, etc.

• Equivalent knowledge of CS229 (machine learning)
  We'll be formulating cost functions, taking derivatives and performing optimization with gradient descent.

Recommended

• Knowledge of natural language processing (cs224n or cs224u)
  We'll discuss a lot of different tasks and you'll appreciate the power of deep learning techniques even more if you kn ow how much work had been do on these tasks and how related models has solved them.

• Convex optimization
  You could find some of the optimization tricks more intuitive with this background.

• Knowledge of convolutional neural networks (cs231n)
  The first problem set would probably is easier for you. We cannot assume you took this class so there would be is to lectures that overlap in content. You can use this time to dive deeper into some aspects.

FAQ

Is this first time this class is offered?

Yes, this is a entirely new class designed to introduce students to deep learning for natural language processing. We'll place a particular emphasis on neural Networks, which is a class of deep learning models that has recently Obtai Ned improvements in many different NLP tasks.

Can I follow along from the outside?

We ' d be happy if you join us! We plan to make the course materials widely available: The assignments, course notes and slides would be avail Able Online. We may provide videos. We won ' t be able to give you course.

Can I Take this course on credit/no cred basis?

Yes. Credit'll is given to those who would has otherwise earned a c-or above.

Can I audit or sit in?

In general we is very open to sitting-in guests if you is a member of the Stanford community (registered student, staff, and/or faculty). Out of courtesy, we would appreciate so you first e-mail us or talk to the instructor after the first class you attend.

Can I work with groups for the Final Project?

Yes, in groups of up to people.

I have a question about the class. What's the best-to-reach the course staff?

Stanford students please use a internal class forum on Piazza So, other students could benefit from your questions and Our answers. If you had a personal matter, email us at the class mailing list [email protected].

Can I Combine the Final Project with another course?

Yes, May. There is a couple of courses concurrently offered with cs224d that is natural choices, such as cs224u (natural Language Understanding, by Prof Chris Potts and Bill Maccartney). If you is taking a related class, please speak to the instructors to receive permission to combine the Final Project Assi Gnments.

Natural language Processing (NLP) is one of the very important technologies of the information age. Understanding complex language utterances is also a crucial part of artificial intelligence. Applications of NLP is everywhere because people communicate most everything in language:web search, advertisement, Emai LS, customer service, language translation, radiology reports, etc. There is a large variety of underlying tasks and machine learning models powering NLP applications. Recently, deep learning approaches has obtained very high performance across many different NLP tasks. These models can often is trained with a single end-to-end model and does not require traditional, task-specific feature Eng Ineering. In this spring quarter course students would learn to implement, train, debug, visualize and invent their own neural networ K Models. The course provides a deep excursion into cutting-edge of the deep learning applied to NLP. The final project would involve training a complex rEcurrent neural network and applying it to a large scale NLP problem. On the model side we'll cover word vector representations, window-based neural networks, recurrent neural networks, long -short-term-memory models, recursive neural networks, convolutional neural networks as well as some very novel models Invo Lving a memory component. Through lectures and programming assignments students would learn the necessary engineering tricks for making neural networ KS work on practical problems.

Course Instructor

Richard Socher

Cs224d:deep Learning for Natural Language Process