Start Date: 06/02/2019
Course Type: Common Course
Course Link: https://www.coursera.org/learn/nlp-sequence-modelsExplore 1600+ online courses from top universities. Join Coursera today to learn data science, programming, business strategy, and more.
This course will teach you how to build models for natural language, audio, and other sequence data. Thanks to deep learning, sequence algorithms are working far better than just two years ago, and this is enabling numerous exciting applications in speech recognition, music synthesis, chatbots, machine translation, natural language understanding, and many others. You will: - Understand how to build and train Recurrent Neural Networks (RNNs), and commonly-used variants such as GRUs and LSTMs. - Be able to apply sequence models to natural language problems, including text synthesis. - Be able to apply sequence models to audio applications, including speech recognition and music synthesis. This is the fifth and final course of the Deep Learning Specialization. deeplearning.ai is also partnering with the NVIDIA Deep Learning Institute (DLI) in Course 5, Sequence Models, to provide a programming assignment on Machine Translation with deep learning. You will have the opportunity to build a deep learning project with cutting-edge, industry-relevant content.
Natural language processing with deep learning is an important combination. Using word vector representations and embedding layers you can train recurrent neural networks with outstanding performances in a wide variety of industries. Examples of applications are sentiment analysis, named entity recognition and machine translation.
This course will teach you how to build models for natural language, audio, and other sequence data.
|Sequence labeling||Most sequence labeling algorithms are probabilistic in nature, relying on statistical inference to find the best sequence. The most common statistical models in use for sequence labeling make a Markov assumption, i.e. that the choice of label for a particular word is directly dependent only on the immediately adjacent labels; hence the set of labels forms a Markov chain. This leads naturally to the hidden Markov model (HMM), one of the most common statistical models used for sequence labeling. Other common models in use are the maximum entropy Markov model and conditional random field.|
|Sequence analysis||In marketing, sequence analysis is often used in analytical customer relationship management applications, such as NPTB models (Next Product to Buy).|
|IBM alignment models||The original work on statistical machine translation at IBM proposed five models, and a model 6 was proposed later. The sequence of the six models can be summarized as:|
|Sequence||An important property of a sequence is "convergence". A sequence converges, it converges to a particular value known as the "limit". If a sequence converges to some limit, then it is convergent. A sequence that does not converge is divergent.|
|Sequence||The first ten terms of this sequence are 0, 1, 1, 2, 3, 5, 8, 13, 21, and 34. A more complicated example of a sequence that is defined recursively is Recaman's sequence. We can define Recaman's sequence by|
|Multiple sequence alignment||Consensus methods attempt to find the optimal multiple sequence alignment given multiple different alignments of the same set of sequences. There are two commonly used consensus methods, M-COFFEE and MergeAlign. M-COFFEE uses multiple sequence alignments generated by seven different methods to generate consensus alignments. MergeAlign is capable of generating consensus alignments from any number of input alignments generated using different models of sequence evolution or different methods of multiple sequence alignment. The default option for MergeAlign is to infer a consensus alignment using alignments generated using 91 different models of protein sequence evolution.|
|Sequence||Formally, a subsequence of the sequence formula_3 is any sequence of the form formula_39, where formula_40 is a strictly increasing sequence of positive integers.|
|Sequence||Normally, the term "infinite sequence" refers to a sequence that is infinite in one direction, and finite in the other—the sequence has a first element, but no final element. Such a sequence is called a singly infinite sequence or a one-sided infinite sequence when disambiguation is necessary. In contrast, a sequence that is infinite in both directions—i.e. that has neither a first nor a final element—is called a bi-infinite sequence, two-way infinite sequence, or doubly infinite sequence. A function from the set Z of "all" integers into a set, such as for instance the sequence of all even integers ( …, −4, −2, 0, 2, 4, 6, 8… ), is bi-infinite. This sequence could be denoted formula_35.|
|Sequence||An ordinal-indexed sequence is a generalization of a sequence. If α is a limit ordinal and "X" is a set, an α-indexed sequence of elements of "X" is a function from α to "X". In this terminology an ω-indexed sequence is an ordinary sequence.|
|Sequence||The length of a sequence is defined as the number of terms in the sequence.|
|Main sequence||The amount of fuel available for nuclear fusion is proportional to the mass of the star. Thus, the lifetime of a star on the main sequence can be estimated by comparing it to solar evolutionary models. The Sun has been a main-sequence star for about 4.5 billion years and it will become a red giant in 6.5 billion years, for a total main sequence lifetime of roughly 10 years. Hence:|
|Sequence alignment||Sequence alignments are useful in bioinformatics for identifying sequence similarity, producing phylogenetic trees, and developing homology models of protein structures. However, the biological relevance of sequence alignments is not always clear. Alignments are often assumed to reflect a degree of evolutionary change between sequences descended from a common ancestor; however, it is formally possible that convergent evolution can occur to produce apparent similarity between proteins that are evolutionarily unrelated but perform similar functions and have similar structures.|
|Sequence||Other examples of sequences include ones made up of rational numbers, real numbers, and complex numbers. The sequence (.9, .99, .999, .9999, ...) approaches the number 1. In fact, every real number can be written as the limit of a sequence of rational numbers, e.g. via its decimal expansion. For instance, π is the limit of the sequence (3, 3.1, 3.14, 3.141, 3.1415, ...). A related sequence is the sequence of decimal digits of π, i.e. (3, 1, 4, 1, 5, 9, ...). This sequence does not have any pattern that is easily discernible by eye, unlike the preceding sequence, which is increasing.|
|Sequence||If a sequence converges, then the value it converges to is unique. This value is called the limit of the sequence. The limit of a convergent sequence formula_34 is normally denoted formula_50. If formula_34 is a divergent sequence, then the expression formula_50 is meaningless.|
|Forward–backward algorithm||The term "forward–backward algorithm" is also used to refer to any algorithm belonging to the general class of algorithms that operate on sequence models in a forward–backward manner. In this sense, the descriptions in the remainder of this article refer but to one specific instance of this class.|
|Sequence||Note that we can consider multiple sequences at the same time by using different variables; e.g. formula_6 could be a different sequence than formula_3. We can even consider a sequence of sequences: formula_8 denotes a sequence whose "m"th term is the sequence formula_9.|
|IBM alignment models||IBM alignment models are a sequence of increasingly complex models used in statistical machine translation to train a translation model and an alignment model, starting with lexical translation probabilities and moving to reordering and word duplication. They have underpinned the majority of statistical machine translation systems for almost twenty years. These models offer principled probabilistic formulation and (mostly) tractable inference.|
|Sequence||Informally, a sequence has a limit if the elements of the sequence become closer and closer to some value formula_41 (called the limit of the sequence), and they become and remain "arbitrarily" close to formula_41, meaning that given a real number formula_43 greater than zero, all but a finite number of the elements of the sequence have a distance from formula_41 less than formula_43.|
|Sequence||An alternative to writing the domain of a sequence in the subscript is to indicate the range of values that the index can take by listing its highest and lowest legal values. For example, the notation formula_10 denotes the ten-term sequence of squares formula_11. The limits formula_12 and formula_13 are allowed, but they do not represent valid values for the index, only the supremum or infimum of such values, respectively. For example, the sequence formula_14 is the same as the sequence formula_3, and does not contain an additional term "at infinity". The sequence formula_16 is a bi-infinite sequence, and can also be written as formula_17.|
|Sequence||In this case we say that the sequence diverges, or that it converges to infinity. An example of such a sequence is .|