Forward algorithm Python implementation

Objective

There is no connection between the forward algorithm and the forward propagation algorithm in the neural network ... The forward algorithm is the first basic problem of the hidden Markov model in the field of natural language processing.

What is the forward algorithm?

Here is an example of a seaweed to describe what the forward algorithm is. There is a rigorous mathematical derivation of the forward algorithm on the internet, but it is better to feel like seaweed. Many of the examples on the Internet are problematic and have been revised accordingly in this article.
State transition Matrix

Correlation matrix

Initial state sequence: Sunny (0.63), cloudy (0.17), Rainy (0.20)
We want to ask for a probability of observing the sequence {Dry, damp, soggy}.
Specifically how to find this probability, mathematically there is a rigorous derivation process, here is no longer elaborated, here only for the forward algorithm to explain.

Python implementation of forward algorithm

The problem with the forward algorithm seems to be different from the one that the Viterbi algorithm solves, but the routines are actually exactly the same.
You can first use NumPy to add data to Python code

ImportNumPy asNp# Forward AlgorithmTran=Np.array ([[0.5,0.375,0.125],                 [0.25,0.125,0.625],                 [0.25,0.375,0.375]]) Laun=Np.array ([[0.6,0.2,0.15,0.05],                 [0.25,0.25,0.25,0.25],                 [0.05,0.1,0.35,0.5]]) init=Np.array ([0.63,0.17,0.2]) look=Np.array ([0,2,3])# observation SequenceKind=[' Sunny ',' Cloudy ',' Rainy ']

P (first day seaweed state for 0& Sunny) =p (seaweed state for 0| sunny days)P (Sunny days | initial state)
In this way, you can put the first day seaweed status of 0, sunny, cloudy, rainy probability are calculated.
P (the next day seaweed state for 2& sunny) = (p (first day seaweed state for 0| sunny Days)p (Today sunny Day) +p (first day seaweed status for 0| rainy days)p (Today sunny Day) +p (first day of seaweed status for 0| Cloudy) P (Sunny Today | The day before)) *p (Seaweed for 2| Sunny)
This is written in Python code as follows:

=sum*forinzip(temp, tran[:,j])]))*laun[j,look[i]]

The complete code is as follows:

dp =  np.array ([a *  b for  A, b in  zip  ( Init, laun[:, Look[0 ]])] for  i in  range  (1 , 3 ): Temp =  Np.copy (DP) for  J in  range  ( 3 ): dp[j] =  sum  (Np.array ([a *  b for  A, b in  zip  (temp, tran[:,j]) ]) *  laun[j,look[i]]print  (sum  (DP)) 

It can be found that this code is basically exactly the same as the Viterbi algorithm, except that the Viterbi algorithm finds the maximum DP value and then obtains the corresponding maximum possible hidden sequence, depending on the method used. And summing, you can get the possibility of the corresponding marker sequence.

Forward algorithm Python implementation