Wsy WS, So WS.S WS.S Ws.s $1 $2 ST 75,X X1 WS,x XT X2 Recurrent neural networks (RNN) can be used as classification models for time series data. Here we have a simple RNN as shown in the figure above, where and St=f1 (W, St-1+Wt), t = 1, 2,..., T y=f2 (WST+ Wo) We assume all offsets are 0 except Wo for the final output layer and we decide the two activation functions to be: f1 (2) = RELU (2) = max (0, 2) and |f2 (2) = sign (2) = { 1, if z≥0 0, if z<0 Note that the RELU (2) can be applied elementwise if z is a vector. Suppose we want to apply this model to classify sentences into different categories (e.g. positive/negative sentiment), we need to encode each word in a sentence into a vector as the input + to the model. One way to do this is to represent the tth word as a column vector of length |VI, where V is the set of the entire vocabulary. The ith element of x is 1 if the word is the ith word in the vocabulary and all other elements are zero. 4. (1) 2 points possible (graded, results hidden) We first explore a simple scenario where our vocabulary contains only 2 words, V = {A, B}. Let s ER² and we set the initial state so and the weights before the last layer as follows: 80 == [8], -1 1 W 8,8 = W 8,2 = Now given 3 training sentences: AA, ABB, BAA Encode each of them into a sequence of vectors. As an example, the sentence AA is encoded as (1) x (¹) = x (¹) = [1,0]7. (To enter the sequence above, type [[1,0], [1,0]].) Now encode the other 2 sentences into (2) and (3). x(2) x(3) == == (x(1) ,₁₁)), where 4. (2) 3 points possible (graded, results hidden) (1) Now compute the final hidden state SSS for each of the three senteces AA, ABB, BAA in this RNN. (Enter [0,0] for ST = [0,0].) 8T = (2) ST == Submit You have used 0 of 5 attempts Save 4. (3) 1 point possible (graded, results hidden) Fixing so, WS,s, Ws, and by only learning the linear classifier in the final layer, can this RNN separate the 3 examples regardless of how they were labeled? Yes No