Recurrent Network

Tap it to read a sentence one word at a time. Try the LSTM style in the Splash tab.

A recurrent network, or RNN, reads a sequence, like the words of a sentence, one piece at a time. It keeps a memory called the hidden state: after each word, the state loops back in with the next word, so what it knows builds up as it reads. An LSTM (long short-term memory) adds gates that decide what to forget, what to take in and what to pass on.

Tap it: THE, CAT and SAT rise into the cell one at a time. The cell flashes, and the glowing orb, the hidden state, takes on the word's color mixed with what it carried and runs around the loop. In the Splash tab, pick the LSTM style to see its three gates open and shut like shutters, or a 3D model. This is a scripted sequence, not a trained sentence reader.

The diagram shows the idea of passing a hidden state through a sequence. Its colors and LSTM shutters are scripted, and it has no learned recurrent weights or numerical cell state.

1. Simplified

   An RNN carries a hidden state from one sequence item to the next.¹

   The diagram loops an orb between THE, CAT, and SAT, matching the flow of a recurrent state.

2. Correct

   An LSTM uses forget, input, and output gates.²

   The three shutter labels are F, I, and O, matching those gate roles.

3. Simplified

   The splash illustrates gate operation without reading a sentence.¹²

   The colors and shutters are stored animation values. The About text explicitly calls this a scripted sequence.

1. 1Recurrent neural networks, Dive into Deep Learning authors
2. 2Long short-term memory, Dive into Deep Learning authors

Simplified

THE CAT SAT, the orb colors, and the gate openings repeat a stored sequence. The LSTM view has no numerical cell-state update or sigmoid gate values.

Made in Splashery. Its code is MIT.

This link opens it in the Playground. To put it on your own page, add the element and one script tag. Every option.