RNN Formulation & Execution Modes

Visualizing inputs, outputs, and temporal recurrence

Folded Mode
U V W X H O Feeds back to next input Feeds back to next input U V W Xโ‚ hโ‚œโ‚‹โ‚ oโ‚œโ‚‹โ‚ U V W Xโ‚‚ hโ‚œ oโ‚œ U V Xโ‚ƒ hโ‚œโ‚Šโ‚ oโ‚œโ‚Šโ‚

Variable Breakdown

INPUT (xโ‚œ)

--

HIDDEN STATE (hโ‚œ)

Memory of past context

OUTPUT (oโ‚œ)

--

Formulas

hโ‚œ = f(Uยทxโ‚œ + Wยทhโ‚œโ‚‹โ‚ + b) Memory
oโ‚œ = g(Vยทhโ‚œ + c) Prediction