The standard artificial neuron is stateless: a weighted sum, then a nonlinearity. Whatever history matters has to live elsewhere in the network. This note is an attempt to write down, plainly, what changes if a unit is allowed to remember.
The smallest possible step
The simplest version adds one internal variable that decays and is pushed by input. Nothing here is new; leaky integrators are old. The question is less about the mechanism and more about what representations become cheap once every unit has one.
Even this tiny change raises immediate questions: what timescale λ should take, whether it is learned or fixed, and whether units with different timescales naturally specialise.
What it might cost
Training becomes sequential; credit has to be assigned through time.
Stability is no longer guaranteed by construction.
Interpretability may improve or worsen. It is not obvious which.
Nothing in this note is a result. It is a list of things worth finding out.