Ghost Nodes: A Trick That Makes Neural Networks Learn Smarter
When we train deep neural networks, they often get stuck — not in a bad result, but in a “flat region” of the loss landscape. The authors of this paper introduce ghost nodes: extra, fake output nodes that aren’t real classes, but help the model explore better paths during training. Imagine you’re rolling a ball into a valley. Sometimes the valley floor is flat and the ball slows down. Ghost nodes are like adding new dimensions to the terrain — giving the ball more freedom to move and find a better path. ...