As you know, I work with babies. Lots and lots of babies. I watch them grow from basically fetuses to term infants, and some of them, due to complex medical issues, may be with us much longer. To say there is an accelerated level of growth and neuroplasticity during this stage of development is an understatement. I've seen brain ultrasounds change significantly from month to month. Frankly, it's an amazing spectacle.
What does happen is that some neuropathways are strengthened over time, and some that start out strong, may weaken and eventually be lost. I suspect that some of these memories are, at the very least, consciously lost. Now, what is interesting, is that some memories are present in the subconscious and can affect behaviors for years without the person knowing why.
Just some examples: In the neonatal ICU, we often have to draw blood for various lab tests. The babies are tiny. We aren't poking veins and arteries, but rather we do with a tiny poke to the heel, much like having your finger pricked at a clinic for a blood sugar or cholesterol level. Fill the little capillary tube and send it to the lab. Well, you do this enough times over weeks and months and obviously the brain understands this and associates something on the foot as uncomfortable. It becomes a sensory issue for the person. So, the person walks on their toes, they don't like wearing socks and shoes, etc.
Another example: In the neonatal ICU, there is a lot of medical equipment, and medical equipment have alarms on them for all sorts of things. The baby may be in this environment for several weeks or months. Well, eventually the baby gets better, you send them home to loving parents, the kid becomes a toddler. The parents decide to introduce them to a local fast-food restaurant, and fast-food restaurants often have alarms on them to tell the workers when to pull the French fries out of the fryer, or pull food out of the microwave, etc. The toddler hears the alarms and starts having a meltdown in front of everyone. The parents can't figure it out. The toddler really doesn't know why it's so upsetting (another sensory issue) and can't articulate it, so everyone is upset and confused, and the experience ruined.