I think this might qualify as a "historic" discussion. I also think I may have previously posted this. If so, apologies.
Yes, this is something that has puzzled me for a long time, and got more puzzling since we now know that habitable planets are plentiful.
Easy to forget how vast even just our galaxy is and at what time scales it operates... easy to understand why no contact.
When aliens fly past Earth they lock their doors and roll up their windows.And once they get to learn about us they may say "the monkeys from the 3rd planet from that sun arent ready to be contacted yet"