1. There is no space trash anywhere near us. Even if a super-intelligent race of extraterrestrials, with a civilization a million years ahead of us, were to exist, they still would have existed in a more primitive state at some point in their past -- perhaps recklessly sending out radio signals, or exploring nearby star systems without much in the way of deliberation. Where is the cosmological fossil record?
2. All life on Earth has descended from a common genetic blueprint; there is no evidence that, even in the earliest days of life on Earth, other attempts at life ever competed with our blueprint. While life evolved in one of the early oceans, why didn't it evolve separately a hundred more times in different parts of the same ocean, or in another ocean altogether? Perhaps one genetic lineage came to predominate over the others, but we have no evidence for this, and a few years, at the least, of isolation could have allowed for the other lineages to thrive in their respective niches.
More to the point, why doesn't life simply emerge at any time, anywhere on Earth? Even if it would get quickly consumed by "our" life, shouldn't it be happening billions of times per year, everywhere? Remember that we are all descendants of a single, microscopic organism; given the vastness of the oceans in contrast to such a small piece of chemistry, you'd think that if its emergence were so simple with the requisite conditions in place, we'd be finding new life all over the Earth -- even if only for a few minutes at a time before it got cannibalized (and who's to say that it wouldn't become a threat to the order of life here? "our" life is quite resilient, so wouldn't other strains also be?). Yes, the chemical composition of the Earth is different from what it was four billion years ago, but we have no evidence for any other life forms ever living here -- ever! On top of that, it took over 400 million years after the Earth cooled before life got started. That's quite a long time, and when it did happen, it only happened once, in the form of a tiny, microscopic cell amid miles and miles of ocean.
3. We haven't been able to recreate life in a laboratory in over fifty years, and still have no idea how it emerged here in the first place. None. A planet with the requisite conditions for life does not necessitate that life will exist on it -- it merely makes life possible there. The actual spark that initiated our evolution still eludes us. What if life occurs once every 300 trillion trillion tries? If the odds of a coin landing on its side are one in a thousand, and a coin actually lands on its side the first time you flip it, that does not mean that you flipped it a thousand times! Perhaps, then, from the complete data set of planets harboring life, Earth is an outlier or minimum for the time it takes for life to evolve, rather than an average.