Should we only send out requests till we reach 5k connections and remove other non-TG audience from our connection list?

Yes. The first 5k connections are important since you need to generate leads from the niche that you have chosen. Hence, they should only belong to your TG and you should filter the ones out that are not. In addition, the content you post will have a compounding effect if your connections are relevant.