My assumption is that you are upstreaming a live audio feed. I might have misunderstood your question.
I'm not sure about your setup, but you are likely to be using some sort of relay server where other peers will connect to downstream your feed. In this case, your main concern will be upload speed and latency and not download speed, since no one will be downloading directly from your local machine.
If low latency is not attainable, you can always delay the transmission by a minute or so.
As for the upload speed, it depends on the quality of the stream, some simple calculations can give you some theoretical requirements:
If it is merely a vocal transmission (i.e., not music) you can opt for lower bitrates without really compromising on the quality too much. You can go as low as CBR 96 or even
ABR (Average Bitrate).
Assuming you opted for 96 kbps, a steady 12 kilobytes per second upload should be enough.