I think the strongest case for Augustine's early bi-sexuality still remains in carefully reading how he writes about women, in general, and, specifically, his most significant heterosexual relationship. I don't see how the passages I quoted in the original post could even remotely be applied to women by Augustine. Augustine simply doesn't describe his relationships with women, on the few occasions he mentions them, with any sense of love, or friendship, or emotional depth. All of these emotions he expresses quite freely towards his male friends.
I'm not saying that every male relationship that Augustine had was a sexual relationship. I'd venture that most weren't and that any affairs that he had were well in his youth.
This lack of feeling expressed towards women, combined with the fact that Carthage seemed to be known as tolerant place for homosexuality, and that Augustine directly connects his adventures to the city of Carthage, I find compelling.
After my first read-through of Confessions, I went back and read the Introduction. I don't like to read Introductions ahead of time because I find they color my reading experience. In the Introduction of the the translation of Confessions which I am reading, I found this:
There was however, a snag to baptism. Anyone who took it seriously knew that post-baptismal sins were unforgivable, if they were big ones, although most pre-baptismal sins would be wiped out by baptism itself. There was then room for calculation. If you delayed baptism and avoided the most unpardonable sins like murder (some writers included adultery) you could be baptized in old age and be destined for heaven in the life to come. If you were baptized too soon, you might damn yourself to hell by sinning badly later. Those who thought hard could see the attractions of remaining a "catechumen": Monica saw them and because she suspected that her son might sow wild oats and be a promiscuous young man, she preferred to keep him as a catechumen. (pp xix, The Confessions, translated by Philip Burton)I found this intriguing because when Augustine writes about the death of his close, male friend, he mentions that after he fell sick, those who were taking care of him baptized him, assuming that he was going to die soon. His friend has a brief rally in his sickness and is coherent enough to talk to Augustine. Augustine relates the baptism in a joking way, assuming that his friend will laugh at the involuntary, unconscious baptism, and is surprised by his friend's response:
He, however, had learnt beforehand of the baptism he had received, and shrank from me as if from an enemy. In a remarkable and sudden burst of plain speaking he warned me that if I wanted to be his friend, I would have to stop talking to him like that. For my part, I was astonished and upset at this, and put all my own feelings on one side until he had recovered and had regained the full vigour of health; then, I thought, I would be able to deal with him as I wished. But he was rescued from my madness, so that in you he might be reserved for my consolation; a few days later, when I was away, the fever struck again, and he died. (pp 70, Confessions 4.3.8)This is intriguing because of the change brought about in his relationship because of the baptism. If baptism was taken so seriously, then it would have natural repercussions in the relationship that Augustine might have had with his friend and the friend's response may not be one merely of annoyance at being mocked, but the realization that the relationship would have to change.
That's speculation, but I think it's fair speculation.
The delaying of baptism, and Augustine's vow of celibacy once he decides to be baptized, may make more sense if we realize that for Augustine the Christian life was an all-or-nothing prospect on the issue of sex. He would have to completely avoid sex of all kinds in order to live in way that he considered faithful. Any sexual pleasure, even within the bonds of marriage, was something he couldn't handle in combination with his faith. All non-pro-creational sex was giving into lust in his eyes, and perhaps, knowing his own proclivities, it was easier to go "cold turkey".