When did Christians Start Baptizing Infants?

"When Polycarp at the trial preceding his martyrdom testifies that he has been serving the Lord for eighty-six years (Mart. Pol., 9), the reference can only be to his membership in the church. Accordingly, his baptism must have taken place in the apostolic age, even prior to the year ad 70. The statement of Justin (Apol. 1:15) that at that time there were many Christians sixty and seventy years old who from the days of their childhood ematheteuthesan to Christo [who had become disciples of Christ] can refer only to members of the church who were baptized as children during the period between ad 80 and 90. We have already mentioned Irenaeus. He testifies that Christ came to save all, “all who by Him are regenerated unto God; babes (infantes), little children, boys, youths and men” (Adv. Haer., II 22:4). In the Church Order of his disciple Hippolytus (ca. 170- c. 235) the baptism of little children is mentioned in so many words. They are to be baptized before the adults, and their parents or some relative are to take their places at the “Amen” and confession of faith by speaking vicariously for them.

"When Tertullian (ca. 155-220) in his Treatise on Baptism directs his polemics against the custom of infant baptism, he certainly is not attacking it as an innovation; even as, later on, Pelagius in his battle against Augustine’s doctrine of original sin had to admit the argument that, after all, infants were baptized too; at least he does not deny the fact. Likewise, Origen (ca. 185- ca. 254) and Cyprian (ca. 200-258) presuppose the baptism of infants: the former in the claim later transmitted to the Middle Ages by Dionysius the Areopagite that the baptism of infants goes back to a tradition given by the Lord to his apostles (Commentary on Romans, 5:9); Cyprian in the well-known instruction given to Bishop Fidus (Ep. 64) not to defer baptism to the eighth day analogous to circumcision. Jeremias is right when he claims that a later introduction of infant baptism would have stirred up a great excitement and thus have left definite traces in the history of the Church. The results of church-historical investigation rather indicate that in the ancient church, precisely as in our modern mission fields, both forms of baptism, adult and infant, have always existed side by side. If that is true, then infant baptism must go back to the apostolic age. The baptism of children must then be included in the baptism of entire families, of which we have examples in the New Testament, even though the children are not specifically mentioned.

Translated from Hermann Sasse, Letters to Lutheran Pastors vol. V, 1949. Found at Mercy Journey's blog.