But there is a danger with Chesterton's statement. The danger is that we will form in our minds a narrow and set idea of what that immorality will be. And, based on that assumption, we will expect those moving into theological compromise to be immoral only in that particular way. If we follow the logic of Paul in the pastoral epistles we will expect false theologies to produce ungodly behaviour.
Yet in the history of the church there have been those who clearly and definitely embraced and taught error who were known for their personal moral integrity. In fact men as notorious as Pelagius and Faustus Socinus we respected in just this way. You would expect the opposite to be true wouldn't you? But there is more to it than a simple, straightforward, personal moral failure.
In recent writing on orthopraxy there has been a stress on the outworking of orthodoxy in terms of changed Christian behaviour along the lines of the fruit of the Spirit. Sometimes this has been married with an affirmation that this kind of orthopraxy is in fact what orthodoxy is really all about. What has been neglected, in my estimation, is the stress on orthopraxy at the very point where it connects to orthodoxy. This is the kind of orthopraxy that values guarding the good deposit, of being found trustworthy with the mysteries of God, of rightly handling the word of truth, of keeping the faith, of holding firm to the trustworthy word as taught. This also is included in biblical orthopraxy.
Holding to orthodoxy involves more than affirming certain things to be true. Holding to the gospel includes being found faithful and trustworthy is receiving and passing on the gospel as revealed in Scripture. This is genuine orthopraxy.