The New Testament connects between truth with godliness, and error with immorality. Paul speaks in Titus 1:1 about the knowledge of the truth that leads to godliness (cf. 1 Timothy 6:3), and in 2 Timothy 2:16 about irreverent babble that leads people into more and more ungodliness (cf. 1 Timothy 6:4-5). Although we should never be glad about it, the truth is that we are not surprised when a false teacher is further compromised by immoral behaviour. As G. K. Chesterton once said "heresy always affects morality, if it's heretical enough."
Heresy, however, does not only lead to sin, it is sin. Believing heresy is wrong not only mentally but also morally. Choosing to believe it is an act of the mind, heart and will that is against God and his Word. Heretics tell lies about God because they do not want to tell the truth. Of course there will always be some who believe error that have never been exposed to the biblical gospel. For others there will have been a choice exercised, rejecting one thing and embracing another.
Paul's reminder to the Corinthians about the gospel that he had preached to them, which they had believed, and on which they had taken their stand, also included the admonition to hold fast to the gospel (1 Cor. 15:1-2). This ongoing act was a vital and necessary part of Christian obedience. It was orthopraxy in action. Not to hold fast would be an act of disobedience. Likewise, John admonishes his readers to keep themselves from idols (1 John 5:21). I think that in the letter these idols are love of the world and false teaching. Embracing either of these would be a substitute for the true God. Embracing them would be to commit the sin of idolatry.
If we realise that the deliberate choice of heresy is itself an immoral act perhaps we will be less impressed by the apparent godliness of heretics. Heresy can come with all the trappings of austerity, self-denial, humility, discipline, and the keeping of rules. These, however, can gloss over the pride that refuses to submit to the truth, and the arrogance that dismisses biblical doctrine.