I think our official view of Confession can be seen in the form for the Visitation of the Sick where it says “Then shall the sick person be moved (i.e., advised, prompted) to make a… Confession… if he feel his conscience troubled in any weighty matter.” That is, where Rome makes Confession compulsory for all, we make it permissible for any: not “generally necessary” but profitable. We do not doubt that here can be forgiveness without it. But, as your own experience shows, many people do not feel forgiven, i.e., do not effectively “believe in the forgiveness of sins,” without it. The quite enormous advantage of coming really to believe in forgiveness is well worth the horrors (I agree they are horrors) of a first confession.
Also, there is gain in self-knowledge: Most of [us] have never really faced the facts about ourselves until we uttered them aloud in plain words, calling a spade a spade. I certainly feel I have benefited enormously by the practice. At the same time I think we are quite right not to make it generally obligatory, which would force it on some who are not ready for it and might do harm.
The Collected Letters of C.S. Lewis, Volume III