Quote 34
For these treasures, and everything that he collected in his lovely house, were to be to him means of forgetfulness, modes by which he could escape, for a season, from the fear that seemed to him at times to be almost too great to be borne. Upon the walls of the lonely locked room where he had spent so much of his boyhood, he had hung with his own hands the terrible portrait whose changing features showed him the real degradation of his life, and in front of it had draped the purple-and-gold pall as a curtain. For weeks he would not go there, would forget the hideous painted thing, and get back his light heart, his wonderful joyousness, his passionate absorption in mere existence. Then, suddenly, some night he would creep out of the house, go down to dreadful places near Blue Gate Fields, and stay there, day after day, until he was driven away. On his return he would sit in front of the picture, sometimes loathing it and himself, but filled, at other times, with that pride of individualism that is half the fascination of sin, and smiling with secret pleasure at the misshapen shadow that had to bear the burden that should have been his own. (11.25)
So, Dorian needs all his luxuriant comforts as a means of "forgetfulness" – which means that he does, in fact, feel some guilt and shame over the state of his soul. However, it's still mingled with the "fascination" and "pride" that he feels looking upon the true image of himself; any guilt he might feel is insignificant compared to his desire to keep his secret.
Quote 35
"To cure the soul by means of the senses, and the senses by means of the soul!" How the words rang in his ears! His soul, certainly, was sick to death. Was it true that the senses could cure it? Innocent blood had been spilled. What could atone for that? Ah! for that there was no atonement; but though forgiveness was impossible, forgetfulness was possible still, and he was determined to forget, to stamp the thing out, to crush it as one would crush the adder that had stung one. (16.4)
At this point, Dorian has given up any hope of redemption – he has enough of a sense of morality to know when he has broken the rules, but he doesn't think there's any way to balance the scales of right and wrong again. The best he can do is try to forget his crimes.
Quote 36
Certainly no one looking at Dorian Gray that night could have believed that he had passed through a tragedy as horrible as any tragedy of our age. Those finely shaped fingers could never have clutched a knife for sin, nor those smiling lips have cried out on God and goodness. He himself could not help wondering at the calm of his demeanour, and for a moment felt keenly the terrible pleasure of a double life. (15.1)
Dorian loves the feeling of secret sin that he gleans from his "double life." The knowledge of his terrible crime is his guilty pleasure.