Nah, you don't actually want to destroy this thing. Your SPIKED GOLDEN KNUCKLES aren't even on you right now, and you're not about to start punching through rock with your bare fists like a complete idiot (even if you do enjoy playing the part most of the time).
Besides, lots of people have tried breaking it open before — including you. If it were possible, someone would have figured it out a long while back. You were just doing a bit to pass the time.