This has got to be my favorite bug (feature? : ) of SharePoint so far. You'll run into an error telling you that the PageLayout is attached, delete all references etc etc. BUT, there is an easy workaround.
Simply open up Designer, make a new folder named "delete" in the same directory, cut the page layout(s) you'd like to get rid of, paste them in the delete folder. And then delete the folder.
Easy as that. But not quite as easy as deleting via the browser!