A while ago, I wrote about the new tagging feature in OS X Mavericks. Back then, it simply wasn’t possible to delete tags (system wide, I mean). Not anymore in Developer Preview 6.
Alex let us know about this little but helpful change: In Finder preferences, under the Tags pane, all your tags will show up. A simple right click on any single one of them will bring up a little menu and the second option is “Delete Tag ‘You Tag’s name’…”
Why this option wasn’t there in earlier builds, only Apple knows.