Exploring
Edinburgh is a beautiful city and the best way to explore it properly is to walk through it. While much of Edinburgh can be enjoyed simply by wandering around and going wherever looks interesting, to get the best experience possible it is advisable to get some form of guide book or go on one of the many guided tours that are available.
A haunting experience
There are many, many ghost stories told about the streets and houses of Edinburgh. If you want to learn a little of Edinburgh's more ghoulish history while touring some of the more hidden parts of the city then a ghost tour may be for you. Many of these operate nightly and finding one is as easy as walking onto the Royal Mile and looking for a sign (an actual physical sign, not a supernatural, burning bush type sign). Most tours start between 6.30 and 9.30 though there are so many that there is a good chance of one running anytime you want to go.
Day Tickets
Edinburgh has an amazing bus network, any time of day or night just about every part of the city is accessible by bus. During the day a single journey (to anywhere on a single route) will cost £1.20. If you're planning on taking multiple buses throughout the day then a ticket can be purchased for £3 that will allow travel all day on any of the Lothian buses.
Guided Tours
A number of guided open top bus tours run in Edinburgh. These provide information on the identity and history of the various landmarks visible from the tour. These tours are often provided in more than one language. Edinburgh has a diverse array of people living here so tours in most languages can be found eventually (except Gaelic, nobody speaks Gaelic).
Many of the bus tours start from outside Waveley train station, so they are easy to find. Most also allow you to get on and off the tour buses throughout the journey. This allows you to explore each part of Edinburgh and rejoin the tour with a later bus.