If it's your sort of thing at all, the Baltimore Aquarium is one of the better ones around. It also puts you pretty close to Harbor East and then Fells Point and Canton if you're traveling East, you could head out that way one day.
Each area of town has their own little charm to it. Federal Hill will be the closest to downtown, definitely walk around to a few different bars and restaurants there. Cross St market is pretty cool, down at the end near the seafood places is a great place to watch an O's game.
Mike had a bunch of great suggestions, all are good. I'd say that my favorite brunch in the city is at Fork & Wrench in Canton on Boston Street. Definitely try to get there one day. Also, if you're out late at night, Blue Moon Cafe in Fells Point is amazing and open 24-hours. It'll take you hours to get a table if you go there in the morning on a weekend, but you may have better luck on Friday morning. They have Captain Crunch French Toast.
If you're looking for a quicker bite rather than a full restaurant Samos in Greektown is fantastic Greek food. Everything there is good. DiPasquale's just north of Canton (Gough St) is a fantastic Italian sandwich shop (Trinacria downtown near Lexington Market is also a really good one if you're staying near downtown). Get the Porketta sandwich. Chap's Pit Beef is the best pit beef in the city. It's a shack in the parking lot of a strip club, though, and a bit far from anything else.
For restaurants, Brewer's Art in Mount Vernon is awesome, Annabelle Lee in Canton is really good. There are a ton of great places to eat in Mount Vernon and Hamden, more of them and more consistently better restaurants IMO than Canton or Fells or Fed, though typically more expensive.
There are a lot of Farmer's markets in the area as well. The huge one Sunday underneath 83 is really great, but there is one in Fells Point on Saturday morning's as well. Kind of cool to walk around, and you can buy sandwiches or coffee or breakfast stuff, not just selling raw produce.
Shopping's not really my thing, but you could take a walk through Fells Point along Aliceanna, lots of small shops in the area. Harbor East has lots of shops too, but I think they are more expensive.