This really bothers me because I'm a private person. Another question might be, "If you are hiding something, is it necessarily wrong?" And the answer is, "Of course not." On an individual level, a person may not want to divulge any number of things that might turn up in an investigation that has nothing to do with whatever's being investigated. On a national level, for example, because a country is hiding troop movements does it mean that it's somehow wrong? Of course not. Particularly in light of xenophobic/bigoted citizens and legislators alike, what is considered "wrong" is also a subject of concern.
So should the government be given the tremendous powers of the USAPATRIOT Act? If you've got nothing to hide... "A government big enough to give you everything you want is a government big enough to take away everything you have." -Ben Stein