Yes, there are privacy concerns with using Google Web Fonts. If you have strict privacy concerns you should probably not use the service. Users of Google Web Fonts are bound by Google's generic API terms of service, which includes this clause:
By using our APIs, you agree that Google can use submitted information
in accordance with our privacy policies, such as
http://www.google.com/privacypolicy.html.
Google's privacy policy allows it to collect a large amount of data about users of its services, both to improve the service and to support it commercially. This includes log data (e.g. browser version) and location data (the IP address of your site's visitors). Sites that use Google Web Fonts are feeding data back to Google. It's possible that Google doesn't actively collect and use that data right now, but it is worth considering alternatives if you are privacy-conscious.
Font Squirrel is a great source of free fonts that can be used commercially. Several of the fonts hosted by Google, such as Open Sans, are available to download from Font Squirrel and host yourself at no charge, and it's not nearly as tricky as you might think. Their downloadable “webfont kits” include a “how to use webfonts” HTML file that talks you through it, but there are other guides to using web fonts available online.
Update:
Google now publishes some info on Google Web Fonts and privacy in their FAQ that makes Google Web Font use seem a little safer from a privacy perspective:
The Google Fonts API is designed to limit the collection, storage, and use of end-user data to what is needed to serve fonts efficiently.
Use of Google Fonts is unauthenticated. No cookies are sent by website visitors to the Fonts API. Requests to the Google Fonts API are made to resource-specific domains, such as fonts.googleapis.com, googleusercontent.com, or gstatic.com, so that your requests for fonts are separate from and do not contain any credentials you send to google.com while using other Google services that are authenticated, such as Gmail.
In order to serve fonts as quickly and efficiently as possible with the fewest requests, we cache all requests made to our servers so that your browser only contacts us when it needs to.
Requests for CSS assets are cached for 1 day. This allows us to update a stylesheet to point to a new version of a font file when it’s updated. This ensures that all visitors to websites using fonts hosted by the Google Fonts API will see the latest fonts within 24 hours of their release.
The font files themselves are cached for one year, which is long enough that the entire web gets substantially faster: When millions of websites all link to the same fonts, they are cached after visiting the first website and appear instantly on all other subsequently visited sites. We do sometimes update font files to reduce their file size, increase coverage of languages, and improve the quality of their design. The result is that website visitors send very few requests to Google: we only see 1 CSS request per font family, per day, per browser.
We do log records of the CSS and the font file requests, and access to this data is on a need-to-know basis and kept secure. We keep aggregated usage numbers to track how popular font families are, and we publish these aggregates in the Google Fonts Analytics site. From the Google web crawl, we detect which websites are using Google Fonts, and publish this in the Google Fonts BigQuery database. To learn more about the information Google collects and how it is used and secured, see Google's Privacy Policy.