If you dig into the Info.plist for the service and change the item under "NSSendFileTypes" from "public.item" to "public.text", the service should only show up in contexts where the selected file is text.
However, as I believe you alluded to, this concept of what is text is dependent on file extension and there are a variety of circumstances (extensionless executables, various XML-based system file types) where the contents are in fact text but OS X doesn't recognize them as such and (if one makes the change I just suggested) the service will not appear.
Really, it all depends on ones' usage for this service and how annoyed you get with Services items appearing in contexts where they're non-functional or irrelevant.
If /usr/local/bin/growlnotify doesn't exist, the script exits with 1, so there's an error dialog every time the service is run.
pbcopy replaces non-ASCII characters with gibberish by default in the environment used by Automator (and do shell script). From the man page:
> pbcopy and pbpaste use locale environment variables to determine the encoding to be used for input and output. For example, absent other locale settings, setting the environment variable LANG=en_US.UTF-8 will cause pbcopy and pbpaste to use UTF-8 for input and output. If an encoding cannot be determined from the locale, the standard C encoding will be used.
Just changing LC_CTYPE is enough though (like LC_CTYPE=UTF-8 pbcopy).
file -b (brief) wouldn't prepend filenames, so grep wouldn't match filenames that contain "text".
file also gives me illegal byte sequence errors with some Japanese files. I'd just do something like [ $(stat -f %z "$f") -le 50000000 ].