Ytdyklly

I am seeing non-printable characters in a lot of system contexts, like man pages and logs. They show up as a highlighted question mark. For example, in one man page I have the following values:

E2 9F A8 ...email address... E2 9F A9

and the author name

F r C3 A9 d C3 A9 r i c k

where the numbers are hex values. I could not find these values in unicode, so I am not sure what they are. When I cat them as escape sequences I get the following:

M-b M-^ M-^_ M-(  ... email address ... M-b M-^_ M-)

How can I get the output translated correctly? Note that my locale is set as:

LANG=en_US.UTF-8

The $TERM is "linux" and I am using a virtual console on the machine itself.

Basically, while you're commenters are not explicitly stating it, your man page is being formatted with UTF-8 characters. Depending on various things, this may be fixed with

1. 
   

   Assuming that you want American English in your man pages,

   
   
   

   LC_CTYPE=C man whatever you wanted to look up

   
   

should suppress that behavior. Judging by your bio, you might like that option. You could specify LC_ALL instead of LC_CTYPE if you don't have LC_CTYPE set. And like all environment variables, you can set that in your shell and have it take effect for all of your commands.

Another locale option that would disable this is en_US, and if you don't want American English, then there's a bunch of other locales, many of which have a .UTF-8 and presumably don't do UTF-8 without it.

1. If your console font happens to have a unicode mapping for the characters in question, you may be able to just type -r in your man pager, and your pager will show the right character. That said, this setting is not without risk; it allows any terminal control sequences in the document you're viewing to take effect. You should be safe doing it while viewing man pages, but if viewing files from untrustworthy sources or viewing files with random data in them, it could lock up your console or worse. (There are some terminal escape sequences to allow a file you're displaying to the console to put characters in the keyboard buffer, having them take effect as if you typed them as soon as the keyboard buffer is read. I don't know if the linux virtual console supports any of these or not.)

I think this is something people notice more due to Linux distributions having switched to defaulting to UTF-8 turned on rather than off. Meanwhile, the Linux console still only supports 512 characters in a font, and that only if half of the 16 colors are disabled. And in other Unicode teething issues, X window fonts only support 64k characters, while Unicode has a lot more than 64k characters.