Ytdyklly

I am searching for a short story I enjoyed in my youth. I think I read it circa 1975 in a hardcover collection of short stories that was old by then. I only recall fragments of it unfortunately.

It was about a man who was visited by an impish alien or maybe a being from another dimension. I recall an orb the man may have used to summon it. The two attempt to communicate including sampling each others' beverages. The man is knocked out by the visitor's drink.

When he awakes he discovers the visitor is gone and he can no longer read. Turns out it isn't just him, the visitor has somehow destroyed language.

He goes on a frantic search and spots the visitor sitting on top of a bus driving down the street.

Damon Knight's Babel II fits the description of your story. It's mentioned in the Encyclopedia of Fictional and Fantastic Languages

In Damon Knight's 1953 short story "Babel II," a comic-book artist named Cavanaugh meets a strange being nicknamed by the narrator "the Hooligan" after its physical resemblance to the title character of Frederick Burr Opper's cartoon strip, Happy Hooligan (begun in 1900). After some friendly trade is established between them, the hero misguidedly pours the Hooligan a glass of wine, who happily reciprocates by activating "a smallish green and white doodad" which causes Cavanaugh to have "the odd sensation that someone was stirring his brains with a swizzle stick".

After the Hooligan leaves, Cavanaugh discovers what the "doodad" has wrought: all human language has become scattered idiolects, and no one person can understand the utterances of another. For example, a cab driver says to Cavanaugh, "Zawss . . . owuh kelg trace wooj'l, fook. Bnog nood ig ye nolik?"

Letters and phonemes have apparently been randomly interchanged for every speaker of every language, but even this is not the fullest extent of the phenomenon. All written words have also been scrambled: every book, every newspaper, every sign or bill or label with letters or numbers is no longer intelligible. Images are not affected, and Cavanaugh manages to communicate a little to others by drawing hieroglyph cartoons.

Despairing of ever understanding or being understood by another person again and yet also glad for the newfound end to previously unchecked human blather, Cavanaugh reencounters the Hooligan and asks, "Could you fix just the writing—not the speech?" . The Hooligan complies; the story ends with Cavanaugh smiling at the thought that "the human race was now permanently a little tipsy" .

Hat tip to user Danny3414 for reminding me of the opening line in a comment

"From the front he looked a little like Happy Hooligan, if you remember that far back"