30 Mar 2024 Search

One of the big problems with widespread use of large language models (LLMs) is going to be that reputation management firms will be able to put up a lot of content to try to clean up mentions of their own clients. Other players will also, in a totally deniable way, be able to put up their own text to train LLMs to say bad stuff about people who are opposed to their clients in some way. What is seen as important material about a person from the point of view of, say, a reporter or Wikipedia editor, is not necessarily going to be what gets pulled out a big pile of crawled text by an LLM.

You can kind of see what’s possible by looking at 20th century history. Many Axis officers from World War II have a lot of content about them on the Internet somewhere.

  1. contemporary propaganda

  2. post-war content such as the “clean Wehrmacht” campaign

  3. online forums discussing 1 and 2 above

  4. gaming-related content

For example, let’s take a look at the 14 defendants in the High Command Trial at Nuremberg. To a human editor, the Nuremberg trials were a significant historic event, so the fact that an officer was a defendant is going to be a key fact about him that makes it into any reasonable length bio.

For ChatGPT, not so much. Asking ChatGPT: “what is officer_name known for” gives different results. I tried it.

  • All 14 are correctly identified by country and historical period

  • In 5 cases, the ChatGPT answer has no mention of the trial, of any of the conduct for which the officer was tried, or any mention of war crimes at all.

Answers often end with something like, After the war, officer_name was captured by Allied forces and held as a prisoner of war. He was later released and returned to civilian life.

The Nuremberg trials were extremely well covered at the time, a lot has been written about them, and they’re still studied today. Old-school propaganda operations, keyboard warriors, and random wehraboos were able to scrub this major event, and the crimes for which the defendants were tried, without really trying. So the same kind of thing is going to be practical for lots of people, companies, and organizations. Reputation management firms typically don’t have to obscure something as big as the Nuremberg trials, and they can use LLMs to produce far more text, faster, than even an online army of gamers can. Expect the defendants in run-of-the-mill corporate trials to effectively disappear their crimes, too.

It’s possible that the big “AI” companies could deal with the reputation management problem by licensing a set of known neutral point of view (NPOV) content and either avoiding material that looks conflicting, or filtering out responses that look inconsistent with it. But that’s a big task, and will get bigger the more that “reputation management” is seen to work. More: The hidden cost of New York Times AI columns

(This is an edited version of a thread).