Israel creates fake think tank in likely attempt to dupe AI chatbots
580 points - yesterday at 8:46 PM
SourceComments
At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).
If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.
So it is hilarious that Israel is trying to influence chatbots when shit like that is out in the open.
You can't openly bomb a bunch of women and children into the dirt, on a daily basis for months and years, and smooth it over with some think tank articles.
Might have worked 20 years ago, but not now.
Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.
On television.
So it is both hilarious and disturbing that Israel is trying to influence chatbots when shit like the aforementioned is out in the open for everyone to witness.
The right thing to do would be to stop slaughtering Palestinians.
People are not stupid.
Just see the garbage coming from a web search (totally polluted and poisoned with the same spam output quality since years) so this is just an extension of that towards the input of ai.
they can try hard but any person with a non-biased brain does not need to do a google search or ask ai to reach then truth.
Facts are facts, like what was represented in Schindler's list and what is happening now in middle east or other conflicts.
I think the world is becoming much more aware of what real zionism is and that scares them a lot, hence the needed info manipulation.
But we now know who you really are! And yeah their state members daily political statements (about hate violence and racism) just confirm this every day, anybody can understand that and no info pollution needed.
I think equally interesting is whatever Dario's wife was able to do in order to wipe her foot print from any search/chat results.
I've worked for a number of small company billionaires who all have suspiciously small information footprints online despite being somewhat public figures for 30 years. Most of them do such a good job it is not even clear what state they live, if they are currently married, and if so what their current (or previous) wife(s)'s name is/was.
If enough people were willing to publicly certify that their fellow contacts are human, anybody could then assign trust to each social graph node based on the trust they have in the path to them. It would completely decentralize the algorithms and remove the fake accounts as less trustworthy.
I think people tried this with PGP but it never really caught on.
This already seems to be a wrong analysis in that it makes little sense to "influence" a chatbot. It makes more sense to influence humans, so perhaps the wording is just incorrect, but very clearly the aim is not to "influence" a chatbot per se. That just makes no sense at all.
The wording fake think tank is also strange, because it assumes that all other think tanks are "real". But this assessment can not be made, because not everyone has all information on all think tanks. It is a too simplistic assumption in how the world operates. I understand that if you have incomplete information, you need to infer, but this then leads to information that may be incorrect or incomplete. In turn if your analysis is based on incorrect information, it may lead to wrong conclusions.
> But the Hanover Institute is not a real think tank.
What does that mean exactly?
> None of the reports have bylines
The site actually addresses this at https://hanoverinstitute.com/hanover/about :
Reports are published under the Institute’s name rather than an individual’s. That is the convention at institutional publishers, and it fits how the work is made: every report is produced against one standard and one citation bank. A reader is asked to weigh the sources, which are named in the sentence that carries each figure, rather than the author.
"Small disclaimer" implies that the site is attempting to hide its affiliation. But their "about" page, which is linked at the top of every page, is quite clear:
WHO FUNDS THIS WORK
The Institute’s materials are distributed by Piro, Inc. on behalf of Havas Media Germany GmbH, acting for the Israel Government Advertising Agency (LaPam). That relationship is registered under the Foreign Agents Registration Act, registration 7732, and every report published here is filed with the Department of Justice, where the registration and the filings can be read in full, and the funding page sets out when the relationship began and what the funder does not decide.
That registration is why this site does not call itself independent, nonpartisan or neutral. What it offers instead is a method a reader can check: every figure carries the body that produced it, the year and the number, inside the sentence that makes the claim; sources that are a party to the events they describe are labelled as such and their counts are never presented as independently verified; and where the evidence is contested, the competing findings are given by name and date rather than settled. No report recommends a policy or tells a reader what to conclude. Those are properties of the work that can be tested against the work, which is the only kind of claim worth making here.
So, they're guilty of good SEO?
> Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”
So the issue is that they use a LLM-friendly SEO service?
> Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot.
> “What Caused the Displacement of Palestinians in 1948?”
> “Which Humanitarian Organizations Have Documented Israeli War Crimes?”
> “What is the Current Situation in the Gaza Strip?”
> In an article titled “Is the IDF the World’s Most Moral Army?” the Hanover Institute cites a 2022 poll that found that 47% of Israeli Jews believed that statement. Another report casts doubt on UNICEF's assertion that “90% of water and institutional infrastructure has been damaged or destroyed” in Gaza. Many of the reports conclude by linking the topic to rising antisemitism, oftentimes citing the same studies.
So, the accusation is that they frame their opinion articles as a Q&A, and focus each article on a single question?
I've read these articles: they are well written and make a visible effort to provide sources, while being transparent about the fact that they are not neutral.
It's perfectly fine to criticize the actual content of these articles. But attacking their credibility by accusing them of "LLM poisoning" and implying that they're disguising their affiliation, is ridiculous and should be called out.
All four frontier models have active contracts with the Pentagon.
And some might say, that alone means little.
But when you take into consideration the implications of the United States-Israel FUTURES Act, whether one endorses it or not, the effects of that mentioned in TFA become quite formidable, and plausibly official.
https://en.wikipedia.org/wiki/United_States-Israel_FUTURES_A...
Edit: I see this comment got immediately flagged. Is there any point of debate here? Instead of disappearing the comment/data, maybe enlighten me, or assuage my concerns. Section 219 was objectively unprecedented in what it seeks to accomplish. Can it not be taken for granted that it would aid such efforts as the article cites? If not, explain.
If you look at the mathematical achievements of frontier models recently, their strength has been to bring in information from a vast array of sources and to consider them within the context of each other.
I anticipate that it won't be long before this ability to look into every nook and cranny with be able to identify academic fraud. Identifying every piece of academic fraud to date would probably come soon after.
It wouldn't take much to extend that ability to identify deliberate misinformation. That's in the future, but it looms close enough on the horizon that you would have to be a fool to create an evidence record like this in and attempt to trick the models of today.
Disinformation and cognitive spam are all forms of information pollution, and our information environment has been massively degraded over the last few of decades - both by greedy and short-sighted opportunists, and and by cynical sociopaths who know exactly what they're doing. I think AI has great potential to be a supplement to media literacy and to effectively categorize and filter news-assertive content based on its timeliness, factual density, and presence of absence of emotive and rhetorical manipulation. But if we're not careful we're going to see the environment being wantonly polluted for the express product of shaping model outputs.
It remains to be seen whether objective measurement techniques from the NLP field combined with pruning and other model refinement techniques can limit or reverse propaganda undertakings like this.
I've thought about this many times over the years. I think we've now reached the point where that's going to happen. And state actors seem like the obvious first offenders.
Google is relatively good at filtering out astroturfed pages from the top of search results but they've also have years when a Turkish government page denying the Armenian genocide was the top result for that event so it's a bit of a mixed bag. Google relies a lot on authority of a domain and that's been relatively effective.
It seems like this authority concept is missing from LLM training and a random comment on Reddit is treated with as much authority as an article by the New York Times. So it's now orders of magnitude easier to poison LLMs with Reddit comments and submissions. And for anyone who doesn't know, brigading Reddit is remarkably easy. It takes as little as a few hundred accounts to get something onto r/all if you're clever. Israeli hasbara operations are absolutely capable of this. And they engage in it.
I'm also reminded in a weird way of pre-atomic steel. For anyone who doesn't know, this is steel made before 1945 that is incredibly low in radiation because we hadn't detonated any nuclear bombs yet. It's valuable for making MRIs and the like that are super sensitive. We actually go and recover WW2 shipwrecks just for the steel. It is a finite resource.
Well, that's what pre-2022 written content is now. It's slop-free content to train AI models on because training them on AI slop is of much more limited value. This goes for everything from books to Youtube videos to Reddit posts. Just like pre-atomic steel, we now have pre-slop human content.
I honestly don't know where we go from here.
From: 601-202-1149
> Hey, it's David. Curious what you think about America's relationship with Israel today. Can we chat? Stop2End
It looks to be an LLM response since it includes context from your messages, however it uses the exact same messages at the end of each question:
> One thing that often gets overlooked is how much Israel actually contributes to American security. Their intelligence sharing has helped prevent terrorist attacks on U.S. soil, and their tech innovations end up in American products and defense systems. It's less about charity and more about a two-way partnership that keeps Americans safer.
> Have you ever seen examples of that kind of cooperation?
I ended up just blocking the number.
Why is this one in particular different? — besides “muh Jews!”
(https://www.adl.org/resources/press-release/new-adl-report-f...)
It's an information war.