Author here. A few people are arguing against a stronger claim than the repo is meant to make. As well, this was very much intended to be a joke and not research level commentary.
This skill is not intended to reduce hidden reasoning / thinking tokens. Anthropicâs own docs suggest more thinking budget can improve performance, so I would not claim otherwise.
What it targets is the visible completion: less preamble, less filler, less polished-but-nonessential text. Therefore, since post-completion output is âcavemannedâ the code hasnât been affected by the skill at all :)
Also surprising to hear so little faith in RL. Quite sure that the models from Anthropic have been so heavily tuned to be coding agents that you cannot âforceâ a model to degrade immensely.
The fair criticism is that my â~75%â README number is from preliminary testing, not a rigorous benchmark. That should be phrased more carefully, and Iâm working on a proper eval now.
Also yes, skills are not free: Anthropic notes they consume context when loaded, even if only skill metadata is preloaded initially.
So the real eval is end-to-end:
- total input tokens
- total output tokens
- latency
- quality/task success
There is actual research suggesting concise prompting can reduce response length substantially without always wrecking quality, though it is task-dependent and can hurt in some domains. (https://arxiv.org/html/2401.05618v3)
So my current position is: interesting idea, narrower claim than some people think, needs benchmarks, and the README should be more precise until those exist.
derefrtoday at 4:46 PM
I've always figured that constraining an LLM to speak in any way other than the default way it wants to speak, reduces its intelligence / reasoning capacity, as at least some of its final layers can be used (on a per-token basis) either to reason about what to say, or about how to say it, but not both at once.
(And it's for a similar reason, I think, that deliberative models like rewriting your question in their own terms before reasoning about it. They're decreasing the per-token re-parsing overhead of attending to the prompt [by distilling a paraphrase that obviates any need to attend to the literal words of it], so that some of the initial layers that would either be doing "figure out what the user was trying to say" [i.e. "NLP stuff"] or "figure out what the user meant" [i.e. deliberative-reasoning stuff] â but not both â can focus on the latter.)
I haven't done the exact experiment you'd want to do to verify this effect, i.e. "measuring LLM benchmark scores with vs without an added requirement to respond in a certain speaking style."
But I have (accidentally) done an experiment that's kind of a corollary to it: namely, I've noticed that in the context of LLM collaborative fiction writing / role-playing, the harder the LLM has to reason about what it's saying (i.e. the more facts it needs to attend to), the spottier its adherence to any "output style" or "character voicing" instructions will be.
padolseytoday at 2:24 PM
This is fun. I'd like to see the same idea but oriented for richer tokens instead of simpler tokens. If you want to spend less tokens, then spend the 'good' ones. So, instead of saying 'make good' you could say 'improve idiomatically' or something. Depends on one's needs. I try to imagine every single token as an opportunity to bend/expand/limit the geometries I have access to. Language is a beautiful modulator to apply to reality, so I'll wager applying it with pedantic finesse will bring finer outputs than brutish humphs of cavemen. But let's see the benchmarks!
Idk I try talk like cavemen to claude. Claude seems answer less good. We have more misunderstandings. Feel like sometimes need more words in total to explain previous instructions. Also less context is more damage if typo. Who agrees? Could be just feeling I have. I often ad fluff. Feels like better result from LLM. Me think LLM also get less thinking and less info from own previous replies if talk like caveman.
tapoxitoday at 6:01 PM
This is neat but my employer rates my performance based on token consumption; is there one that makes Claude needlessly verbose?
alfanicktoday at 7:09 PM
Either this already exists, or someone is going to implement that (should I implement that?):
- assumption LLM can input/output in any useful language,
- human languages are not exactly optimal away to talk with LLM,
- internally LLMs keep knowledge as whole bunch of connections with some weights and multiple layers,
- they need to decode human-language input into tokens, then into something that is easy to digest by further layers, then get some output, translate back into tokens and human language (or programming language, same thing),
- this whole human language <-> tokens <-> input <-> LLM <-> output <-> tokens <-> language is quite expensive.
What if we started to talk to LLMs in non-human readable languages (programming languages are also just human readable)? Have a tiny model run locally that translates human input, code, files etc into some-LLM-understandable-language, LLM gets this as an input, skips bunch of layers in input/output, returns back this non-human readable language, local LLM translates back into human language/code changes.
Yesterday or two days ago there was a post about using Apple Fundamental Models, they have really tiny context window. But I think it could be used as this translation layer human->LLM, LLM->human to talk with big models. Though initially those LLMs need to discover which is "language" they want to talk with, feels like doable with reinforcement learning. So cheap local LLM to talk to big remote LLM.
Either this is done already, or it's a super fun project to do.
crispyambulancetoday at 7:10 PM
I no like.
It sort of reminds me of when palm-pilots (circa late-90's early 2000's) used short-hand gestures for stylus-writing characters. For a short while people's handwriting on white-boards looked really bizarre. Except now we're talking about using weird language to conserve AI tokens.
Maybe it's better to accept a higher token burn-rate until things get better? I'd rather not get used to AI jive-talk to get stuff done.
nayrocladetoday at 10:46 AM
Cute idea, but you're never gonna blow your token budget on output. Input tokens are the bottleneck, because the agent's ingesting swathes of skills, directory trees, code files, tool outputs, etc. The output is generally a few hundred lines of code and a bit of natural language explanation.
Okay, I like how it reduces token usage, but it kind of feels that, it will reduce the overall model intelligence. LLMs are probabilistic models, and you are basically playing with their priors.
Soma (aka tiktok) and Big Brother (aka Meta) already happened without government coercion, only makes sense that we optimize ourselves for newspeak.
Thank God there is still neverending wars, otherwise authoritarian governments would have no fun left.
abejfehrtoday at 2:08 PM
Thereâs a lot of debate about whether this reduces model accuracy, but this is basically Chinese grammar and Chinese vibe coding seems to work fine while (supposedly) using 30-40% less tokens
veselintoday at 11:48 AM
This is an experiment that, although not to this extreme, was tested by OpenAI. Their responses API allow you to control verbosity:
I don't know their internal eval, but I think I have heard it does not hurt or improve performance. But at least this parameter may affect how many comments are in the code.
deletedtoday at 2:49 PM
TeMPOraLtoday at 10:18 AM
Oh boy. Someone didn't get the memo that for LLMs, tokens are units of thinking. I.e. whatever feat of computation needs to happen to produce results you seek, it needs to fit in the tokens the LLM produces. Being a finite system, there's only so much computation the LLM internal structure can do per token, so the more you force the model to be concise, the more difficult the task becomes for it - worst case, you can guarantee not to get a good answer because it requires more computation than possible with the tokens produced.
I.e. by demanding the model to be concise, you're literally making it dumber.
(Separating out "chain of thought" into "thinking mode" and removing user control over it definitely helped with this problem.)
virtualritztoday at 10:31 AM
This is the best thing since I asked Claude to address me in third person as "Your Eminence".
But combining this with caveman? Gold!
somethingsometoday at 5:04 PM
I would like to see a (joke) skill that makes Claude talk in only toki pona. My guess is that it would explode the token count though.
bjackmantoday at 11:32 AM
If this really works there would seem to be a lot of alpha in running the expensive model in something like caveman mode, and then "decompressing" into normal mode with a cheap model.
I don't think it would be fundamentally very surprising if something like this works, it seems like the natural extension to tokenisation. It also seems like the natural path towards "neuralese" where tokens no longer need to correspond to units of human language.
postalcodertoday at 2:58 PM
I disagree with this method and would discourage others from using it too, especially if accuracy, faster responses, and saving money are your priorities.
This only makes sense if you assume that you are the consumer of the response. When compacting, harnesses typically save a copy of the text exchange but strip out the tool calls in between. Because the agent relies on this text history to understand its own past actions, a log full of caveman-style responses leaves it with zero context about the changes it made, and the decisions behind them.
To recover that lost context, the agent will have to execute unnecessary research loops just to resume its task.
goldenarmtoday at 3:16 PM
That's a great idea but has anyone benchmarked the performance difference?
VadimPRtoday at 10:20 AM
Wouldn't this affect quality of output negatively?
Thanks to chain of thought, actually having the LLM be explicit in its output allows it to have more quality.
gozzootoday at 10:26 AM
I think this could be very useful not when we talk to the agent, but when the agents talk back to us. Usually, they generate so much text that it becomes impossible to follow through. If we receive short, focused messages, the interaction will be much more efficient. This should be true for all conversational agents, not only coding agents.
There's linguistic term for this kind of speech: isolating grammars, which don't decline words and use high context and the bare minimum of words to get the meaning across. Chinese is such a language btw. Don't know what Chinese think about their language being regarded as cavemen language...
nharadatoday at 4:08 PM
I wonder if this will actually be why the models move to "neuralese" or whatever non-language latent representation people work out. Interpretability disappears but efficiency potentially goes way up. Even without a performance increase that would be pretty huge.
ajd555today at 12:32 PM
So, if this does help reduce the cost of tokens, why not go even further and shorten the syntax with specific keywords, symbols and patterns, to reduce the noise and only keep information, almost like...a programming language?
dakollitoday at 8:21 PM
The input costs for your prompt is the least expensive, and negligible cost when using agents. Its context & output, why go through all this?
RomanPushkintoday at 4:39 PM
Why the skill should have three absolutely similar SKILL.md files? Just curious
yakattaktoday at 5:12 PM
I was wondering just yesterday if a model of âwhy waste time say lot word when few word do trickâ would be easier on the tokens. Iâll have to give this a try lol
vivid242today at 10:54 AM
Great idea- if the person who made it is reading: Is this based on the board game âpoetry for cavemenâ? (Explain things using only single-syllable words, comes even with an inflatable log of wood for hitting each other!)
shomptoday at 3:09 PM
everyone who thinks this is a costly or bad idea is looking past a very salient finding: code doesn't need much language. sure, other things might need lots of language, but code does not. code is already basically language, just a really weird one. we call them programming languages. they're not human languages. they're languages of the machine. condensing the human-language---machine-language interface, good.
if goal make code, few word better. if goal make insight, more word better. depend on task. machine linear, mind not. consider LLM "thinking" is just edge-weights. if can set edge-weights into same setting with fewer tokens, you are winning.
rschiavonetoday at 10:44 AM
This trick reminds me of "OpenAI charges by the minute, so speed up your audio"
So you are telling me I prompted llms the right way all along
arrty88today at 3:59 PM
Feels like there should be a way to compile skills and readmeâs and even code files into concise maps and descriptions optimized for LLMs. They only recompile if timestamps are modified.
zahirbmirzatoday at 10:19 AM
You can also make huge spelling mistakes and use incomplete words with llms they just sem to know better than any spl chk wht you mean. I use such speak to cut my time spent typing to them.
norskeldtoday at 11:26 AM
APL for talking to LLM when? Also, this reminded me of that episode from The Office where Kevin started talking like a caveman to make communication efficient.
mwcztoday at 3:51 PM
this grug not smart enough to make robot into grugbot. grug just say "Speak to grug with an undercurrent of resentment" and all sicko fancy go way.
semessiertoday at 6:29 PM
the real interesting question would be if it then does its language-based reasoning also in short form and if so if quality is impacted.
herftoday at 4:16 PM
We need a high quality compression function for human readers... because AIs can make code and text faster than we can read.
HarHarVeryFunnytoday at 1:10 PM
More like Pidgin English than caveman, perhaps, although caveman does make for a better name.
andaitoday at 10:19 AM
So it's a prompt to turn Jarvis into Hulk!
andaitoday at 10:17 AM
No articles, no pleasantries, and no hedging. He has combined the best of Slavic and Germanic culture into one :)
ameliustoday at 12:34 PM
By the way why don't these LLM interfaces come with a pause button?
ungreased0675today at 2:36 PM
Does this actually result in less compute, or is it adding an additional âtranslate into cavemanâ step to the normal output?
sebastianconcpttoday at 1:46 PM
Anyone else worried about the long term consequences of the influence of talking like this all day for the cognitive system of the user?
ArekDymalskitoday at 10:17 AM
While really useful now, I'm afraid that in the long run it might accelerate the language atrophy that is already happening. I still remember that people used to enter full questions in Google and write SMS with capital letters, commas and periods.
I mean, I assume you run into the same problem as Kevin in the office; that sort of faux-simple speech is actually very ambiguous.
(Though, I wonder has anyone tried Newspeak.)
anshumankmrtoday at 3:30 PM
Though I do use Claude Code, is it possible to get this for Github Copilot too?
fzeindltoday at 12:00 PM
I tried this with early ChatGPT. Asked it to answer telegram style with as few tokens as possible. It is also interesting to ask it for jokes in this mode.
fnytoday at 12:29 PM
Are there any good studies or benchmarks about compressed output and performance? I see a lot of arguing in the comments but little evidence.
doe88today at 10:51 AM
> If caveman save you mass token, mass money â leave mass star.
Mass fun. Starred.
contingenciestoday at 7:22 PM
Better: use classical Chinese.
yesthisiswestoday at 4:50 PM
Why use lot word when few word do fine.
deletedtoday at 10:14 AM
owenthejumpertoday at 11:57 AM
What is that binary file caveman.skill that I cannot read easily, and is it going to hack my computer.
Suractoday at 7:45 PM
me like that
adam_patarinotoday at 12:33 PM
Or you could use a local model where youâre not constrained by tokens. Like rig.ai
throwatdem12311today at 4:25 PM
Ok but when the model is responding to you isnât the text itâs generating also part of the context itâs using to generate the next token as it goes? Wouldnât this just make the answersâŠdumb?
cadamsdotcomtoday at 10:40 AM
Caveman need invent chalk and chart make argument backed by more than good feel.
bitwizetoday at 1:13 PM
grug have to use big brains' thinking machine these days, or no shiny rock. complexity demon love thinking machine. grug appreciate attempt to make thinking machine talk on grug level, maybe it help keep complexity demon away.
us321today at 5:59 PM
I like
deletedtoday at 10:49 AM
kukakiketoday at 11:59 AM
This is exactly what annoys me most. English is not suitable for computer-human interaction. We should create new programming and query languages for that. We are again in cobol mindset. LLM are not humans and we should stop talking to them as if they are.
saidnooneevertoday at 10:25 AM
LOL it actually reads how humans reply the name is too clever :').
Not sure how effective it will be to dirve down costs, but honestly it will make my day not to have to read through entire essays about some trivial solution.
tldr; Claude skill, short output, ++good.
xgulfietoday at 1:58 PM
Funny how people are so critical of this and yet fawn over TOON
xpetoday at 1:37 PM
Unfrozen caveman lawyer here. Did "talk like caveman" make code more bad? Make unsubst... (AARG) FAKE claims? You deserve compen... AAARG ... money. AMA.
bogtogtoday at 10:31 AM
I'd be curious if there were some measurements of the final effects, since presumably models wont <think> in caveman speak nor code like that
isuckatcodingtoday at 3:55 PM
Oh come on now one referenced this scene from the office??
Oh, another new trend! I love these home-brewed LLM optimizers. They start with XML, then JSON, then something totally different. The author conveniently ignores the system prompt that works for everything, and the extra inference work. So, it's only worth using if you just like this response style, just my two cents. All the real optimizations happen during model training and in the infrastructure itself.
Robdel12today at 12:39 PM
I didnât comment on this when I saw it on threads/twitter. But it made it to HN, surprisingly.
I have a feeling these same people will complain âmy model is so dumb!â. Thereâs a reason why Claude had that âyouâre absolutely right!â for a while. Or codexâs âyouâre right to push on thisâ.
Weâre basically just gaslighting GPUs. That wall of text is kinda needed right now.
thorfinnntoday at 4:43 PM
kevin would be proud
vova_hn2today at 11:21 AM
I don't know about token savings, but I find the "caveman style" much easier to read and understand than typical LLM-slop.
hybrid_studytoday at 12:24 PM
Mongo! No caveman
bhwoo48today at 10:21 AM
I was actually worried about high token costs while building my own project (infra bundle generator), and this gave me a good laugh + some solid ideas. 75% reduction is insane. Starred
DonHopkinstoday at 12:26 PM
Deep digging cave man code reviews are Tha Shiznit: