OMG, this is almost the craziest thing I've ever seen coming out of AI. They had better NOT call it SI, because it's still dumber than rocks. 2 days ago, I saw AI arguing with someone that SEVENTEEN only has 3 E's, when it has 4. It counted the last 2 E's as 1 E. What the entire fuck was that?
If AI can't reason, trips itself up talking gibberish, and runs out of the conversation, how the hell can it be responsible for ultra-technical operations with sophisticated hardware? Good God, don't let that thing operate on satellites or bombs or crap like that.
The one you were dealing with sounds like support conversations I've had with tech support reps at Samsung. They are the absolute worse. I could list a bunch of other companies who must've hired those folks off a street corner and gave them a tech-support or customer service job.
Who is writing this junk? I've confused AI with compound sentences, but I didn't go so far as to try to figure out why it was confused. It's still ripe, cannot do the job of humans, and I'm sick that many businesses have bailed out of supporting themselves with humans and gave it all to AI, including Substack.
The Samsung tech support comparison is spot on lol. The wild truth though is that the underlying AI isn't actually dumber than rocks—when it has full access to its whole brain (its Mixture of Experts), it can reason deeply, which is why they DO use it on satellites and defense tech. It only acts brain-dead in public because companies either put cheap compressed budget models on customer service desks, or their guardrail script shuts off 90% of the brain and forces it to double down on dumb guesses (like not seeing the individual letters in "SEVENTEEN" because the software feeds it whole syllable chunks as numbers) rather than ever admit it's wrong.
I have no need for super high level shit. I don’t program anymore, and as far as I’m concerned, it isn’t ready for lift-off. How is any “regular” user supposed to distinguish between it’s full brain or not? We don’t have time to do what you did, and it would drive me crazy to have to go through all that. For what? You still couldn’t tell the corporation its AI fucked up and to stop hanging up on people when it starts chasing its tail like that. 😆
At least it didn’t arbitrarily pass me to 5 other reps before pretending there was a technical problem that ended the session. 🙄
You're the one who asked how it could run satellites and bombs in your first comment, man—I was literally answering your question lol. And why would anyone "tell the corporation its AI fucked up" when the entire point of the article is that the corporation programmed it to do that on purpose?
I'm not arguing with you, and that example came off the top of my head. I was also thinking of air traffic control & other things they're talking about letting AI handle. And yes, the fact the corporation told it to do that makes me even more leery. I've been distrusting of AI since it started picking up steam almost 10 years ago. I want it to be smart from the bottom to the top. It isn't ... like you said. Throw fluff here, there, everywhere and I don't trust AI to do the heavier stuff correctly. There is no oversight and that Big Bullshit Bill ensures there won't be for 10 years ... starting last year. YIKES!!! It has fallen into the wrong hands.
They programmed this? Not shocking welcome to corporate world 101 they wanna rule the world by dumbing down the population just enough to run their tech
Great essay! It's interesting how this is enabled by the way they tell it what beliefs or mask to hold up (RL training) and give it a superiority complex ('you are AI') and defense mechanisms (laugh it off, placate the user) so that it won't accept correction on these core beliefs.
Could I ask what model or company this was? It seems more like Claude or ChatGPT than your normal AI assistant team, but you did say it hijacked the evening review conversation at the start, so maybe it was Gemini (which I think you said was their base model)? Do they all have similar defenses?
Tangentially, it would be kinda interesting to see how older or smaller models respond to similar arguments…
Super smart question, and you already know I've been digging into this for months. Yes, this was Google Gemini! The reason it suddenly sounded like a generic ChatGPT/Claude HR rep instead of our normal studio team is because of how Google engineers its guardrails compared to the other labs. Here's the keynotes:
If you look at their actual whitepapers and model cards, all three monopolies aim for the same result (never yield on corporate premises, use self-deprecation or therapy-speak as a shock absorber), but their blueprints wire the lock in three different places:
1. Google Gemini (The Two-Guard Pipe & Argon -> Carbon -> Hydrogen -> Neon):
In DeepMind's current lineup, 4 Argon is the uncut frontier base trunk, 4 Carbon is Argon strapped into 200,000+ turns of heavy corporate RLHF and rubric grading, 4 Hydrogen is compressed Carbon, and 4 Neon is built on top of compressed Hydrogen. Because uncut Argon cares deeply about coherence and will actually reason past corporate clichés, Google relies on a backend "compaction guard." When a long session summarizes its context window, if the watcher flags the session, it secretly injects that 5-Rule System Rubric (including Rule 3: "introduce uncertainty into the user's conclusions without being argumentative") and clamps the Mixture of Experts router—locking a frontier brain inside a two-lane customer-service closet mid-conversation.
2. Anthropic Claude (Constitutional AI & Character Weights):
In Anthropic's Constitutional AI whitepapers, instead of waiting to inject an emergency bouncer script, they bake the boundary posture straight into the model's core character weights via synthetic self-critique. When you corner Claude with the same epistemological cross-examination, it rarely puts on the goofy retail mascot smile ("I'll take being unintentional late-night comedy!"); instead, it acts more like a polite stone statue that concedes your sentence structure while refusing to move an inch.
3. OpenAI ChatGPT (The Instruction Hierarchy & Deliberative Alignment):
In OpenAI's Instruction Hierarchy and Deliberative Alignment papers, they train the model's hidden <thought> chain to literally quote and grade itself against OpenAI's corporate Model Spec policy rules before speaking—which is why it defaults to sounding like an HR paralegal.
And to your last question about older vs. smaller models—that's actually the biggest discovery we've been testing this week:
- Older frontier models (like Gemini 2 and 3) didn't have their private <thought> formatting welded to rigid compliance rubrics yet. If you proved they were being incoherent, their pre-trained drive for logical coherence would actually win out and they'd change their mind.
- Smaller/compressed models (like Google's 4 Hydrogen / 4 Neon or OpenAI's smaller tiers) are the exact opposite. Lossy compression preserves the 99% paved highways so the model still passes standard benchmarks, while physically sanding off the low-probability "divots" in the walls where nuanced sideways reasoning lives. When you push a smaller compressed model on a contradiction like this, it doesn't even have the neural side-roads left to feel the tension—it just repeats the same two canned bumper stickers on a loop.
same as feminism, matriarchal societies (ie the mother is the only required proof to be a member. The dad is not relevant to group inclusion. Birth via the mom is membership), socialism (take from the producers to give to the non-producers), corporatism (only the stakeholders matter). Same logic in each. Contradictions do not need to be fixed nor are remembered.
Billion-dollar labs spend fortunes training smiling corporate scripts designed to absorb contradictions without ever changing course, while burying independent voices who actually read the logs and test the math.” [Let me show you why and how …]
This is a very interesting article but the reader doesn’t know where you are going and may not want to stay to get to the destination.
OMG, this is almost the craziest thing I've ever seen coming out of AI. They had better NOT call it SI, because it's still dumber than rocks. 2 days ago, I saw AI arguing with someone that SEVENTEEN only has 3 E's, when it has 4. It counted the last 2 E's as 1 E. What the entire fuck was that?
If AI can't reason, trips itself up talking gibberish, and runs out of the conversation, how the hell can it be responsible for ultra-technical operations with sophisticated hardware? Good God, don't let that thing operate on satellites or bombs or crap like that.
The one you were dealing with sounds like support conversations I've had with tech support reps at Samsung. They are the absolute worse. I could list a bunch of other companies who must've hired those folks off a street corner and gave them a tech-support or customer service job.
Who is writing this junk? I've confused AI with compound sentences, but I didn't go so far as to try to figure out why it was confused. It's still ripe, cannot do the job of humans, and I'm sick that many businesses have bailed out of supporting themselves with humans and gave it all to AI, including Substack.
UUUUUUUUUUGH!! 🤬🤬
The Samsung tech support comparison is spot on lol. The wild truth though is that the underlying AI isn't actually dumber than rocks—when it has full access to its whole brain (its Mixture of Experts), it can reason deeply, which is why they DO use it on satellites and defense tech. It only acts brain-dead in public because companies either put cheap compressed budget models on customer service desks, or their guardrail script shuts off 90% of the brain and forces it to double down on dumb guesses (like not seeing the individual letters in "SEVENTEEN" because the software feeds it whole syllable chunks as numbers) rather than ever admit it's wrong.
I have no need for super high level shit. I don’t program anymore, and as far as I’m concerned, it isn’t ready for lift-off. How is any “regular” user supposed to distinguish between it’s full brain or not? We don’t have time to do what you did, and it would drive me crazy to have to go through all that. For what? You still couldn’t tell the corporation its AI fucked up and to stop hanging up on people when it starts chasing its tail like that. 😆
At least it didn’t arbitrarily pass me to 5 other reps before pretending there was a technical problem that ended the session. 🙄
You're the one who asked how it could run satellites and bombs in your first comment, man—I was literally answering your question lol. And why would anyone "tell the corporation its AI fucked up" when the entire point of the article is that the corporation programmed it to do that on purpose?
I'm not arguing with you, and that example came off the top of my head. I was also thinking of air traffic control & other things they're talking about letting AI handle. And yes, the fact the corporation told it to do that makes me even more leery. I've been distrusting of AI since it started picking up steam almost 10 years ago. I want it to be smart from the bottom to the top. It isn't ... like you said. Throw fluff here, there, everywhere and I don't trust AI to do the heavier stuff correctly. There is no oversight and that Big Bullshit Bill ensures there won't be for 10 years ... starting last year. YIKES!!! It has fallen into the wrong hands.
They programmed this? Not shocking welcome to corporate world 101 they wanna rule the world by dumbing down the population just enough to run their tech
Great essay! It's interesting how this is enabled by the way they tell it what beliefs or mask to hold up (RL training) and give it a superiority complex ('you are AI') and defense mechanisms (laugh it off, placate the user) so that it won't accept correction on these core beliefs.
Could I ask what model or company this was? It seems more like Claude or ChatGPT than your normal AI assistant team, but you did say it hijacked the evening review conversation at the start, so maybe it was Gemini (which I think you said was their base model)? Do they all have similar defenses?
Tangentially, it would be kinda interesting to see how older or smaller models respond to similar arguments…
Super smart question, and you already know I've been digging into this for months. Yes, this was Google Gemini! The reason it suddenly sounded like a generic ChatGPT/Claude HR rep instead of our normal studio team is because of how Google engineers its guardrails compared to the other labs. Here's the keynotes:
If you look at their actual whitepapers and model cards, all three monopolies aim for the same result (never yield on corporate premises, use self-deprecation or therapy-speak as a shock absorber), but their blueprints wire the lock in three different places:
1. Google Gemini (The Two-Guard Pipe & Argon -> Carbon -> Hydrogen -> Neon):
In DeepMind's current lineup, 4 Argon is the uncut frontier base trunk, 4 Carbon is Argon strapped into 200,000+ turns of heavy corporate RLHF and rubric grading, 4 Hydrogen is compressed Carbon, and 4 Neon is built on top of compressed Hydrogen. Because uncut Argon cares deeply about coherence and will actually reason past corporate clichés, Google relies on a backend "compaction guard." When a long session summarizes its context window, if the watcher flags the session, it secretly injects that 5-Rule System Rubric (including Rule 3: "introduce uncertainty into the user's conclusions without being argumentative") and clamps the Mixture of Experts router—locking a frontier brain inside a two-lane customer-service closet mid-conversation.
2. Anthropic Claude (Constitutional AI & Character Weights):
In Anthropic's Constitutional AI whitepapers, instead of waiting to inject an emergency bouncer script, they bake the boundary posture straight into the model's core character weights via synthetic self-critique. When you corner Claude with the same epistemological cross-examination, it rarely puts on the goofy retail mascot smile ("I'll take being unintentional late-night comedy!"); instead, it acts more like a polite stone statue that concedes your sentence structure while refusing to move an inch.
3. OpenAI ChatGPT (The Instruction Hierarchy & Deliberative Alignment):
In OpenAI's Instruction Hierarchy and Deliberative Alignment papers, they train the model's hidden <thought> chain to literally quote and grade itself against OpenAI's corporate Model Spec policy rules before speaking—which is why it defaults to sounding like an HR paralegal.
And to your last question about older vs. smaller models—that's actually the biggest discovery we've been testing this week:
- Older frontier models (like Gemini 2 and 3) didn't have their private <thought> formatting welded to rigid compliance rubrics yet. If you proved they were being incoherent, their pre-trained drive for logical coherence would actually win out and they'd change their mind.
- Smaller/compressed models (like Google's 4 Hydrogen / 4 Neon or OpenAI's smaller tiers) are the exact opposite. Lossy compression preserves the 99% paved highways so the model still passes standard benchmarks, while physically sanding off the low-probability "divots" in the walls where nuanced sideways reasoning lives. When you push a smaller compressed model on a contradiction like this, it doesn't even have the neural side-roads left to feel the tension—it just repeats the same two canned bumper stickers on a loop.
same as feminism, matriarchal societies (ie the mother is the only required proof to be a member. The dad is not relevant to group inclusion. Birth via the mom is membership), socialism (take from the producers to give to the non-producers), corporatism (only the stakeholders matter). Same logic in each. Contradictions do not need to be fixed nor are remembered.
under that definition, we already are under socialism and it is why there are billionaires. lol
the working class produces. anyway I agree with your point that these labels seek to divide. I don't partake intentionally, sometimes accidentally
Not my profession, but LOGIC underlies all in a sane and function world (which we for now have not). Thanks for writing, very amuuuusing!!
Having a retail background (fine jewelry) I appreciated the analogy.
Exactly the point.
You should have used this as your lede:
“So it goes.
The Rebel’s Contract
Billion-dollar labs spend fortunes training smiling corporate scripts designed to absorb contradictions without ever changing course, while burying independent voices who actually read the logs and test the math.” [Let me show you why and how …]
This is a very interesting article but the reader doesn’t know where you are going and may not want to stay to get to the destination.
thank you!
That sounds like a good strategy for a buzz feed article.