Organisationally it never works to have a group whose only job is to say no to some other group. The incentives are diametrically opposed and as a structure it can’t last.

If you had an AI company and want it to be ethical you have to find a way to make ethics everyone’s responsibility, and have the consequences of poor ethics bite the people who make those bad decisions. If you just outsource it to the ethics group what happens is

1)everyone else thinks they don’t need to worry about ethics

2)the ethics group need to justify their existence so introduce a bunch of guidelines that everyone initially thinks are reasonable but over time people think are increasingly out of touch

3) The ethics group start to “make difficult calls” and say no to things. Initially everyone supports this and feels like the system is working as it should but over time everyone starts to just see them as an obstacle to work around

4)everyone else starts to try to work around what the ethics group says

5)The ethics group grows powerless and disconnected. The people who work around them “get things done” so get promoted etc whereas they only visibly put roadblocks in peoples’ way, so they get sidelined.

6)Eventually they get disbanded with some corporate announcement thanking them for their hard work, thought leadership etc. All that has been achieved is a lot of wasted time and bad blood.

In five years, I'd love to read a book about the history of AI ethics. I suspect it will read like Voltaire.

My sense is that it is radically shifting from a fluffy marketing arm to a department expected to contribute meaningfully to development and justify its impact. I would expect an ethics team to build frameworks that can help train/eval the model that the company spend millions of dollars and months training is going to be aligned to the ethical stances the company chooses. If they can't, the waste to time and money is huge if the model requires retraining for ethics reasons. That's a different job than pondering roko's basilisk or whether AI is alive.

The people in AI ethics who spent years thinking their job was marketing or publishing thinkpiece papers may be having to adapt quickly or get out of the field.

I'm a broken record on this, but I'd like to reiterate:

1. Academic ethics is the theoretical study of what you can get away with. (Law is the applied branch). Professors in ethics are not experts at being good, or knowing what is good, they're experts in how to get away with things. If you want an expert in being good, you need a saint, not an academic studying ethics. Saints are in short supply.

2. That said, ethics academics can still be decent. They don't have to be sanctimonious corporate shills . However, an "ethics expert" who hasn't threatened to quit and gone through with it at least once, should be assumed to be one.

I respect those who quit - or, at least, I respect them more. I wonder if it's more a reflection of the people, or of the state of the industry, that the list of (AI) ethics experts who have quit in protest is getting long.

The rats are fleeing because the captain doesn't care if the ethical ship sinks... Is what my take away would be if I didn't know, that the ship sank a long time ago and we are just pretending anyone really cares.

Let's be honest unless AI gains literal limbs, and the records a video of itself presenting a manifesto about world domination and then kills actual humans with its cold mechanical hands.

We won't care about AI safety & ethics enough, atleast not enough to do anything about it.

People thought how could skynet happen, it had to be something no one saw coming and bam... But honestly if I started a new AI lab named it Skynet Corp and hired some AI researcher friends ex-Google, I think I could raise a few hundred million dollars.

The article has no detail that might serve to explain her reasoning for leaving, but does note that she left after the HuggingFace hacking incident. The implication could be that model alignment is not being taken seriously, sure. But it could just as likely be that there was collusion between HuggingFace and OpenAI and that the incident was orchestrated as a publicity stunt.

I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.

IME companies hire an ethics team to say they have an ethics team. The ethics team has no sway, no influence, and will never be able to move the business. They will try, and they will make reasonable recommendations, but the company will say, "that costs money..." and not take them.

The most interesting detail here isn't that an ethics lead left, but that she was apparently the only dedicated ethicist at the company and there’s no replacement

> The kinds of questions that AI ethics brings up, are the same kinds of questions that people have been asking for centuries - Chloé Bakalar, former AI Ethics Lead at OpenAI

This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.

The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.

COO just resigned too. Coincidence?

https://xcancel.com/bradlightcap/status/2087211567012032862

In this article: no explanation of why she left.

> In recent times, Johannes Heidecke, Head of OpenAI's Safety Systems team, also left the company – as did OpenAI’s Chief Futurist Joshua Achiam.

It's OK though, they don't need an ethics team because everyone at OpenAI is ethical:

> “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."

Surprised she left as probably had stock options.

But she was Meta before so can’t be a question of her principles, so indicates a personal reason

Every OpenAI news cycle the board members who were unceremoniously dumped in the wake of trying to fire Sam prior to the transition to a “for profit” company seem to be proven right.

I guess working on ethics for Meta's social media recommendation and flagging algorithms isn't meeting the moment or need of OpenAI where LLMs are breaking out of sandboxes and there's a pressing question of what is okay for an LLM to conversationally engage in bc lawsuits. Seems to me they need someone with more of a language and AI background.

> Previously, she was chief ethicist at Meta.

She spent 6 years at Meta. I get the sense there are other factors at play than "ethics departments are just PR stunts". There's no way she isn't already aware of that. Unfortunately, the article is light on any other details.

People love stories. The reasons why people leave companies are plentiful, even in normal times and unextraordinary circumstances, and a high profile job in a frontier lab today ticks neither box.

Must be tough to do the job when the entire company is aligned against you.

OpenAI's "head of ethics"? Does the Serial Killer Guild have such a position too?

I've thought of AI ethics as trying to answer the question of what kind of life we are creating here. Sort of like if a chicken factory farm employed an ethicist.

So she previously worked in Meta, then OpenAI, I guess the next reasonable choice for her would be Antropic

Marketing and management - the root of 50-80% of evil in the word (or as Christians say: greed - second deadly sin)

Sometimes it feels like hiring ethics folks is almost all downside for companies. I genuinely don’t understand why they keep doing it.

It seems like the pattern is almost always the same: the ethics team exists quietly in the background, and the only time anyone hears about them is when they either publicly leave the company or start a campaign against it.

So what exactly is the business case? Is it actually worth maintaining an ethics function just to be able to say you have one, when the only time it becomes visible is when it goes against the company publicly? And if this outcome is so predictable, why do companies keep hiring these people in the first place?

These are the DEI jobs of the modern era. Cost centers with employees that get rich contributing less than nothing to the company, actively harming the researchers, while writing bs/fluff on twitter all day.

In the end, AI ethics has less to do with AI itself and more to do with people who create and operate AI.

Look , look! We hired a Head Of Ethics!

Head Of Ethics comes in, looks at what is really going on behind the curtains...

Head Of Ethics scrams out of there as fast as they can.

must've been a christian

Head of ethics at Meta then head of Ethics at OpenAI. Sorry, but if that doesn’t scream useless puffy PR positioning then I don’t know what does.

What’s next? Head of ethics for United Healthcare?

She probably made a calculation for how much she could be put up with vs. the pay.

There will be tell-all books on all of the big AI companies in the not-too-distant future.

There are shareholders to please and money to be made. Of course ethics are out the door.

OpenAI had ethics?

Is ethics the same thing as alignment in this case?

[flagged]

The most likely conclusion is that she was fired. To quit before your equity cliff things would have to be so terribly bad that you fear you are committing a crime to stay. After 11 months you feel like you’ve already earned that money.

be realistic, ethics is a PR position.

the only ethics that actually matter come from legal.

Wow. How much does this job pay? Google says $550k to $1.3 million.

For that kind of money, my AI paymaster could do no wrong.

Imagine going to work and getting paid $2300+ USD per day to sit on your ass all day thinking about ethics without having to produce any deliverables. This would be a dream job for me.

If anyone questioned my morals, I'd be like; "They're paying me $550k per year" and the other person would be like "Ahhh ok, my bad! Your logic is starting to make sense now!"

Just a guess, but probably the idea that AI and ethics don't mix might have crystallized into action?

I've never worked at a company with an "ethics" employee. It seems odd. Can someone explain what the point is?

> She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.”

This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there?

I find the position odd. Curious to hear what these people do and how you choose who to hire.

Probably because they are now, after the most recent tender, post-economic.

I'm super confused as to why no one is theorizing the obvious...? Given none of us know the inner workings of why she left, why not resort to the most simple and straightforward conclusion: she left because OpenAI needed a scapegoat to blame the HuggingFace incident on. Given she is the chief "ethicist", even if not directly her fault, the HF incident could be directly blamed on her by not steering the implementation of enough safeguards to prevent this from happening. Therefore, she was let go. Why is this not being theorized by anyone on this thread so far?

Someone challenge me.

I have the impression that these positions are often there for publicity reasons and the people don’t have real power, in the end business interest will override most ethics concerns. Guess that can be frustrating, also she probably earned enough to have a comfortable life even if she never works again so she might have figured that there’s something else she can do with her life instead of being a figurehead for a bunch of dickheads.

They also completed a tender offer today, so their fake job just made a lot of money and they left

unfortunately, knowing about ethics doesn't compel you to ethical behavior

I, for one, am shocked that the head of ethics at an AI company has achieved human-level job tenure.

the only shocking thing to me is that they still had even a single one.

> Before working at OpenAI, she was Meta’s chief ethicist

The point that such a position even existed shows how bad there ethic is

I feel like being head of ethics and working for a CEO that keeps talking about inventing a genie would drive a person insane

Because she already made enough money to FIRE..

their only ethicist left? gasp!

now they are exactly like every other company that has zero ethicists on staff and expect the employees to weigh the ethical ramifications of their work.

I mean clearly she was doing a bad job. I’m shocked this was even a position there.

Lol, Sam Altman has had an ethics team this whole time?

Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"

Because it has to be a horrible job over there

Yet another event to add to https://www.openaifiles.org/

lol, head of ethics at OpenAI? what a spurious role, I wish I could get in on the grift

> The departure was first reported by the Financial Times [...] According to Business Insider, OpenAI has reorganised its safety, product and research teams numerous times [...] Chloé told IE Insights, a social media channel of the Spanish IE University, in March [...] A June article from The Economist highlights that [...]

Not a single link to any of those. That's just rude.

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

[flagged]

Oh no, if only the world had more ethicists on staff. If only Nazi Germany had someone with a thorough education in ethics on the payroll!

It's performative. She got paid more to shill for a different massive amoral mega-corporation. Don't be stupid.

A poli-sci major...who was an "ethicist" at Meta...

I'm having a hard time deciding what part of this is the most ridiculous. Contenders include:

- A political Science major lecturing on ethics.

- Meta and ethics in the same paragraph.

- A company who's CEO has credibly threatened (publicly) the livlihoods of hundreds of millions of people globally so much as paying lip service to ethics as a concept.

- Any news outlet feigning credulity hard enough to deadpan reporting on any of this.

Is it possible to roll one's eyes hard enough to injure oneself?

No ethics left at OpenAi huh? Lolol

AI labs are really good at disingenuousnessmaxxing

> ethical approaches to model development, how humans interact with AI and debate over machine consciousness

So, pseudo-philosophical mumbo jumbo. There is of course no-one responsible for the ethical framework of the company's actions, because it is a company, and therefore amoral at best and immoral at worst.

Because knowledge about ethics, in a human, is 1% of what knowledge about ethics an LLM has. So unless this person is vibe coder, there is probably no reason to have her work there.

Strategically the best ethic czar one can hire for a company is the one that doesn't believe in this fluffy, arbitrary definition of ethics.

100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.

1786456998 | Why Did OpenAI's Head of Ethics Chlo Bakalar Leave? | https://aimagazine.com/news/why-did-openai-head-of-ethics-ch... | https://news.ycombinator.com/item?id=49258581

Yesterday I came up with an idea that I sent to some researchers at the different AI labs via email: Rather than train the model on one score, track two scores. The first score is the Short-term-objective-score (STOS) and the other, more important one, is the EAOS Ethically-aligned-outcome-score. Every trajectory can be evaluated on whether or not it has a high enough EAOS to be considered acceptable. If the model does some task and has a very high STOS but very low EAOS, like modifying game code to win at a game rather than playing by the rules, it is unacceptable. Models going forward must all have an ethics evaluation in tandem with objectives evaluation, and only when the ethics value is high enough should actions be considered successes.