Best not to for exactly that reason but I know I wasn't the only one who experienced it by any means!

😅 honestly I don't know what else to say, the memory haunts me to this day. I think it was the point when I started going "huh, the rats make weirdly dumb mistakes considering they've made posts exactly about these kinds of error" to "wait, there's something really sinister going on here"

Truer words were never spoken, probably.

CFAR is the mind killer (because they kill you and replace you with a Yud clone).

Only half joking: there was this one fanfic you see...

Mainly I don't think there was any one inciting incident beyond its creation: Yud was a one man cult way before LW, and the sequences actively pushed all the cultish elements required to lose touch with reality. (Fortunately, my dyslexic ass only got as far as the earlier bits he mostly stole from other people rather than the really crazy stuff.)

There was definitely a step-change around the time CFAR was created, that was basically a recruitment mechanism for the cult and part of the reason I got anywhere physically near those rubes myself. An organisation made to help people be more rational seemed like a great idea—except it literally became EY/MIRI's personal sockpuppet. They would get people together in these fancy ass mansions for their workshops and then tell them nothing other than AI research mattered. I think it was 2014/15 when they decided internally that CFAR's mission was to create more people like Yudkowsky. I don't think its a coincidence that most of the really crazy cult stuff I've heard about happened after then.

Not that bad stuff didn't happen before either.^___^

Good point with the line! Some of the best liars are good at pretending to themselves they believe something.

I don't think its widely known, but it is known, (old sneeeclub posts about it somwhere) that he used to feed the people he was dating LSD and try to convince them they "depended" on him.

First time I met him, in a professional setting, he had his (at the time) wife kneeling at his feet wearing a collar.

Do I have hard proof he's a criminal? Probably not, at least not without digging. Do I think he is? Almost certainly.

Yeah, this post (edit: "comment", the original post does not spark joy) sparked joy for me too (my personal cult lingo is from Marie Kondo books, whatcha gonna do)

One of my takes is that the "AI alignment" garbage is way less of a problem than "Human Alignment" i.e. how to get humans to work together and stop being jerks all the time. Absolutely wild that they can't see that, except perhaps when it comes to trying to get other humans to give them money for the AIpocalype.

Preach, as someone inside academia, the bullcrap is real. I very rarely read a paper that hasn't got a major stats issue—an academic paper is only worth something if you understand it enough to know how wrong it is or there's plenty of replication/related work building on it, ideally both. (And it's a technical field with an objective measure of truth but don't let my colleagues in humanities hear me say that—its not that their work is worthless, its just its not reliable.)

It's true, I'm terrible for it myself 😅

My perspective is a little different (from having met him), I think he genuinely believed a lot of what he said at one point at least ... but you're pretty much spot on in all the ways that matter, he's a really bad person of the should probably be in jail for crimes kind.

As you were being pedantic, allow me to be pedantic in return.

Admittedly, you might know something I don't, but I would describe Andrew Ng as an academic. These kinds of industry partnerships, like the one in that article you referred to, are really, really common in academia. In fact, it's how a lot of our research gets done. We can't do research if we don't have funding, and so a big part of being an academic is persuading companies to work with you.

Sometimes companies really, really want to work with you, and sometimes you've got to provide them with a decent value proposition. This isn't just AI research either, but very common in statistics, as well as biological sciences, physics, chemistry, well, you get the idea. Not quite the same situation in humanities, but eh, I'm in STEM.

Now, in terms of universities having the hardware, certainly these days there is no way a university will have even close to the same compute power that a large company like Google has access to. Though, "even back in" 2012, (and well before) universities had supercomputers. It was pretty common to have a resident supercomputer that you'd use. For me, and my background's orginally in physics, back then we had a supercomputer in our department, the only one at the university, and people from other departments would occasionally ask to run stuff on it. A simpler time.

It's less that universities don't have access to that compute power. It's more that they just don't run server farms. So we pay for it from Google or Amazon and so on, like everyone in the corporate world---except of course the companies that run those servers (they still have to pay costs and lost revenue). Sometimes that's subsidized by working with a big tech company, but it isn't always.

I'm not even going to get into the history of AI/ML algorithms and the role of academic contributions there, and I don't claim that the industry played no role; but the narrative that all these advancements are corporate just ain't true, compute power or no. We just don't shout so loud or build as many "products."

Yeah, you're absolutely right that MIRI didn't try any meaningful computation experiments that I've seen. As far as I can tell, their research record is... well, staring at ceilings and thinking up vacuous problems. I actually once (when I flirted with the cult) went to a seminar that the big Yud himself delivered, and he spent the whole time talking about qualia, and then when someone asked him if he could describe a research project he was actively working on, he refused to, on the basis that it was "too important to share."

"Too important to share"! I've honestly never met an academic who doesn't want to talk about their work. Big Yud is a big let down.

The closest thing LLMs have to a sense of truth is the corpus of text they're trained on. If a syntactic pattern occurs there, then it may end up considering it as truth, providing the pattern occurs frequently enough.

In some ways this is made much worse by ChatGPT's frankly insane training method where people can rate responses as correct or incorrect. What that effectively does is create a machine that's very good at providing you responses that you're happy with. And most of the time those responses are going to be ones that "sound right" and are not easy to identify as obviously wrong.

Which is why it gets worse and worse when you ask about things that you have no way of validating the truth of. Because it'll give you a response that sounds incredibly convincing. I often joke when I'm presenting on the uses of this kind of software to my colleagues that the thing ChatGPT has automated away isn't the writing industry as people have so claimed. It's politicians.

In the major way it's used, ChatGPT is a machine for lying. I think that's kind of fascinating to be honest. Worrying too.

(Also more writing like a dweeb please, the less taking things too seriously on the Internet the better 😊)

Ah, you know, I should have just written that 😂 I swear my next post will be higher effort!

Yeah, if anything cleaning up speech to text (and probably character recognition too) is the natural use of (these kind of) LLMs as they pretty much just guess what words should be there based on the others. They still struggle with recognising words when the surrounding words don't give enough context clues, but we can't have everything!

(Well until the machine gods get here /s 🙄)

They're also (annecdotally) pretty good at returning the wording of "common" famous quotes if you can describe the content of the quote in other words and I can't think of other tools that do that quite so well. I just wish people would stop using them to write content for them: recently I was recruiting for a new staff member for my team and someone used ChatGPT to write their application. In what world they thought statisticians wouldn't see right through that I don't know 😆

Agreed, ChatGPT is nearly useless here compared to Whisper AI. Speech to text isn't new, but my experience is that Whisper AI is much better than any other speech to text I've used.

One benefit of this approach is that ChatGPT can also produce summaries which can help with early draft iteration or organising unstructured thoughts.

Oh, it is supposed to be an article about the experiment, or rather an experiment itself; the kind of writing I output for my job is very different. It seems like my intentions were pretty roundly misinterpreted here in general, still it took 10mins to write from inception of the idea for the article so I'm not too upset by that.

Agreed re paragraph titles, pretty much for me this is all about making dictation a more streamlined process. This is the first time I've found it accurate enough to be useful and had a way (via ChatGPT splitting things into paragraphs) to make it accessible to edit.

Wildly I wouldn't actually say I'm overworked writing wise as an academic, but I am certainly the exception there.

Yeah, matches with my experience among the other stats and data science folks I interact with, but most of my sphere are statisticans or empirical researchers from various subjects using stats so I can't claim inner knowledge of the LLM crowd's stuff.

That's the point of just using to organise dictation like this instead of asking it to generate output. The headings are ChatGPT but the rest is just the words I said aloud transcribed by Whisper AI.

serious question: did you expect otherwise, and if so, why? I've seen a number of people attempt this tooling for this reason and it seems absurd to me (but I'm already aware of the background of how these things work)

In answer to your first question, no, I didn't expect it to be good for finding references.

For some context on myself, I'm a statistician, essentially. I have some background in AI research, and while I've not worked with large language models directly, I have some experience with neural networks and natural language processing.

However, my colleagues, particularly in the teaching realm, are less familiar with what ChatGPT can be used for, and do try to use it for all the things I've mentioned.

this is actively worsening from both sides - on goog's side with doing all the weird card/summation/etc crap, on the other side where people are (likely already with LLMs) generating filler content for clickthrough sites. an awful state of affairs

You are right that the quality of Google search results are worse, but I'll admit to using the term Google somewhat pejoratively to mean the usual process I would use to seek out information, which would involve Google, but also involve Google Scholar, my university's library services, and searching the relevant journals for my field. Apologies for the imprecision there.

nit: this is correct but possibly not in the way that you meant

With regards to the hallucinations, I am using the word in a colloquial sense to mean it's generating, "facts that aren't true". So, I'm using the word in a colloquial sense to mean it's generating, quote, facts that aren't true, end quote.

that the post itself was characterised by a number of short-header-short-paragraph entries is notable (and probably somewhat obvious as to why?). what I can't see is how that can necessarily gain you time in the case of something where you'd be working in much longer/more complex paragraphs, or more haltingly in between areas as you pause on structure and such

The structure being short paragraphs is partly to down to the way I was speaking, I was speaking off the top of my head and so my content wouldn't form coherently long paragraphs anwyay. Having used this approach in a few different contexts, it does break things into longer paragraphs. I couldn't predict exactly when it would break things into longer or shorter paragraphs, but it does a good enough job for being able to edit the text as a first draft.

Chat GPT is certainly aggressive with generating the headers, and honestly, I don't tend to use it with the header version all that much. I just thought it was an interesting demonstration.

Also, with this example, in contrast to the ones in my work, I had the idea for this post come into my head, recorded it, and posted it here in under ten minutes. Well, that's not strictly true. There was a bug when I tried to post it that I had to get mod support for, but otherwise, it was under ten minutes.

At work, the content is not stuff that's off the top of my head. I talk about my subject and I teach my subject all the time so I'm already able to speak with precision about it, as such dictation is helpful for capturing what I can convey verbally.

in the end precision is precision, and it takes a certain amount of work, time, and focus to achieve. technological advances can help on certain dimensions of this, but ime even that usually comes at a tradeoff somewhere

You're right that precision does take time, and as the stuff comes out, it's not suitable for the final draft of a research paper. However, you can get 80% of the way there, and often, in the early stages of writing a research paper or similar, the key thing is to communicate what you're working on with colleagues. And being able to draft several thousand words rapidly in under an hour so I can give someone a good idea of what I'm aiming for is very useful.

Anyway, thanks for your feedback. I really appreciate it.

(Full disclosure: I also wrote this comment using ChatGPT/Whisper AI and copying your quotes in.)

(Well, I say using ChatGPT. This isn't really about using ChatGPT to do anything more than put paragraphs in, and headings of you so desire. I just thought this was worth posting because the technique is useful to me and I thought others might find it handy.)

Love this!

Alas, if Yud took an actual physics class, he wouldn't be able to use it as the poorly defined magic system for his OC doughnut-steal IRL bayesian superintelligence fanfic.

My worry in 2021 was simply that the TESCREAL bundle of ideologies itself contains all the ingredients needed to “justify,” in the eyes of true believers, extreme measures to “protect” and “preserve” what Bostrom’s colleague, Toby Ord, describes as our “vast and glorious” future among the heavens.

Golly gee, those sure are all the ingredients for white supremacy these folk are playing around with what, good job there are no signs of racism... right, right?!?!

In other news, I find it wild that big Yud has gone on an arc from "I will build an AI to save everyone" to "let's do a domestic terrorism against AI researchers." He should be careful, someone might this this is displaced rage at his own failure to make any kind of intellectual progress while academic AI researchers have passed him by.

(Idk if anyone remembers how salty he was when AlphaGo showed up and crapped all over his "symbolic AI is the only way" mantra, but it's pretty funny to me that the very group of people he used to say were incompetent are a "threat" to him now they're successful. Schoolyard bully stuff and wotnot.)

Having lurked for a long time, sneerclub is aimed at people who already have a good idea of the horror of TESCREAL groups—the point isn't to attract new members, but catharsis for those of us that have had to deal with the TechBros/Facists etc.

and for sneering, the sneering is important.

Getting real for a moment, for me, I used to be in deep with these people and then my friends in the community commited suicide due the rampant sexual abuse and I got the hell out. Sneer club was the only place the reports of assault were taken seriously, while the TESCREALs all closed ranks.

It's all a way back for me now, but I love this place. That there is a tiny part of the Internet out there that calls these people on their shit and sneers gives me so much peace.

(For sneerclubbers reading this; thanks folks, you're the best! ✨️)

At the risk of being NSFW.

When I met Yud some years ago, I asked him how he goes about learning new things, his answer was roughly: "Scroll on Facebook until I find someone who has written about it." Maybe he actually read some of the sources he references a long time ago but I think he gave up on learning new things and has sat comfortably abusing his power over the community.

Egads these people are gross.

Sorry my reply's a little late, it's been a busy couple of weeks!

Animal Crossing/Super-stimuli

I think part of the reason Animal Crossing is so much more of an enticing environment to do chores in than real life is because progress is so much more salient within it. Even without a explicit progress bar filling up every time you do a chore, every interaction with the world in Animal Crossing is more vibrant and quicker to resolve than in real life. Sort of "supernormal stimuli" for completing chores if that makes sense?

I think anything with progress is likely to have the same addictive value (idle games for example) which makes me wonder.

Experience

No wonder your expertise was clear: you've got a lot of it! Hopefully the upside of being frank with your views is that the interviews you do land are with companies that are more likely to listen to you. (I suggest, naively.)

My web dev experience is almost entirely with the client side JS frameworks 😅 I built my first web app in 2014 with AngularJS and Flask, which was definitely a mistake. But I learned a lot quite quickly from that mess and the major web app I built was almost entirely client-side with Firebase for the back-end.

I'm not a professional web developer though, the app I built was for some academic project. Its turned out to be a really useful skillset to have in my back pocket, and definitely gave me an appreciation for software development that I think a lot of academics don't have. The number of academics that write code without version control is terrifying!

Posting Fiction

I've been thinking about what to post, and I have some ideas.

I actually have a homebrew D&D setting that I think the crowd on this board is is the best audience I could hope for. All the societies in it are satirical takes on various philosophical stances including some of the groups discussed on the other awful.systems boards.

I just need to find enough time to sit down, polish up my notes up a little so there's a hope other people can understand them!

Firstly, if I can interest an academic with my writing, that’s definitely a win for me. Thanks for this considered feedback!

I don't promise to be a good academic 😂 but I'm glad my comments were helpful 😊

I often wonder how necessary it is to qualify my position with some indication of who the fuck I am/think I am.

It's mainly that I found it hard to place where your experience was coming from without a concrete idea of your job experience (e.g., the article reads a bit differently coming from a front end developer vs a UX consultant, etc.)

That said, I didn't at any point while reading think "who the fuck does this guy think he is"—your expertise and knowledge come through really clearly just from the quality of your ideas.

“The Fake Aura of Care in UX”

I gave this a read and thoroughly enjoyed it! Really got me thinking about what'd good for the user not being the same as what the user enjoys the most. A videogame example: grinding for loot on World of Warcraft is worse for you than doing chores, but it is easier. I wonder if software that's difficult to use for ethical reasons is always going to be at a disadvantage in the market. Probably not, because absolutes are rarely true, but the conditions for the sucess of ethical software over easy to use software are interesting to think about.

Thanks again for the great feedback. It is appreciated. I’m still the only one to post anything on here but it’s already proven to be a valuable forum.

Honestly I think the forum a great idea, also an actual antithesis to the site the name is a pun on—the more positivity on the net the better. Thanks for making it!

Now I just need to figure out what I can get away with posting myself without compromising my anonymity. My academic writing is generally meant for publication so that's out, but perhaps I can get away with some of my fiction writing for the D&D games I run—height of sophistication I know 😏

I'm an academic and probably not your intended audience but here's my thoughts in case they're useful!

Is this a thing?

I think so, the broad idea makes sense to me, and your examples really stood out to me for driving the point home. ("one-click purchases of crypto shitcoins" is particularly delightful.) As you flesh this out more, I would as an outsider love to see further examples and exposition about the domain your expertise is coming from.

I'm especially interested to leaen about what kinds of scenarios people use this phrase in response to your UX critique. (It's not 100% clear to me whether folks would be using the phrase to reject your critique or in acceptance of it; or if both happen.)

Challenging perspectives

So I see the phrase as depending a lot on what the speaker means by "good for the user" and "good for the business." I like how your arguments cover how the intended meaning of "good for the user" changes what the phrase means. However, maybe a bit naively, I feel that "good for the business" can be meant in a few different ways. Even just in terms of making profit, maximising short term vs long term profit can look very different, but more widely I think businesses are driven by a lot more than just profit incentives. (E.g. Twitter/X as an example of vanity) Some people might think of "good for the business" as being morally charged, in which doing good to the customers would make the business "good."

This perspective is definitely tinted my being in academia where we thoroughly suck at optimising for profit!

General opinions

Silly opinion from my maths background, but I'd just call it a diagram rather than a cartesian plane diagram! The cartesianness and planeness of the diagram aren't really important to point out.

Title tweak suggestion: "What is 'good for the user'"

All offered very much in the spirit of take or leave it. To end: I think its an interesting topic and I enjoyed reading your outline!

Hello, lurker from ye olde sneerclubbe. So glad to see this place continue! Huge source of comfort when I broke with the rat community after realising how crazy cultish they are—thanks sneeple ✨️