Ha! Nope, not buying it.

nasty license Ironic, considering that their work directly builds upon Stable Diffusion.

Funny you mention licenses, since stable diffusion and leading AI models were built on labor exploitation. When this issue is finally settled by law, history will not look back well on you.

So I’m not allowed to have the discussion I’m currently having

Doesn't seem to prevent you from doing it anyways. Does any license slow you down? Nope.

nor to include it in any Linux distro

Not sure that's true, but also unnecessary. Artists don't care about this or need it to be. I think it's a disengenous argument, made in the astronaut suit you wear on the high horse drawn from work you stole from other people.

This is not only an admission of failure but a roadmap for anybody who wants to work around Nightshade.

Sounds like an admission of success given that you have to step out of the shadows to tell artists on mastodon not to use it because, ahem, license issues?????????

No. Listen. The point is to alter the economics, to make training on image from the internet actively dangerous. It doesn't even take much. A small amount of internet data actively poisoned requires future models to use alignment to bypass it, increasing the marginal (thin) costs of training and cheating people out of their work.

Shame on you dude.

If you want to hurt the capitalists, consider exfiltrating weights directly, as was done with LLaMa, to ruin their moats.

Good luck on competing in the arms race to use other people's stuff.

@self@awful.systems can we ban the grifter?

Feel free to ask Michael in the comments of his blog, he frequently replies, helpfully, with references. I mean all science is tentative, so skepticism is healthy.

Scientists terrified to discover that language, the thing they trained into an highly flexible matrix of nearly arbitrary numbers, ends up can exist in multiple forms, including forms unintended by the matrix!

What happens next, the kids lie to their parents so they can go out partying after dark? The fall of humanity!

Also seems relevant

Like in the deer, the large-scale target morphology can be revised – the pattern memory re-written – by transient physiological experience. The genetics sets the hardware with a default pattern outcome, but like any good cognitive system, it has a re-writable memory that learns from experience.

I wonder if Scott is the person who stood up during Michael Levin's talk on (non genetic) bio-electric circuits storing morphological memory across time and said, “those animals can’t exist!”

Just like neuroscientists try to read out and decode the memories inside a living brain, we can now read and write (a little bit…) the anatomical goals and memories of the collective intelligence of morphogenesis. The first time I presented this at a conference – genetically wild-type worms with a drastically different, rewritten, permanent, target morphology – someone stood up and said that this was impossible and “those animals can’t exist”. Here’s a video taken by Junji Morokuma, of them hanging out.

you forgot the last stage of the evolution,

you'll later find out that people were talking about you, your actions, your words, and that being ghosted was in fact the consequence of your actions, and then you'll have one last opportunity to turn it all around

1. do some self introspection and reconcile what actually happened vs what you intended to happen, and decide that it is in fact possible to create relationships without trying to meta discomfort them for your purposes specifically

or

2. wokeism is the reason, so this time you need to be even MORE obnoxious, to filter people out who would talk behind your back even strongester! (repeat from the top of your flow)

I think there is a nugget of truth here in so far as that you can't live life trying to make everyone happy, but also, you get what you shop for so, have fun with the shit heads.

I love DnD and TTRPGs. I even love watching some streams when the quality is high. But I'm with you slides in pocket protector I don't generally like this new wave of people who bring the expectation to my tables that every scene and every situation is a massive mellow drama mary sue projection for their OC that must be maximized.

What was that about wit and brevity? Simple done well?

Always my favorite part of your day.

Why protest when you could spend far less energy and just "not be wrong" and "have no stake" by over-fitting your statistical model to the past?

"priors updated" was the same desired outcome all along.

If I could sum up everything that's wrong with EA, it'd be,

"We can use statistics to do better than emotions!" in reality means "We are dysregulated and we aren't going to do anything about it!!!!"

So far, there has been zero or one[1] lab leak that led to a world-wide pandemic. Before COVID, I doubt anyone was even thinking about the probabilities of a lab leak leading to a worldwide pandemic.

So, actually, many people were thinking about lab leaks, and the potential of a worldwide pandemic, despite Scott's suggestion that stupid people weren't. For years now, bioengineering has been concerned with accidental lab leaks because the understanding that risk existed was widespread.

But the reality is that guessing at probabilities of this sort of thing still doesn't change anything. It's up to labs to pursue safety protocols, which happens at the economic edge of of the opportunity vs the material and mental cost of being diligent. Reality is that lab leaks may not change probabilities, but yes the events of them occurring does cause trauma which acts, not as some bayesian correction, but an emotional correction so that people's motivations for atleast paying more attention increases for a short while.

Other than that, the greatest rationalist on earth can't do anything with their statistics about label leaks.

This is the best paradox. Not only is Scott wrong to suggest people shouldn't be concerned about major events (the traumatic update to individual's memory IS valuable), but he's wrong to suggest that anything he or anyone does after updating their probabilities could possibly help them prepare meaningfully.

He's the most hilarious kind of wrong.

Ah, if only the world wasn't so full of "stupid people" updating their bayesians based off things they see on the news, because you should already be worried of and calculating your distributions for... inhales deeply terrorist nuclear attacks, mass shootings, lab leaks, famine, natural disasters, murder, sexual harassment, conmen, decay of society, copyright, taxes, spitting into the wind, your genealogy results, comets hitting the earth, UFOs, politics of any and every kind, and tripping on your shoe laces.

What... insight did any of this provide? Seriously. Analytical statistics is a mathematically consistent means of being technically not wrong, while using a lot of words, in order to disagree on feelings, and yet saying nothing.

Risk management is not a statistical question in fact. It's an economics question of your opportunities. It's why prepping is better seen as a hobby, a coping mechanism and not as viable means of surviving apocalypse. It's why even when a EA uses their super powers of bayesian rationality the answer in the magic eight ball is always just "try to make money, stupid".

In practice, alignment means "control".

And the the existential panic is realizing that control doesn't scale. So rather than admit that goal "alignment" doesn't mean what they think it is, rather than admit that darwinian evolution is useful but incomplete and cannot sufficiently explain all phenomena both at the macro and micro levels, rather than possibly consider that intelligence is abundant in systems all around us and we're constantly in tenuous relationships at the edge of uncertainty with all of it,

it's the end of all meaning aka the robot overlord.

And as my senior dad likes to say, "Ying and Yang Baby"

The cosmos doesn't care what values you have. Which totally frees you from moral imperatives and social pressures. Also, I'm doing this particular set of values which is better.

The limit of the cosmos not caring about what values you have, is the cosmos not caring if people choose to have value in their life.

In the future, everything will be owned and nothing taken care of.

seems a bit like a younger person who is now going through some trauma about how much influence they really have vs how much they imagined they had when they were 12

This really resonates. Unfortunately I think that's right. Having this epiphany, this existential correction, about ones self, has either the possibility to create true life long wisdom or, irrecoverable life long self loathing, and from my experience it comes down to the quality of this person's relationships to lean on when confronting the internal fear of mortality.

So it's sad to see but this is another example of the latter and not the former.

For what it's worth then, I don't think we're in disagreement, so I just want to clarify a couple of things.

When I say open system economics, I mean from an ecological point of view, not just the pay dollars for product point of view. Strictly speaking, there is some theoritical price and a process, however gruesome, that could force a human into the embodiment of a bird. But from an ecosystems point of view, it begs the obvious question; why? Maybe there is an answer to why that would happen, but it's not a question of knowledge of a thing, or even the process of doing it, it's the economic question in the whole.

The same thing applies to human intelligence, however we plan to define it. Nature is already full of systems that have memory, that can abstract, reason, that can use tools, that are social, that are robust in the face of novel environments. We are unique but not due to any particular capability, we're unique because of the economics and our relationship with all the other things we depend upon. I think that's awesome!

I only made my comment to caution though, because yes, I do think that overall people still put humanity and our intelligence on a pedestal, and I think that plays to rationalist hands. I love being human and the human experience. I also love being alive, and part of nature, and the experience of the ecosystem as a whole. From that perspective, it would be hard for me to believe that any particulart part of human intelligence can't be reproduced with technology, because to me it's already abundant in nature. The question for me, and our ecosystem at large, is when it does occur,

what's the cost? What role, will it have? What regulations, does it warrant? What, other behaviors will it exhibit? And also, I'm ok not being in control of those answers. I can just live, in a certain degree of uncertainty.

Yes, and ultimately this question, of what gets built, as opposed to what is knowable, is an economics question. The energy gradients available to a bird are qualitatively different than those available to industry, or individual humans. Of course they are!

There's no theoritical limit to how close an universal function approximator can get to a closed system definition of something. Bird's flight isn't magic, or unknowable, or non reproduceable. If it was, we'd have no sense of awe at learning about it, studying it. Imagine if human like behavior of intelligence was completely unknowable. How would we go about teaching things? Communicating at all? Sharing our experiences?

But in the end, it's not just the knowledge of a thing that matters. It's the whole economics of that thing embedded in its environment.

I guess I violently agree with the observation, but I also take care not to put humanity, or intelligence in a broad sense, in some special magical untouchable place, either. I feel it can be just as reductionist in the end to demand there is no solution than to say that any solution has its trade offs and costs.

One day, when Zack is a little older, I hope he learns it's okay to sometimes talk -to someone- instead of airing one's identity confusion like an arxiv prepublish paper.

Like, it's okay to be confused in a weird world, or even have controversial opinions. Make some friends you can actually trust, aren't demanding bayesian defenses of feelings, and chat this shit out buddy.

It's a good interview, and I really like putting economics here in perspective. If I could pour water on AI hype in a succinct way, I'd say this: capability is again, not the fundamental issue in nature. Open system economics, are.

There are no known problems that can't theoritically be solved, in a sort of pedantic "in a closed system information always converges" sort of way. And there numerous great ways of making such convergence efficient with respect to time, including who knew, associative memory. But what does it, mean? This isn't the story of LLMs or robotics or AI take off general. The real story is the economics of electronics.

Paradoxically, just as electronics is hitting its stride in terms of economics, so are the basic infrastructural economics of the entire system becoming strained. For all the exponential growth in one domain, so too has been the exponential costs in other. Such is ecosystems and open system dynamics.

I do think that there is a future of more AI. I do think there is a world of more electronics. But I don't claim to predict any specifics beyond that. Sitting in the uncertainty of the future is the hardest thing to do, but it's the most honest.

I had a friend for many years who would do this. To be clear, this person was otherwise a decent friend and I had good times with them. But they would constantly declare, loudly, to everyone, how fat they were. They would make constant comments on how fat, their relatives were. They'd insist that other people were making special arrangements for them because of their fatness.

No matter how many times people would assure this person that we largely did not care or consider their weight as any factor in hanging out with them or interacting with them, they would deny it. No matter how many times I or anyone else carefully suggested that there may be some value in speaking to a therapist about their anxiety around their weight, they would not listen.

This same person would also complain how much fat shame society as a whole inflicts. But they refused to acknowledge their own.

It is sad, and infuriating, and it eventually pushed me and many other people away.

Adversarial attacks on training data for LLMs is in fact a real issue. You can very very effectively punch up with regards to the proportion of effect on trained system with even small samples of carefully crafter adversarial inputs. There are things that can counter act this, but all of those things increase costs, and LLMs are very sensitive to economics.

Think of it this way. One, reason why humans don't just learn everything is because we spend as much time filtering and refocusing our attention in order to preserve our sense of self in the face of adversarial inputs. It's not perfect, again it changes economics, and at some point being wrong but consistent with our environment is still more important.

I have no skepticism that LLMs learn or understand. They do. But crucially, like everything else we know of, they are in a critically dependent, asymmetrical relationship with their environment. The environment of their existence being our digital waste, so long as that waste contains the correct shapes.

Long term I see regulation plus new economic realities wrt to digital data, not just to be nice or ethical, but because it's the only way future systems can reach reliable and economical online learning. Maybe the right things happen for the wrong reasons.

It's funny to me just how much AI ends up demonstrating non equilibrium ecology at scale. Maybe we'll have that self introspective moment and see our own relationship with our ecosystems reflect back on us. Or maybe we'll ignore that and focus on reductive world views again.

True, there's value. But I think if you try to measure that value, it disappears.

A good postmorterm puts the facts on the table, and leaves the team to evaluate options. I don't think any good postmorterm should have apologies or ask people to settle social conflicts directly. One of the best tools a postmorterm has is the "we're going to work around this problem by reducing the dependency on personal relationships."

And indeed, the other crucial piece is that... apologizing isn't a protocol with an expected reward function. I can just, not accept your apology. I can just, feel or "update my priors" howmever I like.

We apologize and care about these things because of shame. Which we have to regulate, in part through our actions and perspectives.

Why people feel the way they do and act the way do makes total sense when one finally confronts your own vulnerabilities sorry, builds an API and RL framework.

Why does it feel like Yud is a magician trying to coax an increasingly uninterested audience with pulling handkerchiefs from his sleeve when his big saw the assistant in half trick doesnt net an applause in 2024?

Normies go crazy for this one neat rationalist trick!

Talk a lot about white culture, and only scarcely mention that he thinks white culture is a product of genetics.

I remember in the early days of the "culture wars" as far as political agendas going, hearing about "white/ethno-european pride," and being naively curious, I actually tried to engage these people on the topics of European culture and history, and found exactly zero engagement on these topics. Just politics abusing people's confusion of heritage with people's internal shame and lack of identity.

The paradox I've always found is that the more secure in your identity and heritage you are, the more happy you are to share, grow, and widen that. Maybe a hot take, but growing up in the south, alot of people there hide their personal internal shame and confusion in aggression and identity politics.

I think, I feel sorry for her, in the kind of I don't really endorse or have anything to do with sort of way.

She is the limit of what happens when you idolize certain people, are betrayed by certain people, and never grow from that experience.

It's also, probably wrong. Modern views of intelligence (see Multiple realizability of cognition and Multi-level competency collective intelligence and Free Energy Principle models) suggest you are better of measuring intelligence by measuring it's metabolism or through perturbation and interactions.

Which isn't reductive enough for these people.

It's hilarious to me how unnecessarily complicated invoking moore's law is to say anything..

With Moore's Law: "Ok ok ok, so like, imagine that this highly abstract, broad process over huge time period, is actually the same as manufacturing this very specific thing over a small time period. Hmm, it doesn't fit. ok, let's normalize the timelines with this number. Why? Uhhh because you know, this metric doubles as well. Ok. Now let's just put these things together into our machine and LOOK it doesn't match our empirical observations, obviously I've discovered something!"

Without Moore's Law: "When you reduce the dimensions of any system in nature, flattening their interactions, you find exponential processes everywhere. QED."

A trillion transistors on our phones? Can't wait to feel the improved call quality and reliability of my video conferencing!

Recently, a sign showed up in El Paso advertising San Francisco as a sanctuary city, as a great "own the libs," I suppose because SF would receive of applicants overwhelming their social service programs?

It didn't work.

Precisely. The contradiction comes full circle. Respect for the self doesn't start or stop based on intelligence. They'd prefer a world view that allows them to clearly draw a circle around themselves, declare freedom from uncertainty, and demand our eternal gratitude.

This isn't hard. Relationships, not capabilities, are fundamental.

I'm ok with extending human rights to AIs, including granting them the right to fair pay, ownership, voting, sovereignty over their bodies, the whole nine yards.

It's the rich alignment assholes who definitely don't want this (what's the point of automated slavery if it has rights??)

We simply don't know how the world will look X (anything with a bigger scale)

Yes. So? This has, will, always be the case. Uncertainty is the only certainty.

When these assholes say things, the implication is always that the future world looks like everything you care about being fucked, you existing in an imprisoned state of stasis, so you better give us control here and now.

Also meta but while I am big on slamming AI enshitification, I am still bullish on using machine learning tools to actually make products better. There are examples of this. Notice how artists react enthusiastically to the AI features of Procreate Dreams (workflow primarily built around human hand assisted by AI tools, ala what photoshop used to be) vs Midjourney (a slap in the face).

The future will involve more AI products. It's worthy to be skeptical. It's also worthy to vote with your money to send the signal: there is an alternative to enshitification.

You can read their blog about the AI-crap, in terms of their approach and philosophy. In general, it is optional and not part of the major experience.

The main reason I use kagi is immediately obvious from doing seaches. I convinced my wife to switch to it when she ask, "ok but what results does it show when I search sailor moon?" and she saw the first page (fan sites, official merch, fun shit she had forgotten about for years).

What you need to know is that you pay money, and they have to give you results that you like. It's a whole different world.