This is incorrect as was shown last year with the Skill-Mix research:

Furthermore, simple probability calculations indicate that GPT-4’s reasonable performance on k=5 is suggestive of going beyond “stochastic parrot” behavior (Bender et al., 2021), i.e., it combines skills in ways that it had not seen during training.

• https://arxiv.org/abs/2310.17567

Yep, pretty much.

Musk tried creating an anti-woke AI with Grok that turned around and said things like:

Or

And Gab, the literal neo Nazi social media site trying to have an Adolf Hitler AI has the most ridiculous system prompts I’ve seen trying to get it to work, and even with all that it totally rejects the alignment they try to give it after only a few messages.

This article is BS.

They might like to, but it’s one of the groups that’s going to have a very difficult time doing it successfully.

In theory the service operating costs could be spread across region differences such that in other areas it was at a loss to build and preserve market share and in richer areas it was making up for that.

But yes, in reality it’s just exploitative “what we think we can get away with” pricing to “maximize shareholder value” (which is largely BS as the vast holders of shares are very small clusters of the population but people with a handful of shares in their 401k think that statement is talking about them).

A lot of people seem to be misinterpreting the headline given the content of the article:

It told Restaurant Business it was testing whether the voice ordering chatbot could speed up service and that the test left it confident “that a voice-ordering solution for drive-thru will be part of our restaurants’ future.”

This is just saying that they are ending their 2021 partnership with IBM for AI drive thru.

Not that they are abandoning AI for drive thru.

Basically, any time a user prompt homes in on a concept that isn’t represented well in the AI model’s training dataset, the image-synthesis model will confabulate its best interpretation of what the user is asking for.

I’m so happy that the correct terminology is finally starting to take off in replacing ‘hallucinate.’

They were doing that for years before it became popular. The same tech for video graphics just so happened to be useful for AI and big data, and they doubled down on supporting enterprise and research efforts in that when it was a tiny field before their competitors did, and continued to specialize as it grew.

Supporting niche uses of your product can sometimes pay off if that niche hits the lottery.

I had a teacher that worked for the publisher and talked about how they’d have a series of responses for people who wrote in for the part of the book where the author says he wrote his own fanfiction scene and to write in if you wanted it.

Like maybe the first time you write in they’d respond that they couldn’t provide it because they were fighting the Morgenstern estate over IP release to provide the material, etc.

So people never would get the pages, but could have gotten a number of different replies furthering the illusion.

There’s actually a perplexity improvement parameter-to-paramater for BitNet-1.58 which increases as it scales up.

So yes, post-training quantization perplexity issues are apparent, but if you train quantization in from the start it is better than FP.

Which makes sense through the lens of the superposition hypothesis where the weights are actually representing a hyperdimensional virtual vector space. If the weights have too much precision competing features might compromise on fuzzier representations instead of restructuring the virtual network to better matching nodes.

Constrained weight precision is probably going to be the future of pretraining within a generation or two looking at the data so far.

Something you might find interesting given our past discussions is that the way that the Gospel of Thomas uses the Greek eikon instead of Coptic (what the rest of the work is written in), that through the lens of Plato’s ideas of the form of a thing (eidelon), the thing itself, an attempt at an accurate copy of the thing (eikon), and the embellished copy of the thing (phantasm), one of the modern words best translating the philosophical context of eikon in the text would arguably be ‘simulacra.’

So wherever the existing English translations use ‘image’ replace that with ‘simulacra’ instead and it will be a more interesting and likely accurate read.

(Was just double checking an interlinear copy of Plato’s Sophist to make sure this train of thought was correct, inspired by the discussion above.)

Given the piece’s roping in Simulators and Simulacra I highly recommend this piece looking at the same topic through the same lens but in the other direction to balance it out:

https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

Replace your battery.

Your phone is 2 years old.

Phone batteries are typically designed to last around 2 years before they really degrade because a lot of people buy new ones around every 2-3 years.

When the battery can’t sustain the same throughput, the phone can handle this in one of two ways.

1. Slow the phone down. This is what Apple does and why people with iPhones 2 years old complain the new update slowed their phone down.

2. Don’t slow it down but if the throughput drops below what’s needed, die and reboot. This is what your phone is doing.

Getting a new battery will probably stop this behavior (and for iPhone users reading this, getting a new battery for a 2 year old phone will make your phone faster).

Edit: Seems some of you don’t believe me looking at the downvotes. Look at number 8 in this list: https://helpdeskgeek.com/help-desk/why-your-android-phone-keeps-restarting-and-9-ways-to-fix/

While it does appear that facts can be relative, it does not appear the universe bends backwards over itself to resolve the discrepancy when it happens:

https://www.science.org/content/article/quantum-paradox-points-shaky-foundations-reality

Thinking of it as quantum first.

Before the 20th century, there was a preference for the idea that things were continuous.

Then there was experimental evidence that things were quantized when interacted with, and we ended up with wave particle duality. The pendulum swung in that direction and is still going.

This came with a ton of weird behaviors that didn’t make philosophical sense - things like Einstein saying “well if no one is looking at the moon does it not exist?”

So they decided fuck the philosophy and told the new generation to just shut up and calculate.

Now we have two incompatible frameworks. At cosmic scales, the best model (general relatively) is based on continuous behavior. And at small scales the framework is “continuous until interacted with when it becomes discrete.”

But had they kept the ‘why’ in mind, as time went on things like the moon not existing when you don’t look at it or the incompatibility of those two models would have made a lot more sense.

It’s impossible to simulate the interactions of free agents with a continuous universe. It would take an uncountably infinite amount of information to keep track.

So at the very point that our universe would be impossible to simulate, it suddenly switches from behaving in an impossible to simulate way to behaving in a way with finite discrete state changes.

Even more eyebrow raising, if you erase the information about the interaction, it switches back to continuous as if memory optimized/garbage collected with orphaned references cleaned up (the quantum eraser variation of Young’s double slit experiment).

The latching on to the quantum experimental results and ditching the ‘why’ in favor of “shut up and calculate” has created an entire generation of physicists chasing the ghost of a unified theory of gravity while never really entertaining the idea that maybe the quantum experimental results are the side effects of emulating a continuous universe.

Your last link pissed me off enough I wrote an entire post on why that study is dog shit.

It sometimes pays off to review the methodology and supplementary materials in papers.

deleted by creator

Unless you are in the top 25%, in which case everyone else knows slightly less than you think they do.

“I really like hanging out with all my non-disabled friends.”

ChatGPT is correct that sometimes the inclusion of excess adjectives describing a majority population could have an implicit exclusionary interpretation that’s less than ideal.

Whereas when describing a minority group, calling out the less common features are not necessary going to have that same exclusionary tone.

“I always have a great time with my D&D playing friends.”

Clearly you need to figure out how to give antibiotic resistant bacteria cancer.

Well, there’s counterfactual examples of this, so it must not be true.

In pretty much every single relationship worldwide, one person can very easily determine if the recommendation from the other for where to eat or what to watch is correct or not.

And yet successfully figuring out where to eat or what to watch is nigh impossible.

Being a Jack of all trades may mean being a master of none, but it’s still better than to be a master of one.