Not leaking!

When you see this kind of text you’re just in a weird state and it’s going to look like GPT was talking to someone, but it’s really just babbling with no purpose

e: Get GPT-4 to complete an empty prompt, then ask it what it was responding to! I just tried with Simonw’s llm CLI like so:

  llm -m 4 ‘’
  # it outputs a weird response
  llm -c ‘What question was that in response to?’

In my case an explanation of euthanasia, and my supposed question was “What is euthanasia?”. I did it again and it said there was no original question, so there’s some randomnes.

It’s so interesting that people understand that it’s a machine mimicking speech, until it starts mimicking something they want to believe is real.

I had a similar situation where I was convinced that Code Interpreter had escaped its sandbox but SimonW was able to reel me in, discussed here https://mastodon.social/@Jackivers/110715505542072107

If you're using the chat interface then it's not really an empty prompt though, as there's a "system prompt".

https://chat.openai.com/share/5bb37148-6ff6-4e4b-9c2f-040295...

It sees a lot of separate conversations in its training. It seems much simpler to assume that it's incorrectly learned that it should shift the conversation into a new one occasionally to match this tendency of the training data rather than a bug actually leaking and blending user conversations together.

If you really think it leaked someone else’s prompts rather than hallucinating them, it just passed a sort of Turing test on your part.

I get the impression overall they have massively under invested in web infra, security, and privacy.

The AI is one thing, running a performant, secure, and private web platform is its own thing.

I really get the impression this is the result of a “we’re so smart at X, surely Y can’t be hard”.