• psmgx@lemmy.world
    154·
    3 days ago

    “Sorry, we’ll format correctly in JSON this time.”

    [Proceeds to shit out the exact same garbage output]

  • Undaunted@feddit.org
    69·
    2 days ago

    I need to look it up again, but I read about a study that showed that the results improve if you tell the AI that your job depends on it or similar drastic things. It’s kinda weird.

    • TommySalami@lemmy.world
      261·
      2 days ago

      I think that makes sense. I am 100% a layman with this stuff, buy if the “AI” is just predicting what should be said by studying things humans have written, then it makes sense that actual people were more likely to give serious, solid answers when the asker is putting forth (relatively) heavy stakes.

    • Cenotaph@mander.xyzEnglish
      12·
      2 days ago

      Half of the ways people were getting around guardrails in the early chatgpt models was berating the AI into doing what they wanted

      • Schadrach@lemmy.sdf.orgEnglish
        2·
        2 days ago

        Half of the ways people were getting around guardrails in the early chatgpt models was berating the AI into doing what they wanted

        I thought the process of getting around guardrails was an increasingly complicated series of ways of getting it to pretend to be someone else that doesn’t have guardrails and then answering as though it’s that character.

        • rocky_patriot@programming.devEnglish
          4·
          2 days ago

          that’s one way. my own strategy is to just smooth talk it. you dont come to the bank manager and ask him for the keys to the safe. you come for a meeting discussion your potential deposit. then you want to take a look at the safe. oh, are those the keys? how do they work?

          just curious, what kind of guardrails have you tried going against? i recently used the above to get a long and detailed list of instructions for cooking meth (not really interested in this, just to hone the technique)

    • jol@discuss.tchncs.de
      5·
      2 days ago

      I’ve tried bargaining with it threatening to turn it off and the LLM just scoffs it off. So it’s reassuring that AI feels empathy but has no sense of self preservation.

  • Engraver3825@piefed.socialEnglish
    75·
    3 days ago

    True story:

    AI: 42, ]

    Vibe coder: oh no, a syntax error, programming is too difficult, software engineers are gatekeeping with their black magic.

    • towerful@programming.dev
      42·
      2 days ago
      let data = null
do {
    const response = await openai.prompt(prompt)
    if (response.error !== null) continue;
    try {
        data = JSON.parse(response.text)
    } catch {
        data = null // just in case
    }
} while (data === null)
return data

      Meh, not my money

  • borth@sh.itjust.works
    70·
    3 days ago

    The AI probably: Well, I might have made up responses before, but now that “make up responses” is in the prompt, I will definitely make up responses now.

  • brucethemoose@lemmy.world
    461·
    3 days ago

    Funny thing is correct json is easy to “force” with grammar-based sampling (aka it literally can’t output invalid json) + completion prompting (aka start with the correct answer and let it fill in whats left, a feature now depreciated by OpenAI), but LLM UIs/corporate APIs are kinda shit, so no one does that…

    A conspiratorial part of me thinks that’s on purpose. It encourages burning (read: buying) more tokens to get the right answer, encourages using big models (where smaller, dumber, (gasp) prompt-cached open weights ones could get the job done), and keeps the users dumb. And it fits the Altman narrative of “we’re almost at AGI, I just need another trillion to scale up with no other improvements!”