• 1 Post
  • 14 Comments
Joined 4 months ago
cake
Cake day: February 24th, 2025

help-circle






  • I think tool calling is just a form of structured output, where the LLM outputs something like json that describes the function to call and the arguments, then you parse that and run the function with the arguments, then feed the output back to the LLM in a new message, if you want. IDK the specific details, I’m guessing there are some special tokens some LLMs produce for tool calling, and I’m also guessing there is “controlled generation” (masking the logits/tokens, and only choosing to generate the tokens that would be valid). Ollama apparently doesn’t support the special tool-calling tokens or output structure that some models use.

    This is how it looks when using OpenAI compatible APIs: https://platform.openai.com/docs/guides/function-calling?api-mode=responses.

    I’ve never heard of TabbyAPI (I’m new to using local models, in general). I’m also not sure what TabbyAPI brings over Ollama, llama.cpp, or vLLM. So would be curious as well.

    Edit: For my little toy project I’m working on, I’m switching to not using tool-calling at all, and building a LangGraph and using structured output, which should be more reliable with my use-case. I.e. just always call the tool manually, feed the output back to the LLM to have the LLM evaluate if the output was correct, retry calling the tool with different arguments if not, and just fail after 5 calls.

    I like watching Yannic Kilcher, to keep up with some of the newer developments, and read various papers. I do have a background in ML (mostly the more traditional, non-generative, supervised learning and reinforcement learning) though.





  • Yeah, that’s how they sell it. Problem is, studies have shown the fraud in these programs isn’t much of an expense, in the broader context. And, Trump has granted clemency to Lawrence Duran, who stole $205M from Medicare, which tells you they don’t really care about that. These programs are actually pretty efficient, and spending funds to investigate small-time fraud would often cost more than just letting it happen. It’s not like tons of people wish to be on our shitty social programs that don’t even supply enough help for the people that absolutely need it.





  • I know a couple life-long Republicans I sometimes briefly talk about politics with (one family, one acquaintance). Neither of them like Trump, but like the idea around Project 2025. One is an evangelical Christian, the other is a Catholic.

    The Catholic strongly believes government should be run like a business, and the president should be like a CEO, so he should be able to fire everyone and replace them, if needed, with workers that will execute his plans. He’s also an anti-abortion, and tough-on-crime/immigration type. However, he strongly disapproves of Trump seemingly being pro-Russian now, Trump and his cabinet’s personal lives (he’s always strangely fixated on people’s personal lives, in a moral sense, for some reason), the take-over of the FBI and CIA, and the tariffs hurting his stock portfolio.

    The evangelical Christian just doesn’t like Trump as a person, and doesn’t like Russia. He’s a just-world-hypothesis, small government, women are subservient, pro-business type; but also low/lower-middle-class, and has needed, and will need the social services he opposes. I guess his opinions are pretty similar to the Catholic’s, just a little more extreme on the social side, and supports policies that have always hurt him. I mean, Republican policies hurt the (fairly wealthy) Catholic too, but at least they get to say their taxes are lower and there’s less red-tape.