Evaluating the Energy-Efficiency of the Code Generated by LLMs
arxiv.org
As the quality of code generated by Large Language Models (LLMs) improves, their adoption in the software industry for automated code generation continues to grow. Researchers primarily focus on enhancing the functional correctness of the generated code while commonly overlooking its energy efficiency and environmental impact. This paper investigates the energy efficiency of the code generated by 20 popular LLMs for 878 programming problems of varying difficulty levels and diverse algorithmic categories selected from the LeetCode platform by comparing them against canonical human-written solutions. Although LLMs can produce functionally correct results in most cases, our findings show that the performance and energy efficiency of LLM-produced solutions are often far below those of human-written solutions. Among the studied LLMs, DeepSeek-v3 and GPT-4o generate the most energy-efficient code, whereas Grok-2 and Gemini-1.5-Pro are among the least energy-efficient models. On average, human-generated canonical solutions are approximately 1.17 times more energy efficient than DeepSeek-v3, 1.21 times more energy efficient than GPT-4o, and over 2 times more energy efficient than Grok-2 and Gemini-1.5-Pro. For specific algorithmic groups such as dynamic programming, backtracking, and bit manipulation, LLM-generated code can consume up to 450 times more energy than human-generated canonical solutions.

AI bros won’t hype this up for the news for sure, but 480x energy doesn’t sound optimistic enough for replacement.

  • herseycokguzelolacak@lemmy.mlEnglish
    1·
    9 hours ago

    LLMs are great at automating tasks where we know the solution. And there are a lot of workflows that fall in this category. They are horrible at solving new problems, but that is not where the opportunity for LLMs is anyway.

  • kadup@lemmy.worldEnglish
    1·
    18 days ago

    LLMs don’t have reasoning nor internal logic. If you take a look at the “thinking” feature AIs like Gemini introduced, this becomes even more obvious. In order to have the most basic type of analysis possible, it must hallucinate an entire context window to force the language model to reach a specific conclusion.

    There’s zero world in which LLMs replace humans. They might, temporarily, be convincing enough to trick a few CEOs… But that period of time won’t last long.

    Now, a human being assisted by AI on Microsoft Word or their Python IDE, sure.

  • hendrik@palaver.p3x.deEnglish
    1·
    18 days ago

    I’m not sure about the significance of this preprint. Writing energy-efficient sorting algorithms and lab course example code is a very specific problem. It doesn’t say a lot about AI in general. Also: Did they forget to tell the AI it’s supposed to write energy-efficient code? I didn’t read the entire paper. But the prompt example doesn’t look like it’s in there.