Llama 2 avoids errors by staying quiet, GPT-4 gives long, if useless, samples
Discover how Llama 2 outperforms GPT-4 in generating reliable code, revealing crucial insights on the effectiveness of large language models.
The article discusses a study conducted by computer scientists at the University of California San Diego on the reliability and robustness of large language models (LLMs) in generating code. The researchers evaluated four different code-capable LLMs using an API checker called RobustAPI. They gathered 1,208 coding questions from StackOverflow involving 24 common Java APIs and tested the LLMs with three different types of questions. The results showed that the LLMs had high rates of API misuse, with GPT-3.5 and GPT-4 from OpenAI exhibiting the highest failure rates. However, Meta’s Llama 2 performed exceptionally well, with a failure rate of less than one percent. The study highlights the importance of assessing code reliability and the need for improvement in large language models’ ability to generate clean code.
Nobody takes you aside anymore
Print taught a generation when to stop. What we lose when the machines absorb the constraints that used to form us.
Your AI agents need a water cooler
Coordination is a property of the room, not the org chart. What that means when your coworkers are agents.
On the death of the author and the birth of the detector
Why worrying about AI authorship is lazier, and more prejudiced, than it looks.
The work of being available now
A book on AI, judgment, and staying human at work.
The practice of work in progress
Practical essays on how work actually gets done.
Memory is (almost) solved. time is next.
AI can't tell if a memory is two minutes or two weeks old. The fix isn't making models feel time — it's cache invalidation: an as-of stamp on every fact, a clock in the context, and a freshness window for anything volatile.
It’s going to take a century for artifical intelligence to be able to perform most human jobs. But there are going to be some key developments during the next decade.
Explore how AI will transform jobs in the next decade, from enhancing security to automating coding, reshaping the future of work.
Many businesses are not yet prepared to fully reap the benefits of AI.
Unlock AI's true potential for your business by integrating it into your strategy, boosting productivity, and enhancing customer experiences.
Rose-tinted predictions for artificial intelligence’s grand achievements will be swept aside by underwhelming performance and dangerous results.
Explore the reality of generative AI in 2024 as hype fades, revealing limitations, job displacement, and the need for regulation.