Author: Web Dev

Large language models (LLMs) face a hurdle in handling long contexts due to their constrained window length. Although the context window length can be extended through fine-tuning, this incurs significant training and inference time costs, adversely affecting the LLM’s core capabilities. Current LLMs, such as Llama-1 and Llama-2, have fixed context lengths, hindering real-world applications….

LLMs are trained on vast amounts of web data, which can lead to unintentional memorization and reproduction of sensitive or private information. This raises significant legal and ethical concerns, especially regarding violating individual privacy by disclosing personal details. To address these concerns, the concept of unlearning has emerged. This approach involves modifying models after training…

LLMs have been at the forefront of recent technological advances, demonstrating remarkable capabilities in various domains. However, enhancing these models’ reflective thinking and self-correction abilities is a significant challenge in AI development. Earlier methods, relying heavily on external feedback, often fail to enable LLMs to self-correct effectively. The Zhejiang University and OPPO Research Institute research…

Drug discovery is an essential process with applications across various scientific domains. However, Drug discovery is a very complex and time-consuming process. The traditional drug discovery approaches require extensive collaboration among teams spanning many years. Also, it involved scientists from various scientific fields working together to identify new drugs that can help the medical domain….

In sequence processing, one of the biggest challenges lies in optimizing attention mechanisms for computational efficiency. Linear attention has proven to be an efficient attention mechanism with its ability to process tokens in linear computational complexities. It has recently emerged as a promising alternative to conventional softmax attention. This theoretical advantage allows it to handle…

There’s a burgeoning interest in technologies that can transform textual descriptions into videos. This area, blending creativity with cutting-edge tech, is not just about generating static images from text but about animating these images to create coherent, lifelike videos. The quest for producing high-fidelity, aesthetically pleasing videos that accurately reflect the described scenarios presents a…

Understanding why things happen, known as causal inference, is a key part of human intelligence. There are two main ways we gain this ability: one is through what we’ve learned from experience, like knowing that touching a hot stove causes burns based on common sense; the other is through pure causal reasoning, where we formally…

The rapid advancements in the field of Artificial Intelligence (AI) have led to the introduction of Large Language Models (LLMs). These highly capable models can generate human-like text and can perform tasks including question answering, text summarization, language translation, and code completion.  AI systems, particularly LLMs, can behave dishonestly strategically, much like how people can…

Mira Murati as CTO, Greg Brockman returns as President. Read messages from CEO Sam Altman and board chair Bret Taylor.

Axel Springer is the first publishing house globally to partner with us on a deeper integration of journalism in AI technologies.