Uncategorized

Graph Transformers need help with scalability in graph sequence modeling due to high computational costs, and existing attention sparsification methods fail to adequately address data-dependent contexts. State space models (SSMs) like Mamba are effective and efficient in modeling long-range dependencies in sequential data, but adapting them to non-sequential graph data is challenging. Many sequence models…

Well-known Large Language Models (LLMs) like ChatGPT and Llama have recently advanced and shown incredible performance in a number of Artificial Intelligence (AI) applications. Though these models have demonstrated capabilities in tasks like content generation, question answering, text summarization, etc, there are concerns regarding possible abuse, such as disseminating false information and assistance for illegal…

The emergence of Large Vision-Language Models (LVLMs) characterizes the intersection of visual perception and language processing. These models, which interpret visual data and generate corresponding textual descriptions, represent a significant leap towards enabling machines to see and describe the world around us with nuanced understanding akin to human perception. A notable challenge that impedes their…

There’s been a significant shift towards creating powerful and pragmatically deployable models in varied contexts. This narrative centers on the intricate balance between developing expansive language models imbued with the capacity for deep understanding and generation of human language and the practical considerations of deploying these models efficiently, especially in environments constrained by computational resources….

Pretrained on trillion-token corpora, large neural language models (LLMs) have achieved remarkable performance strides (Touvron et al., 2023a; Geng & Liu, 2023). However, the scalability benefits of such data for traditional n-gram language models (LMs) still need to be explored. This paper from the University of Washington and Allen Institute for Artificial Intelligence delves into…

Delving into the intricacies of artificial intelligence, particularly within the dynamic reasoning domain, uncovers the pivotal role of Large Language Models (LLMs) in navigating environments that are not just complex but ever-changing. While effective in predictable settings, traditional static reasoning models falter when faced with the unpredictability inherent in real-world scenarios such as market fluctuations…

As enterprises look to deploy LLMs in more complex production use cases beyond simple knowledge assistants, there is a growing recognition of three interconnected needs:   Agents – complex workflows involve multiple steps and require the orchestration of multiple LLM calls; Function Calls – models need to be able to generate structured output that can be…

Nomic AI released an embedding model with a multi-stage training pipeline, Nomic Embed, an open-source, auditable, and high-performing text embedding model. It also has an extended context length supporting tasks such as retrieval-augmented-generation (RAG) and semantic search. The existing popular models, including OpenAI’s text-embedding-ada-002, lack openness and auditability. The model addresses the challenge of developing…

Despite the utility of large language models (LLMs) across various tasks and scenarios, researchers need help to evaluate LLMs properly in different situations. They use LLMs to check their responses, but a solution must be found. This method is limited because there aren’t enough benchmarks, and it often requires a lot of human input. They…

Artificial intelligence has seen remarkable advancements with the development of large language models (LLMs). Thanks to techniques like reinforcement learning from human feedback (RLHF), they have significantly improved performing various tasks. However, the challenge lies in synthesizing novel content solely based on human feedback. One of the core challenges in advancing LLMs is optimizing their…