Connect with us

Technology

Why multi-agent AI tackles complexities LLMs can’t

Published

on

Why multi-agent AI tackles complexities LLMs can't

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


The introduction of ChatGPT has brought large language models (LLMs) into widespread use across both tech and non-tech industries. This popularity is primarily due to two factors:

  1. LLMs as a knowledge storehouse: LLMs are trained on a vast amount of internet data and are updated at regular intervals (that is, GPT-3, GPT-3.5, GPT-4, GPT-4o, and others);
  1.  Emergent abilities: As LLMs grow, they display abilities not found in smaller models.

Does this mean we have already reached human-level intelligence, which we call artificial general intelligence (AGI)? Gartner defines AGI as a form of AI that possesses the ability to understand, learn and apply knowledge across a wide range of tasks and domains. The road to AGI is long, with one key hurdle being the auto-regressive nature of LLM training that predicts words based on past sequences. As one of the pioneers in AI research, Yann LeCun points out that LLMs can drift away from accurate responses due to their auto-regressive nature. Consequently, LLMs have several limitations:

  • Limited knowledge: While trained on vast data, LLMs lack up-to-date world knowledge.
  • Limited reasoning: LLMs have limited reasoning capability. As Subbarao Kambhampati points out LLMs are good knowledge retrievers but not good reasoners.
  • No Dynamicity: LLMs are static and unable to access real-time information.

To overcome LLM’s challenges, a more advanced approach is required. This is where agents become crucial.

Agents to the rescue

The concept of intelligent agent in AI has evolved over two decades, with implementations changing over time. Today, agents are discussed in the context of LLMs. Simply put, an agent is like a Swiss Army knife for LLM challenges: It can help us in reasoning, provide means to get up-to-date information from the Internet (solving dynamicity issues with LLM) and can achieve a task autonomously. With LLM as its backbone, an agent formally comprises tools, memory, reasoning (or planning) and action components.

Components of an agent (Image Credit: Lilian Weng)

Components of AI agents

  • Tools enable agents to access external information — whether from the internet, databases, or APIs — allowing them to gather necessary data.
  • Memory can be short or long-term. Agents use scratchpad memory to temporarily hold results from various sources, while chat history is an example of long-term memory.
  • The Reasoner allows agents to think methodically, breaking complex tasks into manageable subtasks for effective processing.
  • Actions: Agents perform actions based on their environment and reasoning, adapting and solving tasks iteratively through feedback. ReAct is one of the common methods for iteratively performing reasoning and action.

What are agents good at?

Agents excel at complex tasks, especially when in a role-playing mode, leveraging the enhanced performance of LLMs. For instance, when writing a blog, one agent may focus on research while another handles writing — each tackling a specific sub-goal. This multi-agent approach applies to numerous real-life problems.

Role-playing helps agents stay focused on specific tasks to achieve larger objectives, reducing hallucinations by clearly defining parts of a prompt — such as role, instruction and context. Since LLM performance depends on well-structured prompts, various frameworks formalize this process. One such framework, CrewAI, provides a structured approach to defining role-playing, as we’ll discuss next.

Advertisement

Multi agents vs single agent

Take the example of retrieval augmented generation (RAG) using a single agent. It’s an effective way to empower LLMs to handle domain-specific queries by leveraging information from indexed documents. However, single-agent RAG comes with its own limitations, such as retrieval performance or document ranking. Multi-agent RAG overcomes these limitations by employing specialized agents for document understanding, retrieval and ranking.

In a multi-agent scenario, agents collaborate in different ways, similar to distributed computing patterns: sequential, centralized, decentralized or shared message pools. Frameworks like CrewAI, Autogen, and langGraph+langChain enable complex problem-solving with multi-agent approaches. In this article, I have used CrewAI as the reference framework to explore autonomous workflow management.

Workflow management: A use case for multi-agent systems

Most industrial processes are about managing workflows, be it loan processing, marketing campaign management or even DevOps. Steps, either sequential or cyclic, are required to achieve a particular goal. In a traditional approach, each step (say, loan application verification) requires a human to perform the tedious and mundane task of manually processing each application and verifying them before moving to the next step.

Each step requires input from an expert in that area. In a multi-agent setup using CrewAI, each step is handled by a crew consisting of multiple agents. For instance, in loan application verification, one agent may verify the user’s identity through background checks on documents like a driving license, while another agent verifies the user’s financial details.

Advertisement

This raises the question: Can a single crew (with multiple agents in sequence or hierarchy) handle all loan processing steps? While possible, it complicates the crew, requiring extensive temporary memory and increasing the risk of goal deviation and hallucination. A more effective approach is to treat each loan processing step as a separate crew, viewing the entire workflow as a graph of crew nodes (using tools like langGraph) operating sequentially or cyclically.

Since LLMs are still in their early stages of intelligence, full workflow management cannot be entirely autonomous. Human-in-the-loop is needed at key stages for end-user verification. For instance, after the crew completes the loan application verification step, human oversight is necessary to validate the results. Over time, as confidence in AI grows, some steps may become fully autonomous. Currently, AI-based workflow management functions in an assistive role, streamlining tedious tasks and reducing overall processing time.

Production challenges

Bringing multi-agent solutions into production can present several challenges.

  • Scale: As the number of agents grows, collaboration and management become challenging. Various frameworks offer scalable solutions — for example, Llamaindex takes event-driven workflow to manage multi-agents at scale.
  • Latency: Agent performance often incurs latency as tasks are executed iteratively, requiring multiple LLM calls. Managed LLMs (like GPT-4o) are slow because of implicit guardrails and network delays. Self-hosted LLMs (with GPU control) come in handy in solving latency issues.
  • Performance and hallucination issues: Due to the probabilistic nature of LLM, agent performance can vary with each execution. Techniques like output templating (for instance, JSON format) and providing ample examples in prompts can help reduce response variability. The problem of hallucination can be further reduced by training agents.

Final thoughts

As Andrew Ng points out, agents are the future of AI and will continue to evolve alongside LLMs. Multi-agent systems will advance in processing multi-modal data (text, images, video, audio) and tackling increasingly complex tasks. While AGI and fully autonomous systems are still on the horizon, multi-agents will bridge the current gap between LLMs and AGI.

Abhishek Gupta is a principal data scientist at Talentica Software.

Advertisement

DataDecisionMakers

Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation.

If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers.

Advertisement

You might even consider contributing an article of your own!

Read More From DataDecisionMakers


Source link
Continue Reading
Advertisement
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Technology

Apple surveying its own employees about smart-glasses idea

Published

on

Apple surveying its own employees about smart-glasses idea

Apple is asking its employees how they feel about smart glasses as the tech giant considers entering the fledgling market, according to the latest report from the prominent Apple tipster Mark Gurman.

Citing people with knowledge of the matter, Gurman said on Monday that Apple has started gathering feedback from employees in focus groups organized by the company’s Product Systems Quality team, which is part of Apple’s hardware engineering division.

In a leaked email to select employees based at the company’s headquarters in California, the team wrote: “Testing and developing products that all can come to love is very important to what we do at Apple. This is why we are looking for participants to join us in an upcoming user study with current market smart glasses.”

Gurman said it’s not unusual for Apple to use internal focus groups when considering whether to enter a new market, as it can gain valuable feedback while keeping its plans secret, though as in this case, there’s always a chance that such activities may leak.

This is just the latest in a string of reports suggesting that Apple is moving closer toward the idea of building a pair of smart glasses designed to take on Meta’s Ray-Ban glasses, which have a built-in camera for taking photos and videos, and also let wearers make calls, listen to audio content, and ask an AI assistant questions. A recent update added the ability to send voice messages, and have the device read out incoming ones. At a special event in September, Meta CEO Mark Zuckerberg revealed that his team is developing a pair of smart specs with immersive augmented-reality capabilities for future release.

Advertisement

Considering how Apple is struggling to make a success of its larger and clunkier — and pricey at $3,499 — Vision Pro mixed-reality headset, the company is sure to exercise a great deal of caution when moving ahead with any plan for smart glasses, especially as the market for the device continues to be pretty niche.

And as Gurman said in his report, if the feedback from employees is positive enough to prompt Apple to pursue the project, its smart glasses are still likely to be “years away” from launch.






Source link

Continue Reading

Technology

Google updates Pixel Screenshots, tweaks Reminders and Collections

Published

on

Google updates Pixel Screenshots, tweaks Reminders and Collections

A new update has rolled out for the Pixel Screenshots app. The latest update essentially makes “All screenshots” redundant, tweaks carousels, and adds new shortcuts.

Google updates Pixel Screenshots app after the October Feature Drop

Last month, Google rolled out the October Feature Drop for Pixel smartphones. With the update, Google essentially unleashed its Gemini Generative Artificial Intelligence (Gen AI) within the Pixel Screenshots app. Devices from the Pixel 6, going all the way up to the latest Pixel 9 Pro, will be eligible for the October Feature Drop.

Pixel device users eligible for the update will be able to use the power of Gen AI with the app to look for content. Specifically, Google announced how users could append “in Screenshots” to any command to “help you find something you saved in Pixel Screenshots, from wherever you are on your phone.” Gemini would essentially scrub through the images and look for context and content users asked.

The Pixel Screenshots app is now getting another update. It is currently available on the Play Store. Pixel Screenshots v0.24.373.08 brings with it several UI tweaks and improvements.

Advertisement

Google dumps “All screenshots”, and improves the ability to quickly glance at images

The Pixel Screenshots app’s homepage has two carousels for Reminders and Collections. These segments then have a grid for all the captured screenshots. Previously, users had the ability to get a full-screen “All screenshots” view.

The latest update for the Pixel Screenshots app reportedly removes the dedicated All Screenshots page. Instead, Google has placed the grid density options directly on the main screen.

Moving forward, users have the option to switch between four or three squares. Additionally, there are two rectangular previews, which offer an easily glanceable look. Needless to say, this eliminates the step of tapping on the “All screenshots” view and then adjusting grid density.

Google also replaced the “Select screenshots” option in the Collections view with an FAB (Floating Action Button). The “Edit title” and “Delete” options are now in the overflow menu.

Advertisement

The update also introduces app shortcuts that let users quickly “Take a photo” and “Add an image” right from your home screen. Users need not even head over to the FAB for these actions.

Google has also truncated long transcriptions/descriptions. This means the “Add a note” field or “Add to collection” options are easily available without having to scroll to the end of the text.

Source link

Continue Reading

Technology

JBL early Black Friday deals include the Clip 5 Bluetooth speaker at an all-time-low price

Published

on

JBL early Black Friday deals include the Clip 5 Bluetooth speaker at an all-time-low price

Portable Bluetooth speakers are always a good option to pick up for a special someone (or yourself) when early Black Friday tech deals start to pop up. One model worth paying attention in the sales is the JBL Clip 5, which has dropped to an all-time-low price of $50. That’s 30 bucks off the regular price.

We often recommend JBL products in our buying guides, such as our ones on the best portable Bluetooth speakers and best travel gifts. They’re by and large durable devices that deliver ample volume for the beach or a cookout, for instance (but maybe don’t use it on public transit if you don’t want to risk the ire of your fellow passengers).

JBL

The JBL Clip 5 has dropped to an all-time-low price of $50. The small, portable clip-on Bluetooth speaker should come in handy for your next cookout or beach trip.

$50 at Amazon

As the name suggests, the JBL Clip 5 has a built-in carabiner, making it a cinch to attach to a backpack or belt loop, to name a couple of options. JBL said it redesigned the carabiner this time around to make the opening wider. As such, it’ll be easier to attach the speaker to a wider range of things.

Advertisement

Although the speaker measures just 1.75 inches, it’s said to deliver solid audio with punchy bass and a maximum output of seven watts. The JBL Clip 5 should be durable as well, since it has a IP67 waterproof and dustproof rating.

It’s claimed that the speaker will run for up to 12 hours on a single charge but there’s a Playtime Boost option that adds up to three extra hours of listening time. In addition, you can pair two Clip 5s for stereo audio or connect the speaker to other Auracast-enabled models from JBL to boost the sound.

Check out all of the latest Black Friday and Cyber Monday deals here.

Source link

Advertisement

Continue Reading

Technology

xAI woos developers with $25/month worth of API credits

Published

on

xAI woos developers with $25/month worth of API credits

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


We’ve known it for some time, but now it’s certain: the generative AI race is as much a contest for developers as it is for end-users.

Case-in-point: today, Elon Musk’s xAI, the spinoff startup of the social network X that uses its data to train new large language models (LLMs) such as the Grok family, today announced its application programming interface (API) is now open to the public and with it comes $25 free per month in API credits through the end of the year.

Given it’s already November, that’s just 2 months worth of free credits, or $50 total.

Advertisement

Musk previously announced the xAI API was open in beta three weeks ago to the date, but apparently uptake was not enough for his liking, hence the added incentive of free dev credits.

Is $25 per month with 2 months remaining really that much of a carrot?

It doesn’t sound like much coming from the world’s wealthiest man and multi-billionaire, and it’s not really on a per user basis nor in aggregate, but it may be enough to entice some developers to at least check out xAI’s tools and platform for building apps atop of the Grok models.

Specifically, xAI’s API is priced at $5 per million input tokens and $15 per million output, compared to $2.50/$10 for OpenAI’s GPT-4o model and at $3/$15 for Anthropic’s Claude 3.5 Sonnet model. Ultimately, that means xAI’s $25 credit won’t get the developer very far — only about two million tokens in and one million out per month. For reference, a million tokens is equivalent to 7-8 novels worth of words.

Advertisement

The context limit, or how many tokens can be inputted or outputted in one interaction through the API, is around 128,000, similar to OpenAI’s GPT-4o and below Anthropic’s 200,000 token window, and well below Google Gemini 1.5 Flash’s 1-million context window length.

Also, from my brief test of the xAPI, I was only able to access grok-beta and text only, no image generation capabilities such as those found on Grok 2 (powered by Black Forest Labs’ Flux.1 model).

New Grok models coming soon

According to xAI’s blog post, this is actually “a preview of a new Grok model that is currently in the final stages of development,” and a new Grok “vision model will be available next week.”

In addition, xAI notes that the grok-beta supports “function calling,” or the ability for the LLM to take commands from a user and access functions of other connected apps and services, even executing them on the user’s behalf (if the connected app allows such access).

Advertisement

Compatible with the competition

Furthermore, the xAI account on the social network X posted that the xAI API is “compatible with OpenAI & Anthropic SDKs,” or the software development kits of different web tools used by developers of those platforms, meaning it should be relatively easy to switch out those models for grok-beta or others on the xAI platform.

Musk’s xAI recently switched on its “Colossus” supercluster of 100,000 Nvidia H100 GPUs in Memphis, Tennessee, which is being used to train its new models — the largest or one of the largest in the world — so apparently that facility is already hard at work.

What do you think? Is it enough to get the developers out in the VentureBeat audience to try building atop xAI? Let me know: carl.franzen@venturebeat.com.


Source link
Continue Reading

Technology

Datadog challenger Dash0 aims to dash observability bill shock

Published

on

Dash0 Founders

The end of zero-interest rates has driven companies to look for savings wherever they can, but one area continues to be a major budget drain. Observability — collecting and understanding data and systems — typically remains an organization’s second-highest cloud expenditure, right after cloud provisioning itself. People have even gone so far as to talk of an observability cost crisis, underscored by anecdotes like Coinbase spending $65 million on its Datadog bill

And why is observability so pricey and important? Complex cloud architectures and microservices are here to stay, and with security issues and service outages all too common, ops teams need observability data to keep systems running.

Now a startup called Dash0 is launching to address the cost issue — if not by being cheaper, then by at least making buying and paying for their services easier.

Dash0 — pronounced “Dash-zero” — is a Datadog competitor whose pitch doesn’t revolve around drastically lowering observability costs. Founder Mirko Novakovic (left in the picture above) still expects companies to spend 10% to 20% of cloud costs on this budget item. But he and his team want to improve transparency, both in terms of pricing and of observability itself.

Advertisement

Dash0 says it can do this by way of how it’s built, by fully leveraging the open source observability framework OpenTelemetry (aka OTel), Novakovic told TechCrunch, which includes a feature called Semantic conventions that allows someone, “at any given time, [to] see exactly which service or which developer or which application creates how much cost on the observability side,” he said.

There are other companies, such as Signoz, that describe themselves as OTel-native, but Dash0’s positioning has resonated with investors. It raised a $9.5 million seed funding round led by Accel, with participation from Dig Ventures, the investment firm of MulesSoft founder Ross Mason.

Novakovic’s track record may have also helped. His previous company, Instana, also backed by Accel, was acquired by IBM at the end of 2020 for $500 million, a price that has never been publicly disclosed before now. Several other Instana alums are also now part of the Dash0 team.

If Dash0 is built on OTel, it’s also trying to improve it. The framework has actually been around since 2019, but “it is not that easy to use at the moment,” Novakovic said. “Vendors have to do a lot of work in making sure that it gets at least as easy as installing a Datadog agent. That’s where we are still lagging behind the proprietary folks.”

Advertisement

As a company, Dash0 hopes to unlock OTel’s benefits — vendor-agnostic standardized data — but with an intuitive UI, dashboards, and integrations with Slack, email and other tools. Its initial target customers are companies that have between 50 and 5,000 employees.

The company is now launching publicly, but it won’t heavily invest in sales and marketing until it is sure it has hit product-market fit. In the meantime, Novakovic said, its resources will go toward growing the tech and product side of its team, which now consists of 21 people, of whom 19 are engineers, all working remotely. 

Its next 10 hires will include a developer relations specialist who will also contribute to driving the adoption of OpenTelemetry as a solid alternative to proprietary options. On that front, the company intends to work with other OTel-related startups while making sure that “missing parts” like dashboards and query languages fall into place with projects like Perses and PromQL. “That’s a community effort together with the customers,” Novakovic said.

Source link

Advertisement

Continue Reading

Technology

What time do Black Friday deals start at Amazon, Walmart, and other retailers?

Published

on

What time do Black Friday deals start at Amazon, Walmart, and other retailers?

Black Friday officially kicks off on November 29th, but if you plan to make that the only day you buy gifts this holiday season, you’re missing out. Several prominent retailers, including Best Buy and Target, have already started rolling out their holiday discounts online and in-store. Both retailers are offering price protection through most of the holiday shopping season as well, so if you buy early and the same item drops to a lower price, you can retroactively get that better deal.

Whether you take advantage of the slew of early holiday deals or hold off until the day after Thanksgiving, it’s important to know how every major retailer is handling the shopping event this year. If you plan to spend some cash, here are the schedules and expectations for Amazon, Walmart, Target, and all the major retailers we know of.

Amazon has yet to announce a full Black Friday playbook, but it does have a landing page full of holiday deals, which the retailer may update and convert into its main hub for Black Friday — as well as Cyber Monday. There aren’t a ton of noteworthy early deals, at least in terms of tech, but Amazon is currently discounting a few pairs of Beats headphones. This includes the Beats Solo 4 and Beats Studio Pro, which are on sale for $99.99 ($100 off) and $169.99 ($180 off), respectively.

Just be aware that Amazon does not offer price matching, so price “protection” for many Amazon shoppers means buying the item at the lower price point and returning the original — which, unfortunately, is pretty wasteful.

Advertisement

Black Friday has already kicked off to some degree at Best Buy, with the retailer offering various specials via its “Holiday Savings” event, which runs through November 7th. That being said, the first wave of early “doorbuster” Black Friday deals will go live on Friday, November 8th, with new doorbusters to follow every Friday until November 20th. They’re open to everybody, though those who subscribe to My Best Buy Plus or My Best Buy Total will get access every Thursday before they become more widely available.

Best Buy’s main Black Friday event, which will feature returning doorbusters and new deals, is slated to begin on November 21st (yes, that’s a full eight days for a singular “Black Friday Sale”). The retailer’s Cyber Monday promo, meanwhile, will kick off on Sunday, December 1st, meaning Best Buy is running sales throughout the entire month of November.

Best Buy shoppers also get price protection, ensuring that if you purchase a qualifying item and it goes on to receive a bigger discount between November 1st and December 31st, you can request a refund for the difference through January 14th. As with some of the Black Friday promos, you’ll want to make sure you’re signed in with your My Best Buy account while browsing to ensure you’re getting the lowest price.

Apple Watches are popular items that usually receive deep discounts around Black Friday.
Photo by Amelia Holowaty Krales / The Verge

Advertisement

Like Best Buy, Target has launched a “Deal of the Day” holiday promo, which runs through December 24th for members of its free Target Circle program. Target will also host an Early Black Friday Sale, which will kick off on November 7th and last through November 9th.

Each Sunday in November, Target will drop new deals at brick-and-mortar stores and online, which you can preview a week beforehand using Target’s weekly circular. Target says it will be offering the best prices throughout the holiday season; however, the retailer also has a holiday price-match policy in place if it happens to offer a better deal on an item between now and December 24th. Just keep in mind that you may need to return to the store from which you purchased the product to receive a price match or call Target’s guest services for online purchases.

Walmart plans to start rolling out its Black Friday deals in multiple waves. The first begins online at 12PM ET on Monday, November 11th, for paying Walmart Plus subscribers (or 5PM ET for everybody else). An in-store sale will follow on November 15th at 6AM local time.

The second wave begins online at 12PM ET on Monday, November 25th, for Walmart Plus members (or 5PM ET for everybody else). An in-store sale will follow on Black Friday proper (November 29th) at 6AM local time.

Advertisement

Walmart will then hold its Cyber Monday sale online for Walmart Plus members beginning at 5PM ET on December 1st before opening it to non-members at 8PM ET.

If you want to get early access to any of the aforementioned waves, Walmart is offering 50 percent off an annual Walmart Plus membership through December 2nd. The current promo brings the cost of its premium service down to $49 a year. Along with early access to all of Walmart’s holiday discounts, Plus members also get free delivery on millions of products.

There is no word yet on Black Friday details from GameStop. We expect to hear more soon regarding store hours on Black Friday and possibly Thanksgiving, as well as some specifics regarding what kind of gaming deals it plans to offer throughout the holiday season.

Newegg has already started dropping its Black Friday deals. Like other retailers, it also offers price protection, so you can choose to buy early without having to worry about potential discounts down the line. Newegg is noting price-protected items with a colorful badge near the product’s name through November 20th, and if said product drops in price on or before November 30th, Newegg will automatically refund the difference to the original payment method.

Advertisement

Costco recently announced two deals events ahead of Black Friday, including a “Holiday Savings” event that runs through December 2nd online and in-store. There’s also an online “November Savings” promo, which runs through November 30th. Costco will continue to drop new deals throughout November, which you can preview here. There aren’t a ton of notable tech highlights (yet), but Costco will be discounting the noise-canceling Bose QuietComfort Ultra Earbuds to $229.99 ($70 off) starting on November 18th. Costco will also throw in a $50 gift card from one of several retailers, including Apple and Google.

Source link

Continue Reading

Trending

Copyright © 2024 WordupNews.com