Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
Retrieval-augmented generation (RAG) has become a popular method for grounding large language models (LLMs) in external knowledge. RAG systems typically use an embedding model to encode documents in a knowledge corpus and select those that are most relevant to the user’s query.
However, standard retrieval methods often fail to account for context-specific details that can make a big difference in application-specific datasets. In a new paper, researchers at Cornell University introduce “contextual document embeddings,” a technique that improves the performance of embedding models by making them aware of the context in which documents are retrieved.
The limitations of bi-encoders
The most common approach for document retrieval in RAG is to use “bi-encoders,” where an embedding model creates a fixed representation of each document and stores it in a vector database. During inference, the embedding of the query is calculated and compared to the stored embeddings to find the most relevant documents.
Advertisement
Bi-encoders have become a popular choice for document retrieval in RAG systems due to their efficiency and scalability. However, bi-encoders often struggle with nuanced, application-specific datasets because they are trained on generic data. In fact, when it comes to specialized knowledge corpora, they can fall short of classic statistical methods such as BM25 in certain tasks.
“Our project started with the study of BM25, an old-school algorithm for text retrieval,” John (Jack) Morris, a doctoral student at Cornell Tech and co-author of the paper, told VentureBeat. “We performed a little analysis and saw that the more out-of-domain the dataset is, the more BM25 outperforms neural networks.”
BM25 achieves its flexibility by calculating the weight of each word in the context of the corpus it is indexing. For example, if a word appears in many documents in the knowledge corpus, its weight will be reduced, even if it is an important keyword in other contexts. This allows BM25 to adapt to the specific characteristics of different datasets.
“Traditional neural network-based dense retrieval models can’t do this because they just set weights once, based on the training data,” Morris said. “We tried to design an approach that could fix this.”
Advertisement
Contextual document embeddings
Contextual document embeddings Credit: arXiv
The Cornell researchers propose two complementary methods to improve the performance of bi-encoders by adding the notion of context to document embeddings.
“If you think about retrieval as a ‘competition’ between documents to see which is most relevant to a given search query, we use ‘context’ to inform the encoder about the other documents that will be in the competition,” Morris said.
The first method modifies the training process of the embedding model. The researchers use a technique that groups similar documents before training the embedding model. They then use contrastive learning to train the encoder on distinguishing documents within each cluster.
Contrastive learning is an unsupervised technique where the model is trained to tell the difference between positive and negative examples. By being forced to distinguish between similar documents, the model becomes more sensitive to subtle differences that are important in specific contexts.
The second method modifies the architecture of the bi-encoder. The researchers augment the encoder with a mechanism that gives it access to the corpus during the embedding process. This allows the encoder to take into account the context of the document when generating its embedding.
Advertisement
The augmented architecture works in two stages. First, it calculates a shared embedding for the cluster to which the document belongs. Then, it combines this shared embedding with the document’s unique features to create a contextualized embedding.
This approach enables the model to capture both the general context of the document’s cluster and the specific details that make it unique. The output is still an embedding of the same size as a regular bi-encoder, so it does not require any changes to the retrieval process.
The impact of contextual document embeddings
The researchers evaluated their method on various benchmarks and found that it consistently outperformed standard bi-encoders of similar sizes, especially in out-of-domain settings where the training and test datasets are significantly different.
“Our model should be useful for any domain that’s materially different from the training data, and can be thought of as a cheap replacement for finetuning domain-specific embedding models,” Morris said.
Advertisement
The contextual embeddings can be used to improve the performance of RAG systems in different domains. For example, if all of your documents share a structure or context, a normal embedding model would waste space in its embeddings by storing this redundant structure or information.
“Contextual embeddings, on the other hand, can see from the surrounding context that this shared information isn’t useful, and throw it away before deciding exactly what to store in the embedding,” Morris said.
The researchers have released a small version of their contextual document embedding model (cde-small-v1). It can be used as a drop-in replacement for popular open-source tools such as HuggingFace and SentenceTransformers to create custom embeddings for different applications.
Morris says that contextual embeddings are not limited to text-based models can be extended to other modalities, such as text-to-image architectures. There is also room to improve them with more advanced clustering algorithms and evaluate the effectiveness of the technique at larger scales.
Advertisement
VB Daily
Stay in the know! Get the latest news in your inbox daily
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
Mobile growth acceleration platform Liftoff announced today it’s launching a new machine learning platform called Cortex. This next-gen platform uses tailored neural network models to improve mobile ad campaigns. Its enhanced computing power boosts pattern recognition and processing data. It can be used to provide a higher return on investment for advertising by identifying the best channels and audiences for their campaigns.
According to Liftoff, Cortex has already shown some improvements to ad campaigns in the short amount of time it’s been available. Ad campaigns have seen a 23% decrease in CPIs (cost per install), a 21% decrease in CPAs (cost per acquisition) and a 16% increase in ROAS (return on ad spend). Cortex can respond quickly to shifting market conditions and find patterns in large and varied data sets.
Jeremy Bondy, Liftoff CEO, said in a statement, “Cortex marks a significant leap in mobile advertising technology. At Liftoff, success is defined by the measurable business outcomes we deliver to our partners… Our new deep learning models enable us to harness more data from our proprietary platform, optimizing campaigns to deliver superior results. These models also train and iterate faster, giving us the agility to respond swiftly in an ever-evolving market. We believe this innovation opens up substantial new opportunities for growth across the entire Liftoff platform.”
Advertisement
Game developer Playlinks is one client that has already seen the benefits of Cortex. Marketing manager Seokyung Lee said in a statement, “With its strong ML logic, Cortex, Liftoff has shown strong ROAS results compared to other networks, and the proportion of new users is almost 80% high on iOS. Its support for optimization and creativity has been consistently impactful on campaign results as well.”
VB Daily
Stay in the know! Get the latest news in your inbox daily
As we approach the final stretch before TechCrunch’s biggest event of the year — one of the most highly anticipated tech conferences for startups and VCs alike — let’s take a look at the top five reasons you won’t want to miss Disrupt 2024, set to take place at Moscone West in San Francisco from October 28-30.
1. Networking galore
There’s no better place to elevate your networking game than at one of the world’s largest tech events, right in the heart of Silicon Valley. With countless opportunities to meet the right people, you’ll be in a prime position to advance your professional goals. Join 10,000 tech, startup, and VC leaders and connect in meaningful ways.
Braindate networking: Spark engaging conversations on the Braindate app by sharing your discussion topics and exploring the ideas of others. Set up in-person 1:1 or small-group meetings with Disrupt attendees for collaborative brainstorming and problem-solving with peers who share your interests.
Expo Hall: Explore the dynamic Expo Hall to uncover the latest tech breakthroughs and connect with industry leaders or VCs on the lookout for their next major investment.
Advertisement
Receptions: Start your mornings with two exclusive, first-come, first-served breakfast receptions that bring niche communities together. Meet inspiring women in tech at the Women of Disrupt Breakfast or, if you hold a Founder or Investor Pass, connect with other founders and investors at the Founder & Investor Breakfast.
Side Events: Extend your Disrupt experience with a range of company-hosted events happening before and after the main event. From meetups and workshops to happy hours and comedy shows, these gatherings offer more chances to connect with startup and VC leaders. See the full list of Side Events here.
2. Invaluable insights
Immerse yourself in top-tier industry insights across six dedicated stages: AI, Builders, Disrupt, Fintech, SaaS, and Space. Join in for in-depth discussions with industry heavyweights and gain knowledge you won’t find elsewhere. Check out the full Disrupt 2024 agenda here.
AI Stage presented by Google Cloud
Builders Stage
Disrupt Stage
Fintech Stage
SaaS Stage
Space Stage presented by Aerospace
3. Startup Battlefield 200
Make sure to attend one of Disrupt’s signature events: Startup Battlefield 200. Experience the excitement as the leading pre-Series A startups pitch their cutting-edge ideas to a panel of top VC judges. The champion will take home a $100,000 equity-free prize and the sought-after Disrupt Cup.
Advertisement
Featuring leading VC experts, our panel of judges will provide essential feedback as they evaluate each startup’s potential for success based on their criteria.
Our judges will conduct an in-depth Q&A to analyze each startup, highlighting the essential factors that contribute to their viability. Take advantage of this opportunity to gain valuable insights from their expert analysis at Disrupt 2024.
4. Hands-on sessions
Take part in a 30-minute Roundtable with an industry expert for a chance to engage in collaborative conversations in a small-group setting. You can also attend a 50-minute Breakout Session, where leading experts will answer pressing questions about modern entrepreneurship.
5. Scaling startups
We’re thrilled to introduce the ScaleUp Startup Exhibitor Program at this year’s Disrupt. This program will feature a variety of startups across multiple industries showcasing their cutting-edge innovations in the Expo Hall. Experience the future of technology firsthand. If you’re an investor looking to fund the next big idea or an individual interested in contributing to innovation, there’s something here for you.
Don’t miss Disrupt 2024
No matter where you are in your startup journey or professional career, the insights from industry leaders, the meaningful connections made through extensive networking, and the inspiration drawn from engaging discussions are invaluable. Don’t miss the chance to experience all of this at Disrupt 2024, happening October 28-30 at Moscone West in San Francisco.
Amazon isn’t known for offering the best deals for laptops, but its ongoing Prime Big Deal Days sales event has a few solid discounts — and there are a bunch more if you factor in Best Buy’s “48-hour Flash Sale” counter-programming. If you’re in the market for a new MacBook, Windows productivity machine (including new Copilot Plus PCs), or even a gaming laptop, we’ve got you covered with a handful of worthwhile options.
While a specialized electronics retailer like Best Buy may be better known for everyday laptop discounts, Amazon has its fair share. This is especially the case if you’re shopping for MacBooks, some of which Amazon currently has for their lowest prices to date. Granted, you’re not going to get the breadth of options for RAM and storage sizes like you do directly from Apple, but it’s not uncommon to get as much as $400 off some base- and mid-spec configurations (on the pricier MacBook Pros, at least).
Here are the best laptop deals we scrounged for Amazon’s Prime Big Deal Days and some neighboring sales.
The best Prime Day deals on Windows laptops
Advertisement
The 2024 Asus Zenbook Duo laptop is on sale for $1,201.74 (around $298 off) at Amazon. Spec-wise, it’s got an Intel Core Ultra 7 processor, 16GB of RAM, and 1TB SSD. However, its most unique qualities are by far its dual 14-inch OLED touchscreens (with included stylus), one of which is revealed by removing its wireless keyboard deck to allow a multi-screen desktop-like setup you can bring anywhere.
It may look a bit bonkers, but the Zenbook Duo allows for 19.8 inches of screen real estate in a package you can throw in your everyday bag and bring with you to the coffee shop. Talk about a power move. Read our review.
Asus’ dual-screen laptop has a pair of 14-inch OLED touchscreens, each with 2880 x 1800 resolution. The removable keyboard can cover one of the screens for conventional single-monitor laptop use or be used wirelessly with the laptop propped up in dual-screen mode via its kickstand.
The Samsung Galaxy Book4 Edge is on sale in both its 14-inch configuration for as low as $799.99 (a massive $550 off) at Best Buy and in its 16-inch configuration for $1,249.99 ($200 off) at Best Buy. Either model gets you an Arm-based Snapdragon X Elite processor, 16GB of RAM, and a 512GB SSD. It doesn’t have the outright best performance of the recent crop of Windows Copilot Plus PCs running Arm chips, but it’s definitely one of the prettiest to look at with its thin-and-light frame and excellent OLED screen. Plus, its power-sipping processor helps it last all day like a modern MacBook. Read our review.
Samsung’s first entry in the Copilot Plus PC world, running Windows on Arm with a Snapdragon X Elite processor. The Galaxy Book4 Edge comes in two sizes, the 14-inch and 16-inch, both of which feature vivid OLED touchscreen displays with 120Hz refresh rate and all-day battery life. Read our review.
The latest Microsoft Surface Pro with a Snapdragon Elite X processor, 16GB of RAM, and 512GB SSD in sapphire blue is $1,199.99 ($300 off) at Best Buy. The 2-in-1 offers great performance as long as you don’t need certain apps that are lacking full support for Windows on Arm. Its detachable keyboard now has Bluetooth so you can use it separately from the tablet, and the 13-inch, 2880 x 1920 OLED you get in this configuration looks great and sports a speedy 120Hz refresh. It’s a great all-around package if you’re seeking a 2-in-1 with great battery life. Read our review.
Advertisement
The new Surface Pro maintains its winning form factor, which sets it apart from traditional laptops. It now houses the AI-ready Qualcomm Snapdragon X Elite chipset, however, and the detachable keyboard has been improved with bolder, brighter keys and a Copilot button.
The 2024 Microsoft Surface Laptop is on sale for $1,399.99 ($200 off) at Amazon and Best Buy. Much like the Surface Pro, Microsoft’s latest Surface Laptop is sporting a Snapdragon X Elite Arm processor that offers excellent battery life and standby time. It’s got a 13.8-inch touchscreen and comes with 16GB of RAM and a 1TB SSD in this discounted configuration. It’s a great performer if you want a portable Windows laptop for everyday use that can last all day, though be sure to check if any creative apps you plan to use have Arm support or work fine in emulation. Read our review.
The 13.8-inch Surface Laptop is the most affordable of Microsoft’s 2024 Copilot Plus models. The MacBook rival uses Qualcomm’s Snapdragon X chipsets — which are said to offer substantial performance and battery life improvements — and its keyboard is the first with a dedicated Copilot key.
We haven’t tested the 2024 Asus TUF A16, but at $699.99 ($400 off) at Best Buy, it seems like a solid value for some midrange gaming needs. The TUF has a speedy 165Hz display with 1200p resolution that, while not the highest resolution for a 16-inch panel, shouldn’t be as demanding as running a QHD or 4K display. Those graphics are powered by an all-AMD setup with a Ryzen 7 chip and Radeon RX7700S GPU. It’s the same video card we tested in the Framework Laptop 16, which offered some fair benchmarking scores in various games. (That modular laptop’s issues were owed more to glitches and cooling issues than poor GPU performance.)
A 16-inch gaming laptop sporting a 1920 x 1200 display with 165Hz refresh, AMD Ryzen 7 7735HS processor, Radeon RX7700S GPU, 16GB of RAM, and 512GB SSD.
The best Prime Day deals on Apple MacBooks
The 14-inch M3 MacBook Pro with 8GB of RAM, and 512GB SSD is on sale for $1,299 ($300 off) at both Best Buy and Amazon. Or, you can step up to more RAM and storage with 16GB / 1TB for $1,699 ($300 off) at Amazon. That may be a steeper cost, but the extra RAM is especially worth it in the long haul.
Advertisement
This model of MacBook may be a bit of an awkward middle child between the MacBook Air and the beefier 14- and 16-inch MacBook Pros with Pro / Max chips, but it’s still a nice laptop for light creative workflows and all-day battery life (and then some). Compared to a MacBook Air, it has more ports, including an HDMI-out and SD card slot, but it only has two USB-C / Thunderbolt 3 ports compared to the three on the more “pro” MacBook Pros.
Apple’s new entry-level model for the MacBook Pro line is now a 14-inch laptop powered by the new base M3 processor. It uses a similar design to the pricier 14-inch MacBook Pro with Apple’s Pro- / Max-series chips but is offered at a lower price with the similar ports and less RAM.
Speaking of MacBook Airs, the 13- and 15-inch M3 models are currently available for their best prices to date. You can get the 13-inch M3 MacBook Air for $849 ($250 off) at Amazon or the larger 15-inch model for $1,044 ($255 off) at Amazon or Best Buy. Both of these cheap-as-can-be configurations come with 8GB of RAM and 256GB of storage.
The M3 generation of Airs are mostly spec-bumped models, though they do also have Wi-Fi 6E support and the ability to output video to two monitors when their lids are closed. Both are great for everyday work / home use and have great battery life that mostly ensures you don’t have to even worry about charging until the end of the day.
The choice of 13-inch vs. 15-inch really just comes down to personal preference if you want a larger screen (obviously). The 15-inch also has much better speakers, but if you’re the type to always wear headphones, it quickly becomes less of an advantage.
Advertisement
The MacBook Air M3 is a jack-of-all-trades, with a balanced combination of performance and power efficiency. It also now supports dual displays with the lid closed, and the storage speed is noticeably faster. You don’t need to think about if this laptop will meet your needs — it just will. Read our review.
The 15-inch MacBook Air is also equipped with Apple’s M3 chip. It features a larger display and better speaker array than the 13-inch MacBook Air M3. Read our review.
Meta AI is traveling internationally, starting with Brazil, Bolivia, Guatemala, Paraguay, the Philippines, and the UK this week. Over the next few weeks, the tech giant’s AI assistant will eventually debut in 21 countries across Africa, Southeast Asia, and the Middle East. Notable in its absence is any continental European country as Meta wrangles with the European Union (EU) over regulatory demands.
Meta hasn’t set a date for releasing Meta AI in the countries beyond the initial list. Still, fairly soon, people in Algeria, Egypt, Indonesia, Iraq, Jordan, Libya, Malaysia, Morocco, Saudi Arabia, Sudan, Thailand, Tunisia, United Arab Emirates, Vietnam, and Yemen will also be able to ask Meta AI their questions. They’ll also be able to create images and even put their face in the results using the “Imagine Me” feature for creating a digital avatar based on uploaded photos that can then be incorporated into an image created from a text prompt. Those images can then be edited by follow-up prompts.
The wider geographic access to Meta AI will include linguistic expansion. That means the new international users won’t be limited to communicating with the AI assistant in English. The first new language on the list is Tagalog, which is spoken by many in the Philippines. Arabic, Indonesian, Thai, and Vietnamese are next on Meta AI’s roadmap, appropriate for the countries gaining access to the AI assistant, which are also places with many people who regularly use Facebook, Instagram, WhatsApp, and other Meta platforms. Those same people will presumably form a core of Meta AI early adopters.
“Following this gradual rollout, Meta AI will be available in 43 countries and a dozen languages. That means more people than ever will be able to use Meta AI to dive deep on topics that spark their interest, get helpful how-tos and find inspiration for art projects, home decor, OOTDs and more,” Meta boasted in a blog post. “In fact, Meta AI is already on track to become the most used AI assistant in the world by the end of the year with almost 500 million active users monthly.”
Meta’s products are also popular in the EU, but the lack of EU expansion plans isn’t surprising. Meta has already said it won’t bring new AI tools, especially image- and video-related ones because it is leery of violating new regulations and compliance demands that it feels are unfair.
Advertisement
The biggest issue raised by European lawmakers surrounds ethical data use and privacy. The recently adopted rules are designed to slow the launch of AI technology that doesn’t address those potential problems. Meta AI’s current form might transgress some of the new rules. In response, Meta CEO Mark Zuckerberg has suggested that these regulations limit innovation and hurt citizens. For now, Meta has chosen to skip the EU in favor of other markets for its AI products.
Meta AI is also expanding in another direction this week, debuting on the Ray-Ban Meta smart glasses in more regions, which are now available in the UK and Australia. Though Australians will have the full range of features, the UK is only getting voice support for now. The wearable tech integration with Meta AI is part of the company’s push to embed its AI in everything it produces. Meta didn’t say why the UK isn’t getting the augmented reality overlay or image recognition features immediately. Presumably, there are technical issues, regulatory hurdles, or both that need to be overcome.
Sign up for breaking news, reviews, opinion, top tech deals, and more.
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25212415/246951_Asus_Zenbook_Duo_ABenedetto_0213.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25212415/246951_Asus_Zenbook_Duo_ABenedetto_0213.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25543875/_247201_Samsung_Galaxy_Book4_Edge_review_JNELIUS_0003_VERGEDOTCOM.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25543875/_247201_Samsung_Galaxy_Book4_Edge_review_JNELIUS_0003_VERGEDOTCOM.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25456810/Microsoft_Surface_Pro_2024_Press_Image.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25456810/Microsoft_Surface_Pro_2024_Press_Image.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25456824/Microsoft_Surface_Laptop_2024_Press_Image.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25456824/Microsoft_Surface_Laptop_2024_Press_Image.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25666655/asus_tuf_gaming_a16.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25666655/asus_tuf_gaming_a16.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25084047/236896_MacBook_Pro_14_and_16_M3_AKrales_0159.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25084047/236896_MacBook_Pro_14_and_16_M3_AKrales_0159.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25333645/247043_Apple_MacBook_Air_2024_AKrales_0058.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25333645/247043_Apple_MacBook_Air_2024_AKrales_0058.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25356162/DSC_0504.jpg)
:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/25356162/DSC_0504.jpg)
You must be logged in to post a comment Login