Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
Retrieval-augmented generation (RAG) has become a popular method for grounding large language models (LLMs) in external knowledge. RAG systems typically use an embedding model to encode documents in a knowledge corpus and select those that are most relevant to the user’s query.
However, standard retrieval methods often fail to account for context-specific details that can make a big difference in application-specific datasets. In a new paper, researchers at Cornell University introduce “contextual document embeddings,” a technique that improves the performance of embedding models by making them aware of the context in which documents are retrieved.
The limitations of bi-encoders
The most common approach for document retrieval in RAG is to use “bi-encoders,” where an embedding model creates a fixed representation of each document and stores it in a vector database. During inference, the embedding of the query is calculated and compared to the stored embeddings to find the most relevant documents.
Advertisement
Bi-encoders have become a popular choice for document retrieval in RAG systems due to their efficiency and scalability. However, bi-encoders often struggle with nuanced, application-specific datasets because they are trained on generic data. In fact, when it comes to specialized knowledge corpora, they can fall short of classic statistical methods such as BM25 in certain tasks.
“Our project started with the study of BM25, an old-school algorithm for text retrieval,” John (Jack) Morris, a doctoral student at Cornell Tech and co-author of the paper, told VentureBeat. “We performed a little analysis and saw that the more out-of-domain the dataset is, the more BM25 outperforms neural networks.”
BM25 achieves its flexibility by calculating the weight of each word in the context of the corpus it is indexing. For example, if a word appears in many documents in the knowledge corpus, its weight will be reduced, even if it is an important keyword in other contexts. This allows BM25 to adapt to the specific characteristics of different datasets.
“Traditional neural network-based dense retrieval models can’t do this because they just set weights once, based on the training data,” Morris said. “We tried to design an approach that could fix this.”
Advertisement
Contextual document embeddings
Contextual document embeddings Credit: arXiv
The Cornell researchers propose two complementary methods to improve the performance of bi-encoders by adding the notion of context to document embeddings.
“If you think about retrieval as a ‘competition’ between documents to see which is most relevant to a given search query, we use ‘context’ to inform the encoder about the other documents that will be in the competition,” Morris said.
The first method modifies the training process of the embedding model. The researchers use a technique that groups similar documents before training the embedding model. They then use contrastive learning to train the encoder on distinguishing documents within each cluster.
Contrastive learning is an unsupervised technique where the model is trained to tell the difference between positive and negative examples. By being forced to distinguish between similar documents, the model becomes more sensitive to subtle differences that are important in specific contexts.
The second method modifies the architecture of the bi-encoder. The researchers augment the encoder with a mechanism that gives it access to the corpus during the embedding process. This allows the encoder to take into account the context of the document when generating its embedding.
Advertisement
The augmented architecture works in two stages. First, it calculates a shared embedding for the cluster to which the document belongs. Then, it combines this shared embedding with the document’s unique features to create a contextualized embedding.
This approach enables the model to capture both the general context of the document’s cluster and the specific details that make it unique. The output is still an embedding of the same size as a regular bi-encoder, so it does not require any changes to the retrieval process.
The impact of contextual document embeddings
The researchers evaluated their method on various benchmarks and found that it consistently outperformed standard bi-encoders of similar sizes, especially in out-of-domain settings where the training and test datasets are significantly different.
“Our model should be useful for any domain that’s materially different from the training data, and can be thought of as a cheap replacement for finetuning domain-specific embedding models,” Morris said.
Advertisement
The contextual embeddings can be used to improve the performance of RAG systems in different domains. For example, if all of your documents share a structure or context, a normal embedding model would waste space in its embeddings by storing this redundant structure or information.
“Contextual embeddings, on the other hand, can see from the surrounding context that this shared information isn’t useful, and throw it away before deciding exactly what to store in the embedding,” Morris said.
The researchers have released a small version of their contextual document embedding model (cde-small-v1). It can be used as a drop-in replacement for popular open-source tools such as HuggingFace and SentenceTransformers to create custom embeddings for different applications.
Morris says that contextual embeddings are not limited to text-based models can be extended to other modalities, such as text-to-image architectures. There is also room to improve them with more advanced clustering algorithms and evaluate the effectiveness of the technique at larger scales.
Advertisement
VB Daily
Stay in the know! Get the latest news in your inbox daily
I squeezed this server in a rack mounted case, then water cooled it….
Watch me take the server and build it into this rack mounted case step by step.
Asus Z10PE D16 Motherboard
two xeon E5-2683 V3 28c 56T
Kingston 16GB 2133MHz ECC Buffered DDR4
Custom Watercooling setup
To see me show of the server hardware and test the cpu watch here:
To watch the explanation video on the live stream click here:
Check out the live stream here:
Thanks for watching
Check out Josephine’s channel
https://www.youtube.com/channel/UChRIIUpVIG-HMIYoMM3WSSw
Check out and subscribe to my Twitch channel
https://www.twitch.tv/cc_davion
minecraft server
best minecraft servers
dell server
free minecraft server
free minecraft server hosting
iis
load balancers
microsoft exchange
minecraft server hosting
ntp server
server
server rack
synology nas
vps
web server
apache http server
apache server
apache web server
apache windows
application server
aternos minecraft
aternos minecraft server
aws fsx
backup server
bare metal server
bedrock server
best minecraft server hosting
best nas for plex
bitvise ssh
bukkit server
cheap dedicated server
cheap minecraft server hosting
cisco server
client server
client server network
cloud server
conan exiles servers
craftbukkit
dedicated server
dedicated server hosting
dell poweredge
dell poweredge 2950
dell poweredge r710
dell poweredge t30
dell r610
dell r640
dell r710
dell r720
dell r740
dell t30
dhcp option 43
diskstation ds918+
diy rack server
dlna media server
dlna server
dnsmasq
dovecot
esxi server
exchange server
fileserver
free minecraft hosting
free server
free server hosting
free vps
ftp server
ftp site
fun minecraft servers
game hosting
game server hosting
game servers
good minecraft servers
google ntp server
gpu server
gta 5 roleplay servers
hp ilo
hp microserver
hp servers
hpe simplivity
http server
ibm iseries
iis server
internet server
iseries
ldap server
lenovo server
lighttpd
maplestory private server
mc server
microserver
microsoft exchange mail
microsoft exchange server
microsoft iis
microsoft server
minecraft 1.14 server
minecraft bedrock server
minecraft creative servers
minecraft hosting
minecraft java server
minecraft private server
mini server
modded minecraft server hosting
network server
new minecraft servers
ntp pool
ntp time server
online server
op prison servers
osrs private server
plex docker
plex nas
plex synology
poweredge
poweredge r710
printer server
private server
public ntp server
qnap nas
r620
r710
rack case
rack mount
rack mount pc case
rack mount server
rack mount wall
rack mounted case
raspberry pi media server
rsps
runescape private server
rustafied vip
server client
server hosting
skyfactory 4 server
small business server
sql server 2014
starbound servers
storage server
supermicro server
terminal server
tftp
tftpd32
time server
tinyproxy
tower server
vanilla minecraft server
virtual private server
virtual server
vps cheap
vps google
vps server
wall mounted server rack
websphere application server
wireless print server
xming
12u rack case
12u server rack
15 u rack
15u server rack
16u rack case
19 inch rack
19 rack enclosures
19 wall mount rack
1u inch
1u rack
1u rack case
1u rack mount
1u rackmount case
1u server
2 space rack case
2u rack case
2u rack server
2u rackmount
2u rackmount case
2u wall mount rack
3 u rack
3u rack case
4 space rack case
42u rack
4u rackmount
4u rackmount case
4u server rack
5 u rack
5u rack case
6 space rack case
6 u rack
6u rack case
6u server rack
8 u rack
8u rack case
amp rack case
apache http
apache proxy
apache proxypass
apache traffic server
apple xserve
atlas server
atx rackmount case
audio rack case
axigen
azure devops server 2019
best vps
bitvise ssh server
buy server
caddy server
centos server
certbot apache
cloud server hosting
cloud vps
computer rack case
computer rack mount
computer room air conditioning
contabo vps
data center equipment
data center server rack
dedicated hosting
dell idrac
dell poweredge 1950
dell poweredge r230
dell poweredge r440
dell poweredge r610
dell poweredge r620
dell poweredge r630
dell poweredge r640
dell poweredge r720
dell poweredge r730
dell poweredge r740
dell poweredge t130
dell r210
dell r230
dell r330
dell r410
dell r430
dell r440
dell r510
dell r540
dell r620
dell r630
dell r730
dell r740xd
dell rack mount pc
dell rack mount servers
dell t130
dell t20
dell t330
dell t440
dhcp server windows
dhcp server windows 10
dhcpd
diskstation ds218
diskstation ds218+
diskstation ds718+
dl180
dl360
dl360 gen10
dl360 gen10 quickspecs
dl360 gen9
dl380
dl380 g7
dl380 gen10
dl380 gen10 quickspecs
dl380 gen9
dl380p gen8
domain server
ds1515+
ds1815+
ds216
ds416slim
edge server
ejabberd
esx server
exchange 2010
exchange 2013
exchange 2016
exchange server 2016
exchange server 2019
exchange web services
external dns
free ftp server online
free mc server
free sftp server
free vps hosting
free vps server .
Despite how you may feel about the price of groceries, gas, and other essential goods right now, you can actually get quite a bit for under $25. That’s even truer during a major shopping event like Amazon’s Prime Big Deal Days, which only has a few hours remaining.
There are only a few hours left to shop, but we’re continuing to dig through the online bargain bin to find an assortment of inexpensive, Verge-approved tech that comes in under $25. Right now, for instance, you can get your smart home started with a fifth-gen Echo Dot or an Amazon Smart Plug. Need something to watch? The standard Fire TV Stick 4K is a great conduit for streaming endless entertainment. Or you can pick up Prince of Persia: The Lost Crown for its lowest price to date. There’s even some unplugged fun to be had with tabletop party games like Love Letter. All of that and more can be found below.
Update, October 9th: Updated to reflect current availability and the fact Amazon’s fall Prime Day event is almost over.
Strands is the NYT’s latest word game after the likes of Wordle, Spelling Bee and Connections – and it’s great fun. It can be difficult, though, so read on for my Strands hints.
SPOILER WARNING: Information about NYT Strands today is below, so don’t read on if you don’t want to know the answers.
Your Strands expert
Your Strands expert
Marc McLaren
NYT Strands today (game #221) – hint #1 – today’s theme
What is the theme of today’s NYT Strands?
• Today’s NYT Strands theme is… Think about it
Advertisement
NYT Strands today (game #221) – hint #2 – clue words
Play any of these words to unlock the in-game hints system.
MASS
PASS
MAST
PAST
PASTY
PASTRY
NYT Strands today (game #221) – hint #3 – spangram
What is a hint for today’s spangram?
• Where’s Strands?
NYT Strands today (game #221) – hint #4 – spangram position
What are two sides of the board that today’s spangram touches?
First: left, 4th row
Last: right, 4th row
Right, the answers are below, so DO NOT SCROLL ANY FURTHER IF YOU DON’T WANT TO SEE THEM.
Advertisement
NYT Strands today (game #221) – the answers
(Image credit: New York Times)
The answers to today’s Strands, game #221, are…
JUMBLE
SUDOKU
JIGSAW
MAZE
CROSSWORD
CRYPTOGRAM
SPANGRAM: PUZZLES
My rating: Easy
My score: Perfect
Give me a puzzle and I’ll solve it; give me a puzzle about puzzles and I’ll solve it and enjoy every second while doing so. Today’s Strands is exactly that – a puzzle about puzzles. I was somewhat surprised not to see Wordle, Connections or indeed Strands in there, but the likes of SUDOKU, JUMBLE and CROSSWORD were all obvious enough that this didn’t cause me any trouble.
Sign up for breaking news, reviews, opinion, top tech deals, and more.
Yesterday’s NYT Strands answers (Wednesday 9 October, game #220)
HALL
OFFICE
STUDY
KITCHEN
LIBRARY
PANTRY
FOYER
SPANGRAM: HOMEROOM
What is NYT Strands?
Strands is the NYT’s new word game, following Wordle and Connections. It’s now out of beta so is a fully fledged member of the NYT’s games stable and can be played on the NYT Games site on desktop or mobile.
I’ve got a full guide to how to play NYT Strands, complete with tips for solving it, so check that out if you’re struggling to beat it each day.
ATTOM Micro Modular Data Center
Attom Micro Data Center is a plug and play, self-contained and fully integrated solution to protect your critical IT assets with built in Cabinet, Power, Cooling, Monitoring, Fire and Security systems.
ATTOM Micro Data Center has the good solution in the Indoor, Outdoor area and Container system.
ABBA-RACK Rack Server Indonesia adalah solusi terbaik dan terlengkap untuk kebutuhan rack server, rack network, rack telco, rack outdoor dan kebutuhan data center anda.
Dengan senang hati kami mempersembahkan produk-produk ABBA-RACK untuk anda.
ABBA RACK – Server Rack Solutions merupakan solusi terbaik kebutuhan rack server dengan pilihan produk terlengkap didukung dengan desain dan fitur terbaru.
ABBA RACK memiliki desain lebih fleksibel yaitu bisa di kirim dalam kondisi terakit dan bisa juga dalam kondisi belum terakit (knock down).
ABBA RACK mempunyai solusi untuk penggunaan Indoor dan Outdoor.
ABBA RACK Products :
1. Closed Rack : Glass Door, Perforated Door, Split Door
2. Wallmount Rack : Single Door, Double Door
3. Open Rack : Standar, High Density Open Entry Rack
4. Rack AC
5. Colocation Rack : 2, 3, 4 compartment
6. Cold / Hot Aisle Containment System
7. Cages For Data Center
8. Accessories Server Rack
9. Outdoor Cabinet
10. Outdoor Wallmount Rack
11. Telco Rack
12. Battery Rack
13. Micro Data Center
14. Outdoor Micro Data Center
15. Container Micro Data Center
ABBA-RACK Rack Server berkualitas
Solusi Rack Server yang lengkap
Harga Rack Server yang menarik
Layanan Terbaik dan Pengiriman Tepat Waktu
PT. Uni Network Communications adalah Distributor Tunggal ABBA-RACK – Rack Server Indonesia.
Produk ABBA-RACK :
Closed Rack | Wallmount Rack | Open Rack | Colocation Rack | Cages System for Data center | Cold Aisle Containment System | AC Rack | Outdoor Cabinet | Open Entry Rack | Rack Outdoor | Rack AC | Rack Telco | Rack Baterei | ODF
Silahkan menghubungi kami untuk informasi lebih lanjut.
PT. Uni Network Communications
Jl. Batu Jajar No. 11A, Sawah Besar
Jakarta Pusat – 10120 Indonesia
Phone : +62 21 351 2977
Fax : +62 21 351 2526
Email : sales@abba-rack.com / marketing@unc.co.id
distributor rack server
jual rack server
rack server premium
harga rack server
rack server 20u
rack server 42u
rack server jakarta
rack server indonesia
distributor abba rack
rack server abba
rack server murah
micro data center .
Strands is a brand new daily puzzle from the New York Times. A trickier take on the classic word search, you’ll need a keen eye to solve this puzzle.
Like Wordle, Connections, and the Mini Crossword, Strands can be a bit difficult to solve some days. There’s no shame in needing a little help from time to time. If you’re stuck and need to know the answers to today’s Strands puzzle, check out the solved puzzle below.
You start every Strands puzzle with the goal of finding the “theme words” hidden in the grid of letters. Manipulate letters by dragging or tapping to craft words; double-tap the final letter to confirm. If you find the correct word, the letters will be highlighted blue and will no longer be selectable.
If you find a word that isn’t a theme word, it still helps! For every three non-theme words you find that are at least four letters long, you’ll get a hint — the letters of one of the theme words will be revealed and you’ll just have to unscramble it.
Advertisement
Every single letter on the grid is used to spell out the theme words and there is no overlap. Every letter will be used once, and only once.
Each puzzle contains one “spangram,” a special theme word (or words) that describe the puzzle’s theme and touches two opposite sides of the board. When you find the spangram, it will be highlighted yellow.
The goal should be to complete the puzzle quickly without using too many hints.
Buy Dell PowerEdge R830 Server – https://www.serverbasket.ae/dell-poweredge-r830-rack-server.html
IT Rental: https://www.serverbasket.ae/it-rental.html
For Complete Information on Dell PowerEdge R830 Rack Server, Please Contact Us at:
————————————————————————————————————————————————————
Subscribe To Our Channel @ https://www.youtube.com/channel/UCXxaNw3P6OdkW29dMgTsv2A
————————————————————————————————————————————————————
Check out the Powerful Dell PowerEdge R830 Rack Server.
Buy Refurbished Dell PowerEdge R830 Rack Server from Server Basket as it is Optimal for Growing SME & Demanding Businesses. With High Memory Capacity & Huge Storage Ability, Dell PowerEdge R830 Rack Server is an Ideal Server for Tech Startups & Booming SME Businesses.
-Microsoft Windows Server® 2012 R2
-Microsoft Windows Server 2012
-Novell® SUSE® Linux Enterprise Server
-Red Hat® Enterprise Linux®
-VMware vSphere® ESXi®
Optional Embedded Hypervisors:
-Microsoft Windows Server with Hyper-V®
-Citrix® XenServer®
-VMware vSphere ESXi
I/O Slots:
7 PCIe slots:
• 2 x16 half-length, full-height
• 1 x8 half-length, full-height
• 3 x8 half-length, half-height
• 1 x8 half-length, full-height for the RAID controller
————————————————————————————————————————————————————
Check out the Powerful Dell PowerEdge R830 Rack Server from Server Basket. .
You must be logged in to post a comment Login