Connect with us

Technology

Differentiable Adaptive Merging is accelerating SLMs for enterprises

Published

on

Differentiable Adaptive Merging is accelerating SLMs for enterprises

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


Model merging is a fundamental AI process that enables organizations to reuse and combine existing trained models to achieve specific goals.

There are various ways that enterprises can use model merging today, but many approaches are complex. A new approach known as Differentiable Adaptive Merging (DAM) could be the answer, providing a solution to the current challenges of model merging. DAM offers an innovative solution to combining AI models while potentially reducing computational costs.

Arcee AI, a company focusing on efficient, specialized small language models, is leading the charge on DAM research. The company, which raised funding in May 2024, has evolved from providing model training tools to becoming a full-fledged model delivery platform with both open-source and commercial offerings.

Advertisement

How DAM creates a new path forward for model merging

Merging can help companies combine models specialized in different areas to create a new model capable in both areas.

The basic concept of merging data is very well understood with structured data and databases. However, merging models is more abstract than merging structured data, as the internal representations of the models are not as interpretable.

Thomas Gauthier-Caron, research engineer at Arcee AI and one of the authors of the DAM research explained to VentureBeat that traditional model merging has often relied on evolutionary algorithms. That approach can potentially be slow and unpredictable. DAM takes a different approach by leveraging established machine learning (ML) optimization techniques.

Gauthier-Caron explained that DAM aims to solve the problem of complexity in the model merging process. The company’s existing library, MergeKit, is useful for merging different models, but it is complex due to the various methods and parameters involved.

Advertisement

“We were wondering, can we make this easier, can we get the machine to optimize this for us, instead of us being in the weeds tweaking all of these parameters?” Gauthier-Caron said.

Instead of just mixing the models directly, DAM adjusts based on how much each model contributes. DAM uses scaling coefficients for each column in the models’ weight matrices. It automatically learns the best settings for these coefficients by testing how well the combined model performs, comparing the output with the original models and then adjusting the coefficients to get better results.

According to the research, DAM performs competitively with or better than existing methods like evolutionary merging, DARE-TIES and Model Soups. The technology represents a significant departure from existing approaches, according to Gauthier-Caron. He described evolutionary merging as a slow process, where it’s not entirely clear up front how good the result will be or how long the merge process should run.

Merging is not an Mixture of Experts approach

Data scientists combine models in many different ways. Among the increasingly popular approaches is the Mixture of Experts (MoE).

Advertisement

Gauthier-Caron emphasized model merging with DAM is something very different from MoE. He explained that MoE is a specific architecture that can be used to train language models. 

The basic concept behind model merging is that it starts from the point where the organization already has trained models. Training these models usually costs a lot of money, so engineers aim to reuse existing trained models.

Practical applications and benefits of DAM for enterprise AI

One of DAM’s key advantages is its ability to combine specialized models efficiently. 

One such example provided by Gauthier-Caron is if an organization wanted to combine a Japanese model with a math model. The goal of that combination is to make a model that’s good at math in Japanese, without the need to retrain. That’s one area where DAM can potentially excel.

Advertisement

The technology is particularly relevant for enterprise adoption of generative AI, where efficiency and cost considerations are paramount. Helping to create more efficient ways of operating at reduced cost is a key goal for Arcee overall. That’s why DAM research is important to both the company and ultimately its users too.

“Enterprise adoption of gen AI boils down to efficiency, availability, scalability and cost,” Mark McQuade, co-founder and CEO of Arcee AI told VentureBeat.


Source link
Continue Reading
Advertisement
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Technology

Australian cricketer Steve Waugh at Bengaluru Space Expo- The Week

Published

on

Former Australian cricketer Steve Waugh says that he is learning a lot about space technology and is very excited about it. He was a surprise at the Bengaluru Space Expo (BSX) 2024 that began today. “I am surprised myself to be at the space expo. Space is exciting and new for me and I am learning a lot about space as I go along. I am very happy to be involved in this joint venture between Austraila and India. I have been coming to India for the last forty years with charity, with cricket, with business and this is another opportunity to collaborate with India. I am excited to be involved in it and Australia and India can do great things together in space technology,” said Waugh who is also the brand ambassador of Space Machines Company an Australian India in space servicing firm. 

The Space Machines Company has forged strategic partnerships with two Indian companies Ananth Technologies and Digantara. These partnerships are expected to play a significant role in the upcoming Space MAITRI (Mission for Australia-India’Technology, Research and Innovation) mission and the launch of Space Machines Company’s second Optimus satellite. Scheduled for 2026, the satellite will be deployed abroad NewSpace India Limited (NSIL) Small Satellite Launch Vehicle (SSLV).  

It will be the largest Australian-made spacecraft in orbit. The Space Machines Company’s second Optimus spacecraft, a 450 kg Orbital Servicing Vehicle will be launched on NSIL’s Small Satellite Launch Vehicle and is part of the first dedicated launch agreement between Australia and India. 

The mission will focus on debris management and sustainability and will significantly advance Australia’s domestic space industry, by combining Australian spacecraft capabilities with India’s launch expertise. 

Advertisement

“We will work closely with Ananth Technologies and Digantara throughout the space MAITRI project lifecycle, leveraging each company’s advanced engineering, logisitc, and situation space awareness capabilities to fulfil the joint Australian-Indian mission of building a more sustainable space future,” said Rajat Kulshrestha, CEO and Co-founder of Space Machines Company. 

Under the partnership, Ananth Technologies will provide Assembly Integration and Testing (AIT) and comprehensive engineering and logistics support throughout the Space MAITRI program. This will include the safe transportation and handling of all spacecraft components in India, extensive testing and launch site spacecraft fueling. This collaboration with Ananth Technologies between the two companies will ensure that Space Machines Company’s second Optimus spacecraft is successfully integrated into the SSLV and ready for launch. 

On the other hand, the collaboration with Digantara will enable the Optimus spacecraft to track and engage short range resident space objects a vital capability when executing close approach maneuvers during in orbit operations. 

Interestingly the Australian government has invested $ 8.5 million in the Space MAITRI mission in April 2024 through the Australian Space Agency’s $18 million International Space Investment India Projects program. “This mission and the collaborations that underpin it emphasise the role that space can play in enhancing cooperation in the Indo-Pacific region for mutual benefit. This mission leverages our nation’s respective capabilities and advantages to make space activities more sustainable-something the global space community is focussed on to protect and maintain the assets in orbit that are central to a functioning modern society,” remarked Enrico Palermo, head of the Australian Space Agency. 

Advertisement

Space Machines Company is an Australian company that delivers on orbit servicing and protection of critical space infrastructure through its Orbital Servicing Network. This company supports mobility, inspection, deorbiting, repair, life extension and protection capbaility to satellite customers when and where they need it. 

Source link

Continue Reading

Technology

The Call review: Musical AI harmonises with your voice in a transcendent new exhibition

Published

on

Leon Chew, The Call, Holly Herndon and Mat Dryhurst with sub, Serpentine, 2024
Leon Chew, The Call, Holly Herndon and Mat Dryhurst with sub, Serpentine, 2024

A chandelier evokes the rig of microphones used to capture the recordings

Leon Chew, The Call, Holly Herndon and Mat Dryhurst with sub, Serpentine, 2024

The Call
Holly Herndon and Mat Dryhurst
Serpentine North, London Closes 2 February 2025

Step into London’s Serpentine North gallery and the first thing you see is an organ. But it is far from a conventional instrument with gleaming flues and reeds. This organ is made up of fans used to cool graphics processing units. Each fan whirs at a pitch that depends on its oscillation, and the sounds combine in an otherworldly hymn…

Advertisement

Source link

Continue Reading

Science & Environment

What happened when a rock as big as London hit Earth?

Published

on

What happened when a rock as big as London hit Earth?


Getty Images Artwork of an asteroid hitting earthGetty Images

The meteorite was 40-60km in diameter and left a crater 500km across

A huge meteorite first discovered in 2014 caused a tsunami bigger than any in known human history and boiled the oceans, scientists have discovered.

The space rock, which was 200 times the size of the one that wiped out the dinosaurs, smashed into Earth when our planet was in its infancy three billion years ago.

Carrying sledge hammers, scientists hiked to the impact site in South Africa to chisel off chunks of rock to understand the crash.

The team also found evidence that massive asteroid impacts did not bring only destruction to Earth – they helped early life thrive.

Advertisement

“We know that after Earth first formed there was still a lot of debris flying around space that would be smashing into Earth,” says Prof Nadja Drabon from Harvard university, lead author of the new research.

“But now we have found that life was really resilient in the wake of some of these giant impacts, and that it actually bloomed and thrived,” she says.

The meteorite S2 was much larger than the space rock we are most familiar with. The one that led to the dinosaurs’ extinction 66 million years ago was about 10km wide, or almost the height of Mount Everest.

But S2 was 40-60km wide and its mass was 50-200 times greater.

Advertisement

It struck when Earth was still in its early years and looked very different. It was a water world with just a few continents sticking out of the sea. Life was very simple – microorganisms composed of single cells.

Nadja Drabon Nadja and her colleagues went to the Eastern Barberton Greenstone Belt in South Africa to collect rock samplesNadja Drabon

Nadja and her colleagues went to the Eastern Barberton Greenstone Belt in South Africa to collect rock samples

The impact site in Eastern Barberton Greenbelt is one of the oldest places on Earth with remnants of a meteorite crash.

Prof Drabon travelled there three times with her colleagues, driving as far as possible into the remote mountains before hiking the rest of the way with backpacks.

Rangers accompanied them with machine guns to protect them against wild animals like elephants or rhinos, or even poachers in the national park.

Advertisement

They were looking for spherule particles, or tiny fragments of rock, left behind by impact. Using sledge hammers, they collected hundreds of kilograms of rock and took them back to labs for analysis.

Prof Drabon stowed the most precious pieces in her luggage.

“I usually get stopped by security, but I give them a big spiel about how exciting the science is and then they get really bored and let me through,” she says.

Nadja Drabon Nadja and her colleagues in the Eastern Barberton Greenstone Belt in South AfricaNadja Drabon

The team travelled with rangers who could protect them from wild animals like elephants or rhinos

The team have now re-constructed just what the S2 meteorite did when it violently careened into Earth. It gouged out a 500km crater and pulverised rocks that ejected at incredibly fast speeds to form a cloud that circled around the globe.

Advertisement

“Imagine a rain cloud, but instead of water droplets coming down, it’s like molten rock droplets raining out of the sky,” says Prof Drabon.

A huge tsunami would have swept across the globe, ripped up the sea floor, and flooded coastlines.

The 2004 Indian Ocean tsunami would have paled in comparison, suggests Prof Drabon.

All that energy would have generated massive amounts of heat that boiled the oceans causing up to tens of metres of water to evaporate. It would also have increased air temperatures by up to 100C.

Advertisement

The skies would have turned black, choked with dust and particles. Without sunlight penetrating the darkness, simple life on land or in shallow water that relied on photosynthesis would have been wiped out.

Nadja Drabon A rock from the seafloor with a pen for scaleNadja Drabon

The team of geologists analysed rock showing evidence of ripped up seafloor

These impacts are similar to what geologists have found about other big meteorite impacts and what was suspected for S2.

But what Prof Drabon and her team found next was surprising. The rock evidence showed that the violent disturbances churned up nutrients like phosphorus and iron that fed simple organisms.

“Life was not only resilient, but actually bounced back really quickly and thrived,” she says.

Advertisement

“It’s like when you brush your teeth in the morning. It kills 99.9% of bacteria, but by the evening they’re all back, right?” she says.

The new findings suggest that the big impacts were like a giant fertiliser, sending essential ingredients for life like phosphorus around the globe.

The tsunami sweeping the planet would also have brought iron-rich water from the depths to the surface, giving early microbes extra energy.

The findings add to a growing view among scientists that early life was actually helped by the violent succession of rocks striking Earth in its early years, Prof Drabon says.

Advertisement

“It seems that life after the impact actually encountered really favourable conditions that allowed it to bloom,” she explains.

The findings are published in the scientific journal PNAS.



Source link

Continue Reading

Technology

NYT Mini Crossword today: puzzle answers for Thursday, October 24

Published

on

NYT Mini Crossword today: puzzle answers for Saturday, September 21

The New York Times has introduced the next title coming to its Games catalog following Wordle’s continued success — and it’s all about math. Digits has players adding, subtracting, multiplying, and dividing numbers. You can play its beta for free online right now. 
In Digits, players are presented with a target number that they need to match. Players are given six numbers and have the ability to add, subtract, multiply, or divide them to get as close to the target as they can. Not every number needs to be used, though, so this game should put your math skills to the test as you combine numbers and try to make the right equations to get as close to the target number as possible.

Players will get a five-star rating if they match the target number exactly, a three-star rating if they get within 10 of the target, and a one-star rating if they can get within 25 of the target number. Currently, players are also able to access five different puzzles with increasingly larger numbers as well.  I solved today’s puzzle and found it to be an enjoyable number-based game that should appeal to inquisitive minds that like puzzle games such as Threes or other The New York Times titles like Wordle and Spelling Bee.
In an article unveiling Digits and detailing The New York Time Games team’s process to game development, The Times says the team will use this free beta to fix bugs and assess if it’s worth moving into a more active development phase “where the game is coded and the designs are finalized.” So play Digits while you can, as The New York Times may move on from the project if it doesn’t get the response it is hoping for. 
Digits’ beta is available to play for free now on The New York Times Games’ website

Source link

Continue Reading

Science & Environment

Polar bears face higher risk of disease in a warming Arctic

Published

on

Polar bears face higher risk of disease in a warming Arctic


USGS A polar bear mother and cubs USGS

In a warming Arctic, polar bears are spending more of their time on land

As the Arctic warms, polar bears face a growing risk of contracting viruses, bacteria and parasites that they were less likely to encounter just 30 years ago, research has revealed.

In a study that has provided clues about how polar bear disease could be linked to ice loss, scientists examined blood samples from bears in the Chukchi Sea – between Alaska and Russia.

They analysed samples that had been gathered between 1987 and 1994, then collected and studied samples three decades later – between 2008 and 2017.

The researchers found that significantly more of the recent blood samples contained chemical signals that bears had been infected with one of five viruses, bacteria or parasites.

Advertisement

USGS Wildlife biologist Dr Karyn Rode from the US Geological Survey checks on a sedated wild polar bear in the Alaskan Arctic  USGS

Wildlife biologist Karyn Rode (here with a sedated wild polar bear) and her colleagues collected blood samples from wild bears to monitor the animals’ health

It is difficult to know, from blood samples, how the bears’ physical health was affected, but wildlife biologist Dr Karyn Rode from the US Geological Survey said it showed that something was changing throughout the whole Arctic ecosystem.

The researchers tested for six different pathogens in total – viruses, bacteria or parasites that are primarily associated with land-based animals but have been recorded before in marine animals, including species that polar bears hunt.

The study covered three decades, Dr Rode said, “when there had been a substantial loss of sea ice and there’s been increased land use in [this population of polar bears]”.

“So we wanted to know if exposure had changed – particularly for some of these pathogens that we think are primarily land-oriented.”

Advertisement

The five pathogens, as disease-causing agents are collectively called, that have become more common in polar bears, are two parasites that cause toxoplasmosis and neosporosis, two types of bacteria that cause rabbit fever and brucellosis, and the virus that causes canine distemper.

“Bears in general are pretty robust to disease,” explained Dr Rode. “It’s not typically been known to affect bear population, but I think what it just highlights is that things [in the Arctic] are changing.”

Key polar bear facts

  • There are about 26,000 polar bears left in the world, with the majority in Canada. Populations are also found in the US, Russia, Greenland and Norway
  • Polar bears are listed as vulnerable to extinction by the International Union for Conservation of Nature, with climate change a key factor in their decline
  • Adult males can grow to be around 3m long and can weigh close to 600kg
  • Polar bears can eat up to 45kg of blubber in one sitting
  • These bears have a powerful sense of smell and can sniff out prey from up to 16km away
  • They are strong swimmers and have been spotted up to 100km offshore. They can swim at speeds of around 10km per hour, due in part to their paws being slightly webbed
USGS A group of polar bears captured from a collar camera USGS

Studies with collar cameras have revealed what polar bears eat during the ice-free summer, as well as capturing surprising social interactions

In the US, polar bears are classified as a threatened species; scientists say the biggest threat to their future is the continuing loss of sea ice habitat, which they depend on as a platform from which to pounce on their marine prey.

Advertisement

Previous research using collar cameras on bears has shown that, as they spend more of the year on land – when there is no available sea ice to hunt from – the bears are unable to find enough calories.

Dr Rode explained that polar bears are top predators: “Our study suggested that they’re getting their exposure to some pathogens primarily through their prey species.

“So what we saw as changes in pathogen exposure for polar bears is indicative of changes that other species are also experiencing.”

The findings are published in the scientific journal PLOS One.

Advertisement

Thin, green banner promoting the Future Earth newsletter with text saying, “Get the latest climate news from the UK and around the world every week, straight to your inbox”. There is also a graphic of an iceberg overlaid with a green circular pattern.



Source link

Continue Reading

Technology

Mistral AI & Qualcomm partner will boost AI on Snapdragon devices

Published

on

Mistral AI & Qualcomm partner will boost AI on Snapdragon devices

Qualcomm is one of the companies that has been driving the development of artificial intelligence in the tech industry. The company offers powerful AI processing capabilities for both laptops and mobile devices with its Snapdragon chips. There is also the Qualcomm AI Hub, which makes it easier for developers to access multiple AI models from a single site. Now, Qualcomm has announced Mistral AI as a new partner in the integration of more AI models on Snapdragon hardware.

The market for AI models is witnessing an increasing number of alternatives. New companies have emerged to compete with their own models adapted to different needs. For example, Meta recently presented an AI model capable of autonomously evaluating and training other AI models. There is also Personal AI that enables offline assistant experiences with a business focus on Snapdragon-powered laptops.

Mistral AI is the latest Qualcomm partner to bring AI experiences to Snapdragon-powered devices

The Mistral AI models bear similarities to Personal AI, but they cater to a wider audience and possess sufficient versatility to seamlessly integrate with devices such as PCs, smartphones, and vehicles.

Qualcomm has announced the optimization of the Mistral AI models for its multiple hardware platforms. These include the Snapdragon 8 Elite, Snapdragon Cockpit Elite, Snapdragon Ride Elite, and Snapdragon X Elite. For reference, the Mistral AI models share a similar goal to that of the Gemini Nano. That is, the Mistral AI models are designed to be low-power models, making them ideal for enabling on-device AI experiences on mobile devices. However, Mistral AI asserts that its models are also compatible with cars.

Advertisement

“Mistral AI’s Ministral 3B and Ministral 8B will enable device manufacturers, software vendors, and digital service providers to deliver innovative experiences, such as AI assistants and other applications that understand users’ wants and needs, thanks to the immediacy, reliability, and enhanced privacy of on-device AI,” said Durga Malladi, senior vice president and general manager of technology at Mistral AI.

Mistral 7B v0.3 now available on Qualcomm AI Hub

Currently, the Mistral 7B v0.3 model is available on the Qualcomm AI Hub platform. Therefore, developers can now access this model to create experiences specifically tailored for the Snapdragon hardware. On the other hand, the Ministral 3B and Ministral 8B models will be available soon.

Source link

Continue Reading

Trending

Copyright © 2024 WordupNews.com