Cohere today released two new open-weight models in its Aya project to close the language gap in foundation models. 

Aya Expanse 8B and 35B, now available on Hugging Face, expands performance advancements in 23 languages. Cohere said in a blog post the 8B parameter model “makes breakthroughs more accessible to researchers worldwide,” while the 32B parameter model provides state-of-the-art multilingual capabilities. 

The Aya project seeks to expand access to foundation models in more global languages than English. Cohere for AI, the company’s research arm, launched the Aya initiative last year. In February, it released the Aya 101 large language model (LLM), a 13-billion-parameter model covering 101 languages. Cohere for AI also released the Aya dataset to help expand access to other languages for model training. 

Aya Expanse uses much of the same recipe used to build Aya 101. 

“The improvements in Aya Expanse are the result of a sustained focus on expanding how AI serves languages around the world by rethinking the core building blocks of machine learning breakthroughs,” Cohere said. “Our research agenda for the last few years has included a dedicated focus on bridging the language gap, with several breakthroughs that were critical to the current recipe: data arbitrage, preference training for general performance and safety, and finally model merging.”

Aya performs well

Cohere said the two Aya Expanse models consistently outperformed similar-sized AI models from Google, Mistral and Meta. 

Aya Expanse 32B did better in benchmark multilingual tests than Gemma 2 27B, Mistral 8x22B and even the much larger Llama 3.1 70B. The smaller 8B also performed better than Gemma 2 9B, Llama 3.1 8B and Ministral 8B. 

Cohere developed the Aya models using a data sampling method called data arbitrage as a means to avoid the generation of gibberish that happens when models rely on synthetic data. Many models use synthetic data created from a “teacher” model for training purposes. However, due to the difficulty in finding good teacher models for other languages, especially for low-resource languages. 

It also focused on guiding the models toward “global preferences” and accounting for different cultural and linguistic perspectives. Cohere said it figured out a way to improve performance and safety even while guiding the models’ preferences. 

“We think of it as the ‘final sparkle’ in training an AI model,” the company said. “However, preference training and safety measures often overfit to harms prevalent in Western-centric datasets. Problematically, these safety protocols frequently fail to extend to multilingual settings.  Our work is one of the first that extends preference training to a massively multilingual setting, accounting for different cultural and linguistic perspectives.”

Models in different languages

The Aya initiative focuses on ensuring research around LLMs that perform well in languages other than English. 

Many LLMs eventually become available in other languages, especially for widely spoken languages, but there is difficulty in finding data to train models with the different languages. English, after all, tends to be the official language of governments, finance, internet conversations and business, so it’s far easier to find data in English. 

It can also be difficult to accurately benchmark the performance of models in different languages because of the quality of translations. 

Other developers have released their own language datasets to further research into non-English LLMs. OpenAI, for example, made its Multilingual Massive Multitask Language Understanding Dataset on Hugging Face last month. The dataset aims to help better test LLM performance across 14 languages, including Arabic, German, Swahili and Bengali. 

Cohere has been busy these last few weeks. This week, the company added image search capabilities to Embed 3, its enterprise embedding product used in retrieval augmented generation (RAG) systems. It also enhanced fine-tuning for its Command R 08-2024 model this month. 

Electric vertical takeoff and landing vehicle startup Joby Aviation has launched a public offering to sell up to $200 million of its shares of common stock, per a regulatory filing.

Joby said it will use the proceeds from the raise — together with its existing cash — to fund its certification and manufacturing efforts, prepare for commercial launch in 2025, and for general working capital. 

The eVTOL firm added that it intends to grant the underwriters a 30-day option to purchase an additional $30 million shares of its common stock. 

Joby plans to launch air taxis for urban transportation next year in New York City and Los Angeles alongside partners Delta Air Lines and Uber, as well as in Dubai and Abu Dhabi. The startup also has a $55 million contract with the Department of Defense.

Before Joby can launch, it will need to complete its type certification process to ensure the design of its aircraft meets required safety and airworthiness standards. 

The public offering comes as the Federal Aviation Administration this week cleared the way for eVTOLs to share U.S. airspace with helicopters and airplanes, and set up guidelines for eVTOL pilot training and operating rules. It also follows a $500 million injection from Joby’s existing investor Toyota earlier this month.

Joby has raised $2.6 billion to date, according to PitchBook data.

Joby did not respond immediately to a request for comment. 

Perplexity perpetrates an abuse of intellectual property that harms journalists, writers, publishers and News Corp. The perplexing Perplexity has willfully copied copious amounts of copyrighted material without compensation, and shamelessly presents repurposed material as a direct substitute for the original source. Perplexity proudly states that users can “skip the links” — apparently, Perplexity wants to skip the check.

We applaud principled companies like OpenAI, which understands that integrity and creativity are essential if we are to realise the potential of Artificial Intelligence. Perplexity is not the only AI company abusing intellectual property and it is not the only AI company that we will pursue with vigor and rigor. We have made clear that we would rather woo than sue, but, for the sake of our journalists, our writers and our company, we must challenge the content kleptocracy.

Good morning! Let’s play Connections, the NYT’s clever word game that challenges you to group answers in various categories. It can be tough, so read on if you need clues.

What should you do once you’ve finished? Why, play some more word games of course. I’ve also got daily Wordle hints and answers, Strands hints and answers and Quordle hints and answers articles if you need help for those too.

Boeing’s first astronaut mission ended Friday night with an empty capsule landing and two test pilots still in space, left behind until next year because NASA judged their return too risky.

Six hours after departing the International Space Station, Starliner parachuted into New Mexico’s White Sands Missile Range, descending on autopilot through the desert darkness.

It was an uneventful close to a drama that began with the June launch of Boeing’s long-delayed crew debut and quickly escalated into a dragged-out cliffhanger of a mission stricken by thruster failures and helium leaks. For months, Butch Wilmore and Suni Williams’ return was in question as engineers struggled to understand the capsule’s problems.

Boeing insisted after extensive testing that Starliner was safe to bring the two home, but NASA disagreed and booked a flight with SpaceX instead. Their SpaceX ride won’t launch until the end of this month, which means they’ll be up there until February more than eight months after blasting off on what should have been a quick trip.

Wilmore and Williams should have flown Starliner back to Earth by mid-June, a week after launching in it. But their ride to the space station was marred by the cascade of thruster trouble and helium loss, and NASA ultimately decided it was too risky to return them on Starliner.

So with fresh software updates, the fully automated capsule left with their empty seats and blue spacesuits along with some old station equipment.

She’s on her way home, Williams radioed as the white and blue-trimmed capsule undocked from the space station 260 miles (420 kilometers) over China and disappeared into the black void.

Williams stayed up late to see how everything turned out. A good landing, pretty awesome, said Boeing’s Mission Control.

Cameras on the space station and a pair of NASA planes caught the capsule as a white streak coming in for the touchdown, which drew cheer.

There were some snags during reentry, including more thruster issues, but Starliner made a bull’s-eye landing, said NASA’s commercial crew program manager Steve Stich.

Even with the safe return, I think we made the right decision not to have Butch and Suni on board, Stich said at a news conference early Saturday. All of us feel happy about the successful landing. But then there’s a piece of us, all of us, that we wish it would have been the way we had planned it.

Boeing did not participate in the Houston news briefing. But two of the company’s top space and defense officials, Ted Colbert and Kay Sears, told employees in a note that they backed NASA’s ruling.

“While this may not have been how we originally envisioned the test flight concluding, we support NASA’s decision for Starliner and are proud of how our team and spacecraft performed,” the executives wrote.

Starliner’s crew demo capped a journey filled with delays and setbacks. After the space shuttles retired more than a decade ago, NASA hired Boeing and SpaceX for orbital taxi service. Boeing ran into so many problems on its first test flight with no one aboard in 2019 that it had to repeat it. The 2022 do-over uncovered even more flaws and the repair bill topped $1 billion.

SpaceX’s crew ferry flight later this month will be its 10th for NASA since 2020. The Dragon capsule will launch on the half-year expedition with only two astronauts since two seats are reserved for Wilmore and Williams for the return leg.

As veteran astronauts and retired Navy captains, Wilmore and Williams anticipated hurdles on the test flight. They’ve kept busy in space, helping with repairs and experiments. The two are now full-time station crew members along with the seven others on board.

Even before the pair launched on June 5 from Cape Canaveral, Florida, Starliner’s propulsion system was leaking helium. The leak was small and thought to be isolated, but four more cropped up after liftoff. Then five thrusters failed. Although four of the thrusters were recovered, it gave NASA pause as to whether more malfunctions might hamper the capsule’s descent from orbit.

Boeing conducted numerous thruster tests in space and on the ground over the summer, and was convinced its spacecraft could safely bring the astronauts back. But NASA could not get comfortable with the thruster situation and went with SpaceX.

Flight controllers conducted more test firings of the capsule’s thrusters following undocking; one failed to ignite. Engineers suspect the more the thrusters are fired, the hotter they become, causing protective seals to swell and obstruct the flow of propellant. They won’t be able to examine any of the parts; the section holding the thrusters was ditched just before reentry.

Starliner will be transported in a couple weeks back to NASA’s Kennedy Space Center, where the analyses will unfold.

NASA officials stressed that the space agency remains committed to having two competing U.S. companies transporting astronauts. The goal is for SpaceX and Boeing to take turns launching crews one a year per company until the space station is abandoned in 2030 right before its fiery reentry. That doesn’t give Boeing much time to catch up, but the company intends to push forward with Starliner, according to NASA.

Stich said post-landing it’s too early to know when the next Starliner flight with astronauts might occur.

It will take a little time to determine the path forward,” he said.