Meta made several major announcements for robotics and embodied AI systems this week. This includes releasing benchmarks and artifacts for better understanding and interacting with the physical world. Sparsh, Digit 360 and Digit Plexus, the three research artifacts released by Meta, focus on touch perception, robot dexterity and human-robot interaction. Meta is also releasing PARTNR a new benchmark for evaluating planning and reasoning in human-robot collaboration.

The release comes as advances in foundational models have renewed interest in robotics, and AI companies are gradually expanding their race from the digital realm to the physical world.

There is renewed hope in the industry that with the help of foundation models such as large language models (LLMs) and vision-language models (VLMs), robots can accomplish more complex tasks that require reasoning and planning.

Tactile perception

Sparsh, which was created in collaboration with the University of Washington and Carnegie Mellon University, is a family of encoder models for vision-based tactile sensing. It is meant to provide robots with touch perception capabilities. Touch perception is crucial for robotics tasks, such as determining how much pressure can be applied to a certain object to avoid damaging it. 

The classic approach to incorporating vision-based tactile sensors in robot tasks is to use labeled data to train custom models that can predict useful states. This approach does not generalize across different sensors and tasks.

Meta describes Sparsh as a general-purpose model that can be applied to different types of vision-based tactile sensors and various tasks. To overcome the challenges faced by previous generations of touch perception models, the researchers trained Sparsh models through self-supervised learning (SSL), which obviates the need for labeled data. The model has been trained on more than 460,000 tactile images, consolidated from different datasets. According to the researchers’ experiments, Sparsh gains an average 95.1% improvement over task- and sensor-specific end-to-end models under a limited labeled data budget. The researchers have created different versions of Sparsh based on various architectures, including Meta’s I-JEPA and DINO models.

Touch sensors

In addition to leveraging existing data, Meta is also releasing hardware to collect rich tactile information from the physical. Digit 360 is an artificial finger-shaped tactile sensor with more than 18 sensing features. The sensor has over 8 million taxels for capturing omnidirectional and granular deformations on the fingertip surface. Digit 360 captures various sensing modalities to provide a richer understanding of the environment and object interactions. 

Digit 360 also has on-device AI models to reduce reliance on cloud-based servers. This enables it to process information locally and respond to touch with minimal latency, similar to the reflex arc in humans and animals.

“Beyond advancing robot dexterity, this breakthrough sensor has significant potential applications from medicine and prosthetics to virtual reality and telepresence,” Meta researchers write.

Meta is publicly releasing the code and designs for Digit 360 to stimulate community-driven research and innovation in touch perception. But as in the release of open-source models, it has much to gain from the potential adoption of its hardware and models. The researchers believe that the information captured by Digit 360 can help in the development of more realistic virtual environments, which can be big for Meta’s metaverse projects in the future.

Meta is also releasing Digit Plexus, a hardware-software platform that aims to facilitate the development of robotic applications. Digit Plexus can integrate various fingertip and skin tactile sensors onto a single robot hand, encode the tactile data collected from the sensors, and transmit them to a host computer through a single cable. Meta is releasing the code and design of Digit Plexus to enable researchers to build on the platform and advance robot dexterity research.

Meta will be manufacturing Digit 360 in partnership with tactile sensor manufacturer GelSight Inc. They will also partner with South Korean robotics company Wonik Robotics to develop a fully integrated robotic hand with tactile sensors on the Digit Plexus platform.

Evaluating human-robot collaboration

Meta is also releasing Planning And Reasoning Tasks in humaN-Robot collaboration (PARTNR), a benchmark for evaluating the effectiveness of AI models when collaborating with humans on household tasks. 

PARTNR is built on top of Habitat, Meta’s simulated environment. It includes 100,000 natural language tasks in 60 houses and involves more than 5,800 unique objects. The benchmark is designed to evaluate the performance of LLMs and VLMs in following instructions from humans. 

Meta’s new benchmark joins a growing number of projects that are exploring the use of LLMs and VLMs in robotics and embodied AI settings. In the past year, these models have shown great promise to serve as planning and reasoning modules for robots in complex tasks. Startups such as Figure and Covariant have developed prototypes that use foundation models for planning. At the same time, AI labs are working on creating better foundation models for robotics. An example is Google DeepMind’s RT-X project, which brings together datasets from various robots to train a vision-language-action (VLA) model that generalizes to various robotics morphologies and tasks.

For startups that hope to save the world, or at least make it a better place, balancing impact with profit can be tricky.

“Investor and shareholder expectations are often not aligned with how hard and intractable the problems are that we face as a society,” Allison Wolff, co-founder and CEO of Vibrant Planet, said on the Builders Stage at TechCrunch Disrupt 2024. “I think in some ways, we’re a little bit stuck.”

But it’s not impossible.

Wolff’s company develops cloud-based software for utilities, insurers, and land managers like the U.S. Forest Service to model and respond to wildfire risk. To ensure the company keeps its eye on the mission, it has registered as a public benefit corporation, which requires companies to report on impact in addition to the usual financial information. 

“That’s an elegant structure to consider if you haven’t already, and it’s easy to convert,” she said. “And it’s a good forcing function to do the reporting side of that, to really think through every year, what impact are we having, and how do we account for it.”

Apple announced a new service program to fix iPhone 14 Plus phones that have rear cameras that won’t show a preview.

Apple has determined that the rear camera on a very small percentage of iPhone 14 Plus devices may exhibit no preview. Affected devices were manufactured between April 10, 2023 to April 28, 2024.

If your iPhone 14 Plus is affected — and you can enter your serial number on the program page to see if yours is — Apple says it or an Authorized Service Provider will service your phone for free. If you’ve already paid to have the camera repaired, Apple says to reach out to ask if you can get a refund.

Eligible phones will be covered by this program for three years after they were originally sold.

Quordle was one of the original Wordle alternatives and is still going strong now more than 1,000 games later. It offers a genuine challenge, though, so read on if you need some Quordle hints today – or scroll down further for the answers.

Enjoy playing word games? You can also check out my Wordle today, NYT Connections today and NYT Strands today pages for hints and answers for those puzzles.

MGM+ subscribers are ahead of the curve on From, a sci-fi/horror series that follows people who have been trapped in a small town in Middle America that will literally not let them leave. There are even monsters that come out at night to hunt people for sport, and the human survivors have largely split into two factions: the Township and Colony House. But there’s one thing that almost everyone in the town wants, and that’s a way to escape once and for all.

From is headlined by Harold Perrineau, an actor who knows a thing or two about playing a single father who is cut off from the rest of the world after his stint on Lost. On this series, Perrineau plays Boyd Stevens, the father of Ellis Stevens (Corteon Moore), and the closest thing that the town has to both a sheriff and a mayor. Boyd also has plenty of inner demons to contend with in addition to the creatures lurking inside and outside of the town.

Die-hard fans of From have been hyping up the series ahead of its return for season 3. But for newcomers and veterans alike, we’ve put together this guide to everything you need to know about From season 3. And as soon as we know more, we’ll share it here as well.

What’s coming up in the remaining episodes of From season 3?

Is there a trailer for From season 3?

Has From season 3 begun filming?

We started filming Season 3….
So I think you’re good 👍🏾 #FROMily@FROMonMGM https://t.co/sKxkRP4cWa

— Harold Perrineau (@HaroldPerrineau) December 29, 2023

So far, Perrineau hasn’t shared an update about whether filming has been completed. But he has actively promoted the show on social media.

The official From account on the social media service formerly known as Twitter hasn’t had much new in 2024. The account did share this image of one of the protective talismans back in February in response to, of all things, the announcement of a new Taylor Swift album.

👀 https://t.co/Onulvh0HoB pic.twitter.com/oBjXhrLdHN

— FROM on MGM+ (@FROMonMGM) February 5, 2024

What will happen in From season 3?

This show makes it difficult to predict what will happen next, but the season 2 finale did end on a pretty big cliffhanger. Tabitha is back in the outside world, and that opens up a lot of possibilities. Will she try to warn people about the town? Does she feel obligated to rescue the people that she left behind? And more importantly, can she find the town again without getting trapped herself? Having Tabitha outside of the town also offers the show a chance to reveal whether the disappearance of the various characters who are trapped in the town has been noticed or investigated by the people they knew and loved.

Meanwhile, back in the town, Boyd was warned that fighting the entity would just prolong everyone’s suffering, but he did it anyway. Presumably, there will be consequences for disregarding that warning.

Who is in From’s season 3 cast?

Perrineau leads From season 3’s large ensemble cast. Featured performers include:

• Harold Perrineau as Boyd Stevens
• Catalina Sandino Moreno as Tabitha Matthews
• Eion Bailey as Jim Matthews
• Avery Konrad as Sara Myers
• David Alpay as Jade Herrera
• Elizabeth Saunders as Donna Raines
• Scott McCord as Victor
• Ricky He as Kenny Liu
• Corteon Moore as Ellis Stevens
• Chloe Van Landschoot as Kristi Miller
• Pegah Ghafoori as Fatima Hassan
• Hannah Cheramy as Julie Matthews
• Simon Webster as Ethan Matthews
• Elizabeth Moy as Tian-Chen Liu
• Nathan D. Simmons as Elgin
• Kaelen Ohm as Marielle
• Angela Moore as Bakta
• A.J. Simmons as Randall
• Deborah Grover as Tillie

The biggest season 3 additions include Robert Joy as Henry, a curmudgeon to whom the years have not been kind, and Samantha Brown as Acosta, a new-to-the-force police officer who is in over her head.

Marketing is one of the most important aspects of a tech product’s success. Samsung Mobile knows this and typically employs an aggressive advertising strategy, particularly targeting Apple, its primary rival. The South Korean giant takes advantage of every opportunity to make fun of new launches and devices from the brand. Now, Samsung has shared an ad in the form of a “documentary” with former Apple Geniuses who made the switch.

There are companies whose marketing toward their direct rival uses a conciliatory and even friendly tone, such as Google Pixel’s “Best Phones Forever” ads. However, Samsung employs a radically different approach, which has not always yielded the best results. Many people mocked the company for adopting decisions made by Apple, which they had previously criticized in previous ads, for instance.

“Former Apple Geniuses” tell why they switched to Samsung in this documentary

Anyway, Samsung’s new “documentary” features “former Apple Geniuses” who share firsthand accounts of their experiences and the reasons behind their switch to Galaxy devices. The three-minute spot has three chapters, starting with “Becoming a Genius.” During this section, former Apple Geniuses tell why they joined the Apple ecosystem. Some say they joined to help people, while others did so to feel part of something “bigger than them,” among other motivations.

However, after a while, Apple Geniuses became aware of the competition’s products and started to question their own position. This is where Chapter II, “Questioning Our Own Genius,” begins, as the now doubtful Apple fans discover that Samsung devices offer advanced features unavailable on iPhones. They also questioned Apple’s decision-making process.

In Chapter III: “Sweet Taste of Freedom,” the former Apple fans finally make the switch to Galaxy devices. Here, former Apple Geniuses describe how parting ways with Apple brought a breath of freedom into their lives. Plus, they now enjoy the AI-powered features of the Galaxy AI suite on a daily basis. They mention some favorites like the sketch-to-art generator, Live Translate, and AI filters. The Night Mode of the Galaxy Z Flip 6’s camera impresses another ex-Apple fan.

A short and funny ad that you can watch right now

Despite the ad’s “serious” documentary-style filming, its humor is evident. Samsung even makes this clear with a line at the end of the video. “Their former employers didn’t comment on geniuses’ claims, probably because we didn’t reach out,” the ad shows. You can watch it below.

When these real Former 🍎 Geniuses started to think outside the white box, they discovered a Galaxy full of possibilities. 🌌 Watch to see their story. pic.twitter.com/ubvMGT27ox

— Samsung Mobile US (@SamsungMobileUS) November 1, 2024