Connect with us

Technology

Runway Act-One: AI motion capture with your smartphone camera

Published

on

Runway Act-One: AI motion capture with your smartphone camera

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


AI video has come incredibly far in the years since the first models debuted in late 2022, increasing in realism, resolution, fidelity, prompt adherence (how well they match the text prompt or description of the video that the user typed) and number.

But one area that remains a limitation to many AI video creators — myself included — is in depicting realistic facial expressions in AI generated characters. Most appear quite limited and difficult to control.

But no longer: today, Runway, the New York City-headquartered AI startup backed by Google and others, announced a new feature “Act-One,” that allows users to record video of themselves or actors from any video camera — even the one on a smartphone — and then transfers the subject’s facial expressions to that of an AI generated character with uncanny accuracy.

Advertisement

The free-to-use tool is gradually rolling out “gradually” to users starting today, according to Runway’s blog post on the feature.

While anyone with a Runway account can access it, it will be limited to those who have enough credits to generate new videos on the company’s Gen-3 Alpha video generation model introduced earlier this year, which supports text-to-video, image-to-video, and video-to-video AI creation pipelines (e.g. the user can type in a scene description, upload an image or a video, or use a combination of these inputs and Gen-3 Alpha will use what its given to guide its generation of a new scene).

Despite limited availability right now at the time of this posting, the burgeoning scene of AI video creators online is already applauding the new feature.

As Allen T. remarked on his X account “This is a game changer!”

It also comes on the heels of Runway’s move into Hollywood film production last month, when it announced it had inked a deal with Lionsgate, the studio behind the John Wick and Hunger Games movie franchises, to create a custom AI video generation model based on the studio’s catalog of more than 20,000 titles.

Advertisement

Simplifying a traditionally complex and equipment-heavy creative proccess

Traditionally, facial animation requires extensive and often cumbersome processes, including motion capture equipment, manual face rigging, and multiple reference footages.

Anyone interested in filmmaking has likely caught sight of some of the intricacy and difficulty of this process to date on set or when viewing behind the scenes footage of effects-heavy and motion-capture films such as The Lord of the Rings series, Avatar, or Rise of the Planet of the Apes, wherein actors are seen covered in ping pong ball markers and their faces dotted with marker and blocked by head-mounted apparatuses.

Accurately modeling intricate facial expressions is what led David Fincher and his production team on The Curious Case of Benjamin Button to develop whole new 3D modeling processes and ultimately won them an Academy Award, as reported in a prior VentureBeat report.

Advertisement

Yet in the last few years, new software and AI-based startups such as Move have sought to reduce the equipment necessary to perform accurate motion capture — though that company in particular has concentrated primarily on full-body, more broad movements, whereas Runway’s Act-One is focused more on modeling facial expressions.

With Act-One, Runway aims to make this complex process far more accessible. The new tool allows creators to animate characters in a variety of styles and designs, without the need for motion-capture gear or character rigging.

Instead, users can rely on a simple driving video to transpose performances—including eye-lines, micro-expressions, and nuanced pacing—onto a generated character, or even multiple characters in different styles.

As Runway wrote on its X account: “Act-One is able to translate the performance from a single input video across countless different character designs and in many different styles.”

Advertisement

The feature is focused “mostly” on the face “for now,” according to Cristóbal Valenzuela, co-founder and CEO of Runway, who responded to VentureBeat’s questions via direct message on X.

Runway’s approach offers significant advantages for animators, game developers, and filmmakers alike. The model accurately captures the depth of an actor’s performance while remaining versatile across different character designs and proportions. This opens up exciting possibilities for creating unique characters that express genuine emotion and personality.

Cinematic realism across camera angles

One of Act-One’s key strengths lies in its ability to deliver cinematic-quality, realistic outputs from various camera angles and focal lengths.

This flexibility enhances creators’ ability to tell emotionally resonant stories through character performances that were previously hard to achieve without expensive equipment and multi-step workflows.

Advertisement

The tool’s ability to faithfully capture the emotional depth and performance style of an actor, even in complex scenes.

This shift allows creators to bring their characters to life in new ways, unlocking the potential for richer storytelling across both live-action and animated formats.

While Runway previously supported video-to-video AI conversion as previously mentioned in this piece, which did allow users to upload footage of themselves and have Gen-3 Alpha or other prior Runway AI video models such as Gen-2 “reskin” them with AI effects, the new Act-One feature is optimized for facial mapping and effects.

As Valenzuela told VentureBeat via DM on X: “The consistency and performance is unmatched with Act-One.”

Advertisement

Enabling more expansive video storytelling

A single actor, using only a consumer-grade camera, can now perform multiple characters, with the model generating distinct outputs for each.

This capability is poised to transform narrative content creation, particularly in indie film production and digital media, where high-end production resources are often limited.

In a public post on X, Valenzuela noted a shift in how the industry approaches generative models. “We are now beyond the threshold of asking ourselves if generative models can generate consistent videos. A good model is now the new baseline. The difference lies in what you do with the model—how you think about its applications and use cases, and what you ultimately build,” Valenzuela wrote.

Safety and protection for public figure impersonations

As with all of Runway’s releases, Act-One comes equipped with a comprehensive suite of safety measures.

Advertisement

These include safeguards to detect and block attempts to generate content featuring public figures without authorization, as well as technical tools to verify voice usage rights.

Continuous monitoring also ensures that the platform is used responsibly, preventing potential misuse of the tool.

Runway’s commitment to ethical development aligns with its broader mission to expand creative possibilities while maintaining a strong focus on safety and content moderation.

Looking ahead

As Act-One gradually rolls out, Runway is eager to see how artists, filmmakers, and other creators will harness this new tool to bring their ideas to life.

Advertisement

With Act -ne, complex animation techniques are now within reach for a broader audience of creators, enabling more people to explore new forms of storytelling and artistic expression.

By reducing the technical barriers traditionally associated with character animation, the company hopes to inspire new levels of creativity across the digital media landscape.

It also helps Runway stand out and differentiate its AI video creation platform against the likes of an increasing swath of competitors, including Luma AI from the U.S. and Hailuo and Kling from China, as well as open source rivals such as Genmo’s Mochi 1, which also just debuted today.


Source link
Continue Reading
Advertisement
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Technology

NYT Strands today — hints, answers and spangram for Wednesday, October 23 (game #234)

Published

on

NYT Strands homescreen on a mobile phone screen, on a light blue background

Strands is the NYT’s latest word game after the likes of Wordle, Spelling Bee and Connections – and it’s great fun. It can be difficult, though, so read on for my Strands hints.

Want more word-based fun? Then check out my Wordle today, NYT Connections today and Quordle today pages for hints and answers for those games.

Source link

Continue Reading

Technology

India’s first integrated digital clinic sehatUP aims to deliver personalised integrated treatment plans- The Week

Published

on

India's first integrated digital clinic sehatUP aims to deliver personalised integrated treatment plans- The Week

Health-tech startup sehatUP has launched India’s first integrated digital health clinic. The platform claims to combine allopathy, ayurveda and homoeopathy to deliver personalized health and wellness solutions, marking a significant step towards making healthcare more comprehensive and accessible.

The startup collaborates with over 50 expert doctors and health coaches to create tailored wellness journeys. The company states that it uses advanced technology to develop treatment plans that seamlessly integrate various medical systems.

In recent years, the ‘One Nation, One Health System’, as a concept, has gained momentum in India. This is a concept that aims to establish a unified healthcare system that ensures consistent standards and promotes social equity by providing all citizens with access to quality services. It also focuses on improving efficiency by streamlining resources to reduce costs and enhance service delivery, while strengthening disaster preparedness to create a system capable of responding effectively during health emergencies. 

However, many experts have noted that India’s geographical diversity and infrastructure gaps present challenges in establishing an integrated healthcare system. Encouraging private players to participate in the national health framework and effectively utilizing digital technology have been suggested as key strategies for addressing these challenges.

Advertisement

According to sehatUP’s leadership, their platform was created in alignment with the ‘One Nation, One Health System’ initiative. sehatUP’s platform provides personalized healthcare plans and free doctor consultations, initially focusing on sexual wellness and weight management. The platform also offers a range of affordable,  clinically-approved medicines and natural supplements.

Speaking about the launch, Karan Bhargava, Founder and CEO of sehatUP, said, “sehatUP is designed to offer a more holistic and inclusive approach to healthcare. By  blending modern medicine with traditional practices like Ayurveda and homeopathy, we aim to provide users with a balanced path to well-being that addresses root causes and promotes long-term health.”

Source link

Continue Reading

Science & Environment

COP16: What is biodiversity and how are we protecting it?

Published

on

COP16: What is biodiversity and how are we protecting it?


From 21 October until 1 November, delegates are meeting in Cali, Colombia to take stock of national pledges to protect nature, amid concerns countries are back-sliding on their promises.

Recent analysis suggests, external most countries are set to miss the deadline to submit new national action plans for preserving nature.

Key issues include the scale of ambition in meeting specific targets, finance for biodiversity projects in poorer countries and making sure profits from genetic resources are shared fairly.

Colombian environment minister, Susana Muhamad, who is overseeing the meeting, has set the theme, ‘Peace with Nature’, a call to rethink our relationship with the natural world.

Advertisement

Several presidents are expected to attend, including Brazil’s Luiz Inacio Lula da Silva and Mexico’s incoming president, Claudia Sheinbaum.



Source link

Continue Reading

Technology

The 10 most popular movies on Netflix right now

Published

on

The 10 most popular movies on Netflix right now

Netflix is one of the most popular streaming services in the world, with nearly 280 million subscribers. And just what do those people tend to watch? In particular, what is the most popular movie on Netflix? Each week, the streaming service releases a list of its 10 most-watched movies over a recent seven-day period to keep subscribers in the loop regarding its most popular titles.

Anna Kendrick is a talented actress, but her directorial debut, Woman of the Hour, proves she can do it all. The true crime movie debuted at No. 1, followed by Unhinged, a 2020 road rage movie starring Russell Crowe, and Bad Boys: Ride or Diethe action film with Will Smith and Martin Lawrence. Below, we’ve listed the top 10 movies in the U.S. from October 14 to October 20, along with general information about each film, such as genre, rating, cast, and synopsis.

Looking for something else? We’ve also rounded up the best new movies to stream this week, the best movies on Netflix, the best movies on Amazon Prime, and the best movies on Disney+. For Netflix fans, check out the 10 most popular shows on Netflix right now.






Source link

Continue Reading

Technology

Samsung Electronics is in crisis, say employees

Published

on

Featured image for Samsung Electronics is in crisis, say employees

News continues to pour in about the delicate situation at Samsung Foundry, the company’s semiconductor production division. Things seemed to be improving following the good performance of the Exynos 2400 chip. However, recent developments revealed severe hurdles to overcome in order to get back on track. Interviews with Samsung employees indicate that the company is currently facing a “crisis.”

Samsung Foundry and its low yield rate issues

Samsung Foundry’s main problems stem from the low yield rate of its 3nm GAA wafers. The company had hoped to make the Eynos 2500 chip an attractive, powerful, and efficient SoC thanks to the new process. However, its factories have not been up to the task. Recent reports claim that only about 25–30 chips out of every 100 produced are usable. Meanwhile, at their worst, wafers have shown a yield rate of just 10%.

Samsung is reportedly trying to improve the yield rate of its 3nm GAA wafers before deciding which chip to use in the Galaxy S25 series. Given the severity of the situation, it is highly likely that Samsung will have to replace its Exynos 2500 chip with the more expensive (yet powerful) Snapdragon 8 Elite. However, the South Korean giant is keeping the faith and is still testing the Exynos 2500 in case they manage to turn things around at the last minute.

Samsung employees and former employees see a crisis in the company

Damnangyi, a YouTuber and former Samsung employee, shared a video with views from people related to the company. The opinions are about the current situation and the potential causes. First, seven former employees and 24 current employees claim that Samsung is in crisis. Most of them believe that Samsung’s current pace of innovation is slow. This would be a direct consequence of a lack of vision on the part of the board of directors. Some have even expressed concerns about the management’s treatment of employees.

Advertisement

Damnangyi currently works at Qualcomm in the US. However, his time at Samsung was enough to know what was going on internally. Plus, the situation is no secret, as representatives apologized to investors this month for the disappointing financial results and lack of innovation.

The company will seek to change the situation with more innovation

In its official apology, Samsung Electronics teased its strategy to try to turn things around. The company mentioned that it will bet more on innovation and on the development of long-term solutions. Only time will tell if Samsung manages to get back on track or if the internal crisis worsens. Recently, Samsung Mobile launched the Galaxy Z Fold Special Edition in an attempt to compete with the ultra-thin foldables of Chinese brands.

Source link

Continue Reading

Technology

Huawei appears to still be using TSMC chips despite US sanctions

Published

on

Huawei appears to still be using TSMC chips despite US sanctions

A Canadian research firm called TechInsights took a deep dive on one of Huawei’s artificial intelligence accelerators and found a chip manufactured by Taiwan Semiconductor Manufacturing Company (TSMC). Bloomberg spoke with several people familiar with the investigation who asked to remain anonymous since TechInsights’ report has been released to the public.

The anonymous sources says TechInsights’ investigation found an Ascend 910B chip made by TSMC in one of Huawei’s AI accelerators. The company that conducted the investigation declined to comment.

Reuters reports that TechInsights informed TSMC of its findings before publishing its report. This prompted TSMC to notify the US Commerce Department earlier this month. The Financial Times reports, citing people familiar with the matter, that TSMC told the department that a customer had placed orders for a chip similar to Huawei’s Ascend 910B. One of the FT‘s sources said that TSMC “had spoken both to the customer involved and to the commerce department” after recieving the suspicious order.

The US Commerce Department implemented additional trade restrictions against Huawei that barred the electronics company from obtaining chips made by foreign firms. Earlier this year, the US government tightened its restrictions even further by revoking its licenses with Intel and Qualcomm to produce chips for its devices.

Advertisement

TSMC denied that it had a working relationship with Huawei since mid-September of 2020 in a statement provided to the Commerce Department. TSMC also told Bloomberg that it hasn’t produced any chips for Huawei due to the amended restrictions. Huawei denied that it had ever “launched the 910B chip.”

This isn’t the first time Huawei has been caught trying to subvert US sanctions and trade restrictions. Bloomberg also uncovered in May that Huawei funded secret research in the US at universities including Harvard by funneling the money through a Washington-based scientific research foundation called Optica. The foundation said it decided to return the money in June and chief executive officers Elizabeth Rogen and Chad Stark stepped down the following August.

Update, October 22, 7:10PM ET: This story was updated after publish with more details from the Financial Times‘ reporting on the matter.

Source link

Advertisement

Continue Reading

Trending

Copyright © 2024 WordupNews.com