Connect with us

Technology

SambaNova and Gradio are making high-speed AI accessible to everyone—here’s how it works

Published

on

SambaNova and Gradio are making high-speed AI accessible to everyone—here’s how it works

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


SambaNova Systems and Gradio have unveiled a new integration that allows developers to access one of the fastest AI inference platforms with just a few lines of code. This partnership aims to make high-performance AI models more accessible and speed up the adoption of artificial intelligence among developers and businesses.

“This integration makes it easy for developers to copy code from the SambaNova playground and get a Gradio web app running in minutes with just a few lines of code,” Ahsen Khaliq, ML Growth Lead at Gradio, said in an interview with VentureBeat. “Powered by SambaNova Cloud for super-fast inference, this means a great user experience for developers and end-users alike.”

The SambaNova-Gradio integration enables users to create web applications powered by SambaNova’s high-speed AI models using Gradio’s gr.load() function. Developers can now quickly generate a chat interface connected to SambaNova’s models, making it easier to work with advanced AI systems.

Advertisement
A snippet of Python code demonstrates the simplicity of integrating SambaNova’s AI models with Gradio’s user interface. Just a few lines are needed to launch a powerful language model, underscoring the partnership’s goal of making advanced AI more accessible to developers. (Credit: SambaNova Systems)

Beyond GPUs: The rise of dataflow architecture in AI processing

SambaNova, a Silicon Valley startup backed by SoftBank and BlackRock, has been making waves in the AI hardware space with its dataflow architecture chips. These chips are designed to outperform traditional GPUs for AI workloads, with the company claiming to offer the “world’s fastest AI inference service.”

SambaNova’s platform can run Meta’s Llama 3.1 405B model at 132 tokens per second at full precision, a speed that is particularly crucial for enterprises looking to deploy AI at scale.

This development comes as the AI infrastructure market heats up, with startups like SambaNova, Groq, and Cerebras challenging Nvidia’s dominance in AI chips. These new entrants are focusing on inference — the production stage of AI where models generate outputs based on their training — which is expected to become a larger market than model training.

SambaNova’s AI chips show 3-5 times better energy efficiency than Nvidia’s H100 GPU when running large language models, according to the company’s data. (Credit: SambaNova Systems)

From code to cloud: The simplification of AI application development

For developers, the SambaNova-Gradio integration offers a frictionless entry point to experiment with high-performance AI. Users can access SambaNova’s free tier to wrap any supported model into a web app and host it themselves within minutes. This ease of use mirrors recent industry trends aimed at simplifying AI application development.

The integration currently supports Meta’s Llama 3.1 family of models, including the massive 405B parameter version. SambaNova claims to be the only provider running this model at full 16-bit precision at high speeds, a level of fidelity that could be particularly attractive for applications requiring high accuracy, such as in healthcare or financial services.

The hidden costs of AI: Navigating speed, scale, and sustainability

While the integration makes high-performance AI more accessible, questions remain about the long-term effects of the ongoing AI chip competition. As companies race to offer faster processing speeds, concerns about energy use, scalability, and environmental impact grow.

Advertisement

The focus on raw performance metrics like tokens per second, while important, may overshadow other crucial factors in AI deployment. As enterprises integrate AI into their operations, they will need to balance speed with sustainability, considering the total cost of ownership, including energy consumption and cooling requirements.

Additionally, the software ecosystem supporting these new AI chips will significantly influence their adoption. Although SambaNova and others offer powerful hardware, Nvidia’s CUDA ecosystem maintains an edge with its wide range of optimized libraries and tools that many AI developers already know well.

As the AI infrastructure market continues to evolve, collaborations like the SambaNova-Gradio integration may become increasingly common. These partnerships have the potential to foster innovation and competition in a field that promises to transform industries across the board. However, the true test will be in how these technologies translate into real-world applications and whether they can deliver on the promise of more accessible, efficient, and powerful AI for all.


Source link
Continue Reading
Advertisement
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Technology

ChatGPT has a Windows app now

Published

on

ChatGPT has a Windows app now

Just like the Mac version of the app, ChatGPT on Windows lets you ask the AI-powered chatbot questions in a dedicated window that you can keep open alongside your apps. You can quickly access the app by using the Alt + Space shortcut.

It also lets you upload files and photos to ChatGPT and comes with access to a preview of OpenAI’s o1 model capable of “reasoning.” The app is still missing some capabilities, however, such as advanced voice mode.

Even though only ChatGPT Plus, Enterprise, Team, and Edu subscribers can use the app on Windows, OpenAI says it plans on bringing it to everyone later this year.

Source link

Continue Reading

Technology

Meta’s AI chief is right to call AI fearmongering ‘BS’ but not for the reason he thinks

Published

on

Yann LeCun

AI is the latest technology monster scaring people about the future. Legitimate concerns around things like ethical training, environmental impact, and scams using AI morph into nightmares of Skynet and the Matrix all too easily. The prospect of AI becoming sentient and overthrowing humanity is frequently raised, but, as Meta’s AI chief Yann LeCun told The Wall Street Journal, the idea is “complete B.S.” LeCun described AI as less intelligent than a cat and incapable of plotting or even desiring anything at all, let alone the downfall of our species.

LeCun is right that AI is not going to scheme its way into murdering humanity, but that doesn’t mean there’s nothing to be worried about. I’m much more worried about people relying on AI to be smarter than it is. AI is just another technology, meaning it’s not good or evil. But the law of unintended consequences suggests relying on AI for important, life-altering decisions isn’t a good idea.

Source link

Continue Reading

Technology

NYT Crossword: answers for Thursday, October 17

Published

on

NYT Crossword: answers for Monday, September 23


The New York Times crossword puzzle can be tough! If you’re stuck, we’re here to help with a list of today’s clues and answers.

Source link

Continue Reading

Technology

FCC mandates all phones will have to be hearing aid-compatible

Published

on

FCC mandates all phones will have to be hearing aid-compatible

The FCC has approved regulations that mandate phone manufacturers to ensure their products work with hearing aids. Making phones compatible with hearing aids could help the aging American population.

All phones in the US to be compatible with hearing aids

The Apple AirPods Pro 2 has a hearing aid feature. With this feature activated, these premium True Wireless Stereo earbuds can transform into over-the-counter hearing aids. Needless to say, this approval must have shaken up the hearing aid market.

The US FCC has now reportedly approved new regulations that require all phone makers to make their handsets compatible with hearing aids. While announcing the new rules for phone manufacturers, the FCC stated, “Under the new rules, after a transition period, Americans with hearing loss will no longer be limited in their choice of technologies, features, and prices available in the mobile handset marketplace.”

The FCC has appreciated cell carriers, phone makers, and researchers for coming up with the final draft of the new rules. The agency categorically noted, “Establishing a 100% hearing aid compatibility requirement for all mobile handsets was made possible by the collaborative efforts of members of the Hearing Aid Compatibility Task Force — an independent organization of wireless service providers, handset manufacturers, research institutions, and advocates for those with hearing loss.”

Advertisement

Appreciating the industries involved seems necessary because these tech and hardware companies would have to collaborate closely for this to work. Ensuring every cellphone can work reliably with any hearing aid would need wireless communication protocols to be perfectly cross-compatible.

Hearing aids work quite differently compared to regular headphones. Although the basic premise is the same, hearing aids have to correctly amplify different aspects or frequencies of incoming sound. This needs careful calibration of multiple hardware and software components. It’s no wonder that only a few companies such as Apple and Samsung were able to lead this nascent but growing market.

Phone makers must ensure distortion-free sound at high volume

Incidentally, the onus of ensuring phones work with hearing aids doesn’t rest solely on the device manufacturers. Even hearing aid makers will have to make some fundamental changes.

Specifically speaking, the FCC has practically banned proprietary Bluetooth coupling standards in assistive devices. This rule extends to OTC hearing aids such as Apple’s AirPods Pro 2, and other products that would launch in the future.

Advertisement

The FCC is also forcing phone manufacturers to embed better-quality audio enhancement chips in their products. Moving forward, all new mobile handsets sold in the US will have to allow users to raise the volume without introducing distortion. In other words, handset makers will have to ensure their devices deliver clear and crisp sound, even at high volumes.

Cellphones sold in the US would have to carry labels that clearly state hearing aid compatibility. The packaging must also mention whether the handsets meet Bluetooth or telecoil coupling requirements.

Source link

Continue Reading

Technology

Sam Altman’s Worldcoin startup is dropping the coin and doubling down on Orbs

Published

on

Sam Altman’s Worldcoin startup is dropping the coin and doubling down on Orbs

Sam Altman’s Worldcoin is going to need some new business cards printed up because it’s dropping the “coin” in its name. The OpenAI CEO’s startup is shifting from cryptocurrency to focus more on its identification technology and it just unveiled a new version of its signature gadget.

reported that the new company called (wait for it) World will focus its eye scanning tech on confirming identities, something that could come in handy in a world of deep fake videos popping up all over the internet.

Co-founder and CEO Alex Blania introduced the World’s newest device called Orb, a biometric eye scanner used to confirm human identities through an identity service called Deep Face.

The latest model of the Orb, which uses NVIDIA’s Jetson chipset, will roll out to customers as the need arises. Chief Device Officer Rich Heley said at the San Francisco event that access to the Orb will be on demand and delivered the same way that people order pizza. A company statement says, “These advancements make it possible to offer new ways of providing World ID’s proof of human verification in more places around the world.”

Advertisement

According to , almost 7 million people have been scanned by World Orbs to date. Everyone in attendance at the San Francisco launch event received a free Orb for their human identifying needs.

Source link

Continue Reading

Technology

Small but mighty: H2O.ai’s new AI models challenge tech giants in document analysis

Published

on

Small but mighty: H2O.ai's new AI models challenge tech giants in document analysis

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


H2O.ai, a provider of open-source AI platforms, announced today two new vision-language models designed to improve document analysis and optical character recognition (OCR) tasks.

The models, named H2OVL Mississippi-2B and H2OVL-Mississippi-0.8B, show competitive performance against much larger models from major tech companies, potentially offering a more efficient solution for businesses dealing with document-heavy workflows.

David vs. Goliath: How H2O.ai’s tiny models are outsmarting tech giants

The H2OVL Mississippi-0.8B model, with only 800 million parameters, surpassed all other models, including those with billions more parameters, on the OCRBench Text Recognition task. Meanwhile, the 2-billion parameter H2OVL Mississippi-2B model demonstrated strong general performance across a range of vision-language benchmarks.

Advertisement

“We’ve designed H2OVL Mississippi models to be a high-performance yet cost-effective solution, bringing AI-powered OCR, visual understanding, and Document AI to businesses,” Sri Ambati, CEO and Founder of H2O.ai said in an exclusive interview with VentureBeat. “By combining advanced multimodal AI with efficiency, H2OVL Mississippi delivers precise, scalable Document AI solutions across a range of industries.”

The release of these models marks a significant step in H2O.ai’s strategy to make AI technology more accessible. By making the models freely available on Hugging Face, a popular platform for sharing machine learning models, H2O.ai is allowing developers and businesses to modify and adapt the models for specific document AI needs.

H2O.ai’s new H2OVL Mississippi-0.8B model (far right, in yellow) outperforms larger models from tech giants in text recognition tasks on the OCRBench dataset, demonstrating the potential of smaller, more efficient AI models for document analysis. (Credit: H2O.ai)

Efficiency meets effectiveness: A new approach to document processing

Ambati highlighted the economic advantages of smaller, specialized models. “Our approach to generative pre-trained transformers stems from our deep investment in Document AI, where we collaborate with customers to extract meaning from enterprise documents,” he said. “These models can run anywhere, on a small footprint, efficiently and sustainably, allowing fine-tuning on domain-specific images and documents at a fraction of the cost.”

The announcement comes as businesses seek more efficient ways to process and extract information from large volumes of documents. Traditional OCR and document analysis methods often struggle with poor-quality scans, challenging handwriting, or heavily modified documents. H2O.ai’s new models aim to address these issues while offering a more resource-efficient alternative to larger language models that may be excessive for specific document-related tasks.

Industry analysts note that H2O.ai’s approach could disrupt the current landscape dominated by tech giants. By focusing on smaller, more specialized models, H2O.ai may be able to capture a significant portion of the enterprise market that values efficiency and cost-effectiveness.

Advertisement
A comparison of average scores on eight single image benchmarks shows H2O.ai’s new H2OVL Mississippi-2B model (in yellow) outperforming several competitors, including offerings from Microsoft and Google. The model trails only Qwen2 VL-2B in overall performance among similarly sized vision-language models. (Credit: H2O.ai)

Open source and enterprise-ready: H2O.ai’s strategy for AI adoption

“At H2O.ai, making AI accessible isn’t just an idea. It’s a movement,” Ambati told VentureBeat. “By releasing a series of small foundational models that can be easily fine-tuned to specific tasks, we are expanding the possibilities for creating and using AI.”

H2O.ai has raised $256 million from investors including Commonwealth Bank, Nvidia, Goldman Sachs, and Wells Fargo. The company’s open-source approach and focus on practical, enterprise-ready AI solutions have helped it build a community of over 20,000 organizations and more than half of the Fortune 500 companies as customers.

As businesses continue to grapple with digital transformation and the need to extract value from unstructured data, H2O.ai’s new vision-language models could provide a compelling option for those looking to implement document AI solutions without the computational overhead of larger models. The true test will be in real-world applications, but H2O.ai’s demonstration of competitive performance with much smaller models suggests a promising direction for the future of enterprise AI.


Source link
Continue Reading

Trending

Copyright © 2024 WordupNews.com