Connect with us

Technology

Going beyond GPUs: The evolving landscape of AI chips and accelerators

Published

on

Going beyond GPUs: The evolving landscape of AI chips and accelerators

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


This article is part of a VB Special Issue called “Fit for Purpose: Tailoring AI Infrastructure.” Catch all the other stories here.

Data centers are the backend of the internet we know. Whether it’s Netflix or Google, all major companies leverage data centers, and the computer systems they host, to deliver digital services to end users. As the focus of enterprises shifts toward advanced AI workloads, data centers’ traditional CPU-centric servers are being buffed with the integration of new specialized chips or “co-processors.”

At the core, the idea behind these co-processors is to introduce an add-on of sorts to enhance the computing capacity of the servers. This enables them to handle the calculational demands of workloads like AI training, inference, database acceleration and network functions. Over the last few years, GPUs, led by Nvidia, have been the go-to choice for co-processors due to their ability to process large volumes of data at unmatched speeds. Due to increased demand GPUs accounted for 74% of the co-processors powering AI use cases within data centers last year, according to a study from Futurum Group.

Advertisement

According to the study, the dominance of GPUs is only expected to grow, with revenues from the category surging 30% annually to $102 billion by 2028. But, here’s the thing: while GPUs, with their parallel processing architecture, make a strong companion for accelerating all sorts of large-scale AI workloads (like training and running massive, trillion parameter language models or genome sequencing), their total cost of ownership can be very high. For example, Nvidia’s flagship GB200 “superchip”, which combines a Grace CPU with two B200 GPUs, is expected to cost between $60,000 and $70,000. A server with 36 of these superchips is estimated to cost around $2 million.

While this may work in some cases, like large-scale projects, it is not for every company. Many enterprise IT managers are looking to incorporate new technology to support select low- to medium-intensive AI workloads with a specific focus on total cost of ownership, scalability and integration. After all, most AI models (deep learning networks, neural networks, large language models etc) are in the maturing stage and the needs are shifting towards AI inferencing and enhancing the performance for specific workloads like image recognition, recommender systems or object identification — while being efficient at the same time.  

>>Don’t miss our special issue: Fit for Purpose: Tailoring AI Infrastructure.<<

This is exactly where the emerging landscape of specialized AI processors and accelerators, being built by chipmakers, startups and cloud providers, comes in. 

Advertisement

What exactly are AI processors and accelerators?

At the core, AI processors and accelerators are chips that sit within servers’ CPU ecosystem and focus on specific AI functions. They commonly revolve around three key architectures: Application-Specific Integrated Circuited (ASICs), Field-Programmable Gate Arrays (FPGAs), and the most recent innovation of Neural Processing Units (NPUs).

The ASICs and FPGAs have been around for quite some time, with programmability being the only difference between the two. ASICs are custom-built from the ground up for a specific task (which may or may not be AI-related), while FPGAs can be reconfigured at a later stage to implement custom logic. NPUs, on their part, differentiate from both by serving as the specialized hardware that can only accelerate AI/ML workloads like neural network inference and training. 

“Accelerators tend to be capable of doing any function individually, and sometimes with wafer-scale or multi-chip ASIC design, they can be capable of handling a few different applications. NPUs are a good example of a specialized chip (usually part of a system) that can handle a number of matrix-math and neural network use cases as well as various inference tasks using less power,” Futurum group CEO Daniel Newman tells Venturebeat.

The best part is that accelerators, especially ASICs and NPUs built for specific applications, can prove more efficient than GPUs in terms of cost and power use.

Advertisement

“GPU designs mostly center on Arithmetic Logic Units (ALUs) so that they can perform thousands of calculations simultaneously, whereas AI accelerator designs mostly center on Tensor Processor Cores (TPCs) or Units. In general, the AI accelerators’ performance versus GPUs performance is based on the fixed function of that design,” Rohit Badlaney, the general manager for IBM’s cloud and industry platforms, tells VentureBeat. 

Currently, IBM follows a hybrid cloud approach and uses multiple GPUs and AI accelerators, including offerings from Nvidia and Intel, across its stack to provide enterprises with choices to meet the needs of their unique workloads and applications — with high performance and efficiency.

“Our full-stack solutions are designed to help transform how enterprises, developers and the open-source community build and leverage generative AI. AI accelerators are one of the offerings that we see as very beneficial to clients looking to deploy generative AI,” Badlaney said. He added while GPU systems are best suited for large model training and fine-tuning, there are many AI tasks that accelerators can handle equally well – and at a lesser cost.

For instance, IBM Cloud virtual servers use Intel’s Gaudi 3 accelerator with a custom software stack designed specifically for inferencing and heavy memory demands. The company also plans to use the accelerator for fine-tuning and small training workloads via small clusters of multiple systems.

Advertisement

“AI accelerators and GPUs can be used effectively for some similar workloads, such as LLMs and diffusion models (image generation like Stable Diffusion) to standard object recognition, classification, and voice dubbing. However, the benefits and differences between AI accelerators and GPUs entirely depend on the hardware provider’s design. For instance, the Gaudi 3 AI accelerator was designed to provide significant boosts in compute, memory bandwidth, and architecture-based power efficiency,” Badlaney explained. 

This, he said, directly translates to price-performance benefits. 

Beyond Intel, other AI accelerators are also drawing attention in the market. This includes not only custom chips built for and by public cloud providers such as Google, AWS and Microsoft but also dedicated products (NPUs in some cases) from startups such as Groq, Graphcore, SambaNova Systems and Cerebras Systems. They all stand out in their own way, challenging GPUs in different areas.

In one case, Tractable, a company developing AI to analyze damage to property and vehicles for insurance claims, was able to leverage Graphcore’s Intelligent Processing Unit-POD system (a specialized NPU offering) for significant performance gains compared to GPUs they had been using.

Advertisement

“We saw a roughly 5X speed gain,” Razvan Ranca, co-founder and CTO at Tractable, wrote in a blog post. “That means a researcher can now run potentially five times more experiments, which means we accelerate the whole research and development process and ultimately end up with better models in our products.”

AI processors are also powering training workloads in some cases. For instance, the AI supercomputer at Aleph Alpha’s data center is using Cerebras CS-3, the system powered by the startup’s third-generation Wafer Scale Engine with 900,000 AI cores, to build next-gen sovereign AI models. Even Google’s recently introduced custom ASIC, TPU v5p, is driving some AI training workloads for companies like Salesforce and Lightricks.

What should be the approach to picking accelerators?

Now that it’s established there are many AI processors beyond GPUs to accelerate AI workloads, especially inference, the question is: how does an IT manager pick the best option to invest in? Some of these chips may deliver good performance with efficiencies but might be limited in terms of the kind of AI tasks they could handle due to their architecture. Others may do more but the TCO difference might not be as massive when compared to GPUs. 

Since the answer varies with the design of the chips, all experts VentureBeat spoke to suggested the selection should be based upon the scale and type of the workload to be processed, the data, the likelihood of continued iteration/change and cost and availability needs. 

Advertisement

According to Daniel Kearney, the CTO at Sustainable Metal Cloud, which helps companies with AI training and inference, it is also important for enterprises to run benchmarks to test for price-performance benefits and ensure that their teams are familiar with the broader software ecosystem that supports the respective AI accelerators.

“While detailed workload information may not be readily in advance or may be inconclusive to support decision-making, it is recommended to benchmark and test through with representative workloads, real-world testing and available peer-reviewed real-world information where available to provide a data-driven approach to choosing the right AI accelerator for the right workload. This upfront investigation can save significant time and money, particularly for large and costly training jobs,” he suggested.

Globally, with inference jobs on track to grow, the total market of AI hardware, including AI chips, accelerators and GPUs, is estimated to grow 30% annually to touch $138 billion by 2028.


Source link
Continue Reading
Advertisement
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Servers computers

#RAckOm_System | Best Networking Server Rack | ALL OUT DOOR IP RACK MANUFACTURER IN INDIA |

Published

on

#RAckOm_System | Best Networking Server Rack | ALL OUT DOOR IP RACK MANUFACTURER IN INDIA |



#RAckOm_System

We are India’s leading “Manufacturer” & supplier of IP55/IP65 outdoor and Indoor cabinets used in IT/Telecom, surveillance, smart city ,wifi application. Our Rack Certifications (An ISO QMS 9001: 2001 IP55-IP66 Certified Products Brand ) (RoHS IEC 62321 : 2013 Compliance Product Co.). We have concluded many smart cities projects in Karnal (IEC 60529 : 2001), (Harayana) ,Delhi (NDMC-MTNL ),Panchkula etc. We have also supplied our Racks to defence, Ministries, Telecom Cos ,Metro Projects, Banks, Power generation Cos, Police and to many system Integrators.

We are also registered in GEM under OEM supplier and supplied Racks all over India with 4.5 star Rating.

Our product range includes
• Server Rack & Networking Racks (4U to 50 U)
• IP55 Outdoor cabinets for Smart city & Safe city surveillance projects
• #IP55_cabinets for Telecom projects
• #BTS_cabinets
• #Indoor_&_Outdoor_Fiber_Termination_Cabinet
• #Pole_Mount_IP55_Junction_Box
• #Outdoor_IP55_Junction_Box
• #DVR_NVR_Cabinets
• #Cable_TV_Headend_Racks
• #Broadcast_Equipments_Rack (Broadcasting Industries)
• #CISCO_CCNA_Lab_Racks
• #IP55_Outdoor_Battery_Banks
• #Power_Distribution_Units (PDUs)
• FMS-LIU Fiber Patch Panel ( 6 core to 144 core)

source

Continue Reading

Technology

Get peace of mind with the Blink Outdoor 4 camera for $40

Published

on

Get peace of mind with the Blink Outdoor 4 camera for $40

If you own a home and want a solution to keeping track of the outside of your home, then check out this deal on the Blink Outdoor 4 camera that Amazon is currently offering. There are two parts to this deal that are worth mentioning. Or rather, they’re two different deal options for the same product.

The first deal option is that you can now pick up the Blink Outdoor 4 camera for just $40. This is the lowest this camera has ever been and it typically sits around $87.35 on Amazon. So this is more than half of the regular price and slightly more than half of the typical sale price. That’s quite good. And if you don’t already have a security camera setup for your home, this is definitely worth considering.

The other deal option that Amazon is currently offering, is that you can get the Blink Outdoor 4 camera for free if you purchase a 1-year subscription to the Blink Subscription Plus plan. That plan is $100, but it covers all of your other Blink devices. So this deal option is tailored to people who either already have other Blink devices set up, or plan to buy others in addition to this Blink Outdoor 4.

The Blink Outdoor 4 is a wire-free camera so you don’t have to worry about a complicated installation. The setup is a pretty quick and painless process. It has a 2-year battery life too. Additionally, it has two-way audio so you can talk to people on the other end, enhanced motion control, and it even works with Alexa. You can check the camera’s footage from your phone, where you can also manage its controls and features. But you can also use your voice for that so things stay hands-free.

Advertisement

Blink Outdoor 4 Amazon Price History

Buy at Amazon

Source link

Continue Reading

Technology

Google launches Gemini’s contextual smart replies in Gmail

Published

on

Google launches Gemini's contextual smart replies in Gmail

When Google rolled out Gemini side panels for Gmail and its other Workspace apps, it revealed that its generative AI chatbot will also be able to offer contextual smart replies for its email service in the future. Now, the company has officially released that feature. Smart replies have existed in Gmail since 2017, giving you a quick, albeit impersonal, way to respond to messages, even if you’re in a hurry or on the go. These machine-generated responses are pretty limited, though, and they’re often just one liners to tell the recipient that you understand what they’re saying or that you agree with whatever they’re suggesting.

The new Gemini-generated smart replies take the full content of the email thread into consideration. While you may still have to edit them a bit if you want them to be as close to something you’d write as possible, they are more detailed and more personable. When you get the feature, you’ll see several response options at the bottom of your screen when you reply through the Gmail app. Just hover over each of them to get a detailed preview before choosing one that you think makes for the best response.

You’ll get access to the feature if you have a Gemini Business, Enterprise, Education or Education Premium add-on, or if you have a Google One AI Premium subscription. Google says it could take up to 15 days before you see Gemini’s smart replies in your app — just make sure you’ve ticked on “Smart features and personalization” in your Gmail app’s Settings page.

Source link

Continue Reading

Servers computers

Our Blade server Hardware Products Available #shorts

Published

on

Our Blade server  Hardware Products Available #shorts



We are best server provider company in India. We provide service in Pan India. Our available Blade server products are Hardware (Brand-HPE/DELL/LENOVO/IBM/FUJITSHU).

Contact us today at 075570 10709 for service.

Visit our website for more details:
www.netgroup.in

Connect with us:
Facebook: https://lnkd.in/g2bEaF2
Instagram: https://lnkd.in/gAawSh6y
Linkedin: https://lnkd.in/gWdfuKk
Google: https://lnkd.in/dNH22GA6
Justdial: https://lnkd.in/d59kNCts
Indiamart: https://lnkd.in/dEdHzQkX
Twitter: https://lnkd.in/gYnHNFRa

Subscribe to our channel for more update.

#netgroup #ITserver #ITservice #servers #server #serverproblems #serverlife #hosting #datacenter #networking #technology #webhosting #dell #lenovo #bladeserver #itsolutions #itcompany #internet #itworks #itissues #itconsulting #connectivity .

source

Continue Reading

Technology

AI is changing enterprise computing — and the enterprise itself

Published

on

AI is changing enterprise computing -- and the enterprise itself

Presented by AMD


This article is part of a VB Special Issue called “Fit for Purpose: Tailoring AI Infrastructure.” Catch all the other stories here.

It’s hard to think of any enterprise technology having a greater impact on business today than artificial intelligence (AI), with use cases including automating processes, customizing user experiences, and gaining insights from massive amounts of data.

As a result, there is a realization that AI has become a core differentiator that needs to be built into every organization’s strategy. Some were surprised when Google announced in 2016 that they would be a mobile-first company, recognizing that mobile devices had become the dominant user platform. Today, some companies call themselves ‘AI first,’ acknowledging that their networking and infrastructure must be engineered to support AI above all else.

Advertisement

Failing to address the challenges of supporting AI workloads has become a significant business risk, with laggards set to be left trailing AI-first competitors who are using AI to drive growth and speed towards a leadership position in the marketplace.

However, adopting AI has pros and cons. AI-based applications create a platform for businesses to drive revenue and market share, for example by enabling efficiency and productivity improvements through automation. But the transformation can be difficult to achieve. AI workloads require massive processing power and significant storage capacity, putting strain on already complex and stretched enterprise computing infrastructures.

>>Don’t miss our special issue: Fit for Purpose: Tailoring AI Infrastructure.<<

In addition to centralized data center resources, most AI deployments have multiple touchpoints across user devices including desktops, laptops, phones and tablets. AI is increasingly being used on edge and endpoint devices, enabling data to be collected and analyzed close to the source, for greater processing speed and reliability. For IT teams, a large part of the AI discussion is about infrastructure cost and location. Do they have enough processing power and data storage? Are their AI solutions located where they run best — at on-premises data centers or, increasingly, in the cloud or at the edge?

Advertisement

How enterprises can succeed at AI

If you want to become an AI-first organization, then one of the biggest challenges is building the specialized infrastructure that this requires. Few organizations have the time or money to build massive new data centers to support power-hungry AI applications.  

The reality for most businesses is that they will have to determine a way to adapt and modernize their data centers to support an AI-first mentality.

But where do you start? In the early days of cloud computing, cloud service providers (CSPs) offered simple, scalable compute and storage — CSPs were considered a simple deployment path for undifferentiated business workloads. Today, the landscape is dramatically different, with new AI-centric CSPs offering cloud solutions specifically designed for AI workloads and, increasingly, hybrid AI setups that span on-premises IT and cloud services.

AI is a complex proposition and there’s no one-size-fits-all solution. It can be difficult to know what to do. For many organizations, help comes from their strategic technology partners who understand AI and can advise them on how to create and deliver AI applications that meet their specific objectives — and will help them grow their businesses.

Advertisement

With data centers, often a significant part of an AI application, a key element of any strategic partner’s role is enabling data center modernization. One example is the rise in servers and processors specifically designed for AI. By adopting specific AI-focused data center technologies, it’s possible to deliver significantly more compute power through fewer processors, servers, and racks, enabling you to reduce the data center footprint required by your AI applications. This can increase energy efficiency and also reduce the total cost of investment (TCO) for your AI projects.

A strategic partner can also advise you on graphics processing unit (GPU) platforms. GPU efficiency is key to AI success, particularly for training AI models, real-time processing or decision-making. Simply adding GPUs won’t overcome processing bottlenecks. With a well implemented, AI-specific GPU platform, you can optimize for the specific AI projects you need to run and spend only on the resources this requires. This improves your return on investment (ROI), as well as the cost-effectiveness (and energy efficiency) of your data center resources.

Similarly, a good partner can help you identify which AI workloads truly require GPU-acceleration, and which have greater cost effectiveness when running on CPU-only infrastructure. For example, AI Inference workloads are best deployed on CPUs when model sizes are smaller or when AI is a smaller percentage of the overall server workload mix. This is an important consideration when planning an AI strategy because GPU accelerators, while often critical for training and large model deployment, can be costly to obtain and operate.

Data center networking is also critical for delivering the scale of processing that AI applications require. An experienced technology partner can give you advice about networking options at all levels (including rack, pod and campus) as well as helping you to understand the balance and trade-off between different proprietary and industry-standard technologies.

Advertisement

What to look for in your partnerships

Your strategic partner for your journey to an AI-first infrastructure must combine expertise with an advanced portfolio of AI solutions designed for the cloud and on-premises data centers, user devices, edge and endpoints.

AMD, for example, is helping organizations to leverage AI in their existing data centers. AMD EPYC(TM) processors can drive rack-level consolidation, enabling enterprises to run the same workloads on fewer servers, CPU AI performance for small and mixed AI workloads, and improved GPU performance, supporting advanced GPU accelerators and minimize computing bottlenecks.  Through consolidation with AMD EPYC™ processors data center space and power can be freed to enable deployment of AI-specialized servers.

The increase in demand for AI application support across the business is putting pressure on aging infrastructure. To deliver secure and reliable AI-first solutions, it’s important to have the right technology across your IT landscape, from data center through to user and endpoint devices.

Enterprises should lean into new data center and server technologies to enable them to speed up their adoption of AI. They can reduce the risks through innovative yet proven technology and expertise. And with more organizations embracing an AI-first mindset, the time to get started on this journey is now.

Advertisement

Learn more about AMD.

Robert Hormuth is Corporate Vice President, Architecture & Strategy — Data Center Solutions Group, AMD


Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.

Source link

Continue Reading

Servers computers

Dell server R720 Review | Dell R720 | Dell PowerEdge R720 | Tech Saqi Mirza

Published

on

Dell server R720 Review | Dell R720 | Dell PowerEdge R720 | Tech Saqi Mirza



In This video i telling about of Dell PowerEdge R720. Dell R720 is designed to running a wide range of applications and virtualization environments for normal nd professional users .
Dell R720 comes after Dell R710 model .
Dell R720 rack based server is a general purpose server with highly memory (max 768GB) and best I / O capabilities to complete its task . Dell R720 supported Intel® Xeon® E5-2600 series processors, and the capability to support dual RAID controllers,Dell R720 have ability to handle challenging tasks
like database storage , high end computing nd virtual infra structure . .

source

Continue Reading

Trending

Copyright © 2024 WordupNews.com