Connect with us

Technology

From cost center to competitive edge: The strategic value of custom AI Infrastructure

Published

on

From cost center to competitive edge: The strategic value of custom AI Infrastructure

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More


This article is part of a VB Special Issue called “Fit for Purpose: Tailoring AI Infrastructure.” Catch all the other stories here.

AI is no longer just a buzzword — it’s a business imperative. As enterprises across industries continue to adopt AI, the conversation around AI infrastructure has evolved dramatically. Once viewed as a necessary but costly investment, custom AI infrastructure is now seen as a strategic asset that can provide a critical competitive edge.

Mike Gualtieri, vice president and principal analyst at Forrester, emphasizes the strategic importance of AI infrastructure. “Enterprises must invest in an enterprise AI/ML platform from a vendor that at least keeps pace with, and ideally pushes the envelope of, enterprise AI technology,” Gualtieri said. “The technology must also serve a reimagined enterprise operating in a world of abundant intelligence.” This perspective underscores the shift from viewing AI as a peripheral experiment to recognizing it as a core component of future business strategy.

Advertisement

The infrastructure revolution

The AI revolution has been fueled by breakthroughs in AI models and applications, but those innovations have also created new challenges. Today’s AI workloads, especially around training and inference for large language models (LLMs), require unprecedented levels of computing power. This is where custom AI infrastructure comes into play.

>>Don’t miss our special issue: Fit for Purpose: Tailoring AI Infrastructure.<<

“AI infrastructure is not one-size-fits-all,” says Gualtieri. “There are three key workloads: data preparation, model training and inference.” Each of these tasks has different infrastructure requirements, and getting it wrong can be costly, according to Gualtieri. For example, while data preparation often relies on traditional computing resources, training massive AI models like GPT-4o or LLaMA 3.1 necessitates specialized chips such as Nvidia’s GPUs, Amazon’s Trainium or Google’s TPUs.

Nvidia, in particular, has taken the lead in AI infrastructure, thanks to its GPU dominance. “Nvidia’s success wasn’t planned, but it was well-earned,” Gualtieri explains. “They were in the right place at the right time, and once they saw the potential of GPUs for AI, they doubled down.” However, Gualtieri believes that competition is on the horizon, with companies like Intel and AMD looking to close the gap.

Advertisement

The cost of the cloud

Cloud computing has been a key enabler of AI, but as workloads scale, the costs associated with cloud services have become a point of concern for enterprises. According to Gualtieri, cloud services are ideal for “bursting workloads” — short-term, high-intensity tasks. However, for enterprises running AI models 24/7, the pay-as-you-go cloud model can become prohibitively expensive.

“Some enterprises are realizing they need a hybrid approach,” Gualtieri said. “They might use the cloud for certain tasks but invest in on-premises infrastructure for others. It’s about balancing flexibility and cost-efficiency.”

This sentiment was echoed by Ankur Mehrotra, general manager of Amazon SageMaker at AWS. In a recent interview, Mehrotra noted that AWS customers are increasingly looking for solutions that combine the flexibility of the cloud with the control and cost-efficiency of on-premise infrastructure. “What we’re hearing from our customers is that they want purpose-built capabilities for AI at scale,” Mehrotra explains. “Price performance is critical, and you can’t optimize for it with generic solutions.”

To meet these demands, AWS has been enhancing its SageMaker service, which offers managed AI infrastructure and integration with popular open-source tools like Kubernetes and PyTorch. “We want to give customers the best of both worlds,” says Mehrotra. “They get the flexibility and scalability of Kubernetes, but with the performance and resilience of our managed infrastructure.”

Advertisement

The role of open source

Open-source tools like PyTorch and TensorFlow have become foundational to AI development, and their role in building custom AI infrastructure cannot be overlooked. Mehrotra underscores the importance of supporting these frameworks while providing the underlying infrastructure needed to scale. “Open-source tools are table stakes,” he says. “But if you just give customers the framework without managing the infrastructure, it leads to a lot of undifferentiated heavy lifting.”

AWS’s strategy is to provide a customizable infrastructure that works seamlessly with open-source frameworks while minimizing the operational burden on customers. “We don’t want our customers spending time on managing infrastructure. We want them focused on building models,” says Mehrotra.

Gualtieri agrees, adding that while open-source frameworks are critical, they must be backed by robust infrastructure. “The open-source community has done amazing things for AI, but at the end of the day, you need hardware that can handle the scale and complexity of modern AI workloads,” he says.

The future of AI infrastructure

As enterprises continue to navigate the AI landscape, the demand for scalable, efficient and custom AI infrastructure will only grow. This is especially true as artificial general intelligence (AGI) — or agentic AI — becomes a reality. “AGI will fundamentally change the game,” Gualtieri said. “It’s not just about training models and making predictions anymore. Agentic AI will control entire processes, and that will require a lot more infrastructure.”

Advertisement

Mehrotra also sees the future of AI infrastructure evolving rapidly. “The pace of innovation in AI is staggering,” he says. “We’re seeing the emergence of industry-specific models, like BloombergGPT for financial services. As these niche models become more common, the need for custom infrastructure will grow.”

AWS, Nvidia and other major players are racing to meet this demand by offering more customizable solutions. But as Gualtieri points out, it’s not just about the technology. “It’s also about partnerships,” he says. “Enterprises can’t do this alone. They need to work closely with vendors to ensure their infrastructure is optimized for their specific needs.”

Custom AI infrastructure is no longer just a cost center — it’s a strategic investment that can provide a significant competitive edge. As enterprises scale their AI ambitions, they must carefully consider their infrastructure choices to ensure they are not only meeting today’s demands but also preparing for the future. Whether through cloud, on-premises, or hybrid solutions, the right infrastructure can make all the difference in turning AI from an experiment into a business driver


Source link
Continue Reading
Advertisement
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Technology

Microsoft to re-launch ‘privacy nightmare’ AI screenshot tool

Published

on

Microsoft to re-launch ‘privacy nightmare’ AI screenshot tool

Microsoft says it has “listened to feedback” following a privacy row over a new tool which takes regular screenshots of users’ online activity.

It was labelled a potential “privacy nightmare” by critics when it was unveiled in May 2024 – prompting the tech giant to postpone its release.

It now plans to relaunch the artificial intelligence (AI) powered tool in November on its new CoPilot+ computers.

Some of its more controversial features have been stripped out – for example, it will be opt-in whereas the original version could not be switched off.

Advertisement

The controversy over Recall had led the Information Commissioner’s Office (ICO), the UK’s data watchdog, to “make enquiries” with the tech giant about the tool.

It said it had now been informed that a “series of changes” had been made to the product.

“We will be continuing to assess Recall as Microsoft moves toward launch”, it said in a statement.

When it initially announced the tool at its developer conference in May, Microsoft said it used AI “to make it possible to access virtually anything you have ever seen on your PC”, and likened it to having photographic memory.

Advertisement

It said Recall could search through a users’ past activity, including their files, photos, emails and browsing history.

It was designed to help people find things they had looked at or worked on previously by searching through desktop screenshots taken every few seconds.

But critics quickly raised concerns, given the quantity of sensitive data the system would harvest, with one expert labelling it a potential “privacy nightmare.”

Recall was never made publicly available.

Advertisement

A version of the tool was set to be rolled out with CoPilot+ computers – which Microsoft billed as the fastest, most intelligent Windows PCs ever built – when they launched in June, after Microsoft told users it had made changes to make it more secure.

But its launch was delayed further and has now been pushed back to the autumn. The company has also announced extra security measures for it.

“Recall is an opt-in experience. Snapshots and any associated information are always encrypted,” said Pavan Davuluri, Microsoft’s corporate vice president of Windows and devices.

He added that “Windows offers tools to help you control your privacy and customise what gets saved for you to find later”.

Advertisement

However a technical blog about it states that “diagnostic data” from the tool may be shared with the firm depending on individual privacy settings.

The firm added that screenshots can only be accessed with a biometric login, and sensitive information such as credit card details will not be snapped by default.

Recall is only available on the CoPilot+ range of bespoke laptops featuring powerful inbuilt AI chips.

Professor Alan Woodward, a cybersecurity expert at Surrey University, said the new measures were a significant improvement.

Advertisement

“Before any functionality like Recall is deployed the security and privacy aspects will need to be comprehensively tested,” he said.

However he added he would be rushing to use it.

“Personally I would not opt-in until this has been tested in the wild for some time.”

Source link

Advertisement

Continue Reading

Science & Environment

America’s coal communities could help the U.S. triple nuclear power

Published

on

America's coal communities could help the U.S. triple nuclear power


A bulldozer moves coal that will be burned to generate electricity at the American Electric Power coal-fired power plant in Winfield, West Virginia.

Luke Sharrett | Bloomberg | Getty Images

The planned restart of Three Mile Island is a step forward for nuclear power, but the U.S. needs to deploy new plants to keep up with rising electricity demand, one of the nation’s top nuclear officials said this week.

The U.S. needs to at least triple its nuclear fleet to keep pace with demand, slash carbon-dioxide emissions and ensure the nation’s energy security, said Mike Goff, acting assistant secretary for the Office of Nuclear Energy at the Department of Energy.

Advertisement

The U.S. currently maintains the largest nuclear fleet in the world with 94 operational reactors totaling about 100 gigawatts of power. The fleet supplied more than 18% of the nation’s electricity consumption in 2023.

The U.S. needs to add 200 gigawatts of nuclear, Goff told CNBC in an interview. This is roughly equivalent to building 200 new plants, based on the current average reactor size in the U.S. fleet of about a gigawatt.

“It’s a huge undertaking,” Goff said. The U.S. led a global coalition in December that formally pledged to meet this goal by 2050. Financial institutions including Goldman Sachs and Bank of America endorsed the target at a climate conference in New York City this week.

Constellation Energy‘s plan to restart Three Mile Island by 2028 is a step in the right direction, Goff said. The plant operated safely and efficiently, only shutting down in 2019 for economic reasons, he said.

Advertisement

The reactor that Constellation plans to re-open, Unit 1, is not the one the partially melted down in 1979.

Microsoft will purchase electricity from the plant to help power its data centers. Goff said the advent of large data centers that consume up to a gigawatt of electricity only reinforces the need for new reactors.

“A lot of the data centers are coming in and saying they do need firm, 24/7, baseload clean electricity,” Goff said. “Nuclear is obviously a perfect match for that,” he said.

But restarting reactors in the U.S. will only provide a small fraction of the nuclear power that is needed, he said. There are only a handful of shuttered plants that are potential candidates for restarts, according to Goff.

Advertisement

“It’s not a huge number,” Goff said of potential restarts. “We need to really be moving forward also on deploying plants,” he said.

From coal to nuclear

Coal communities across the U.S. could provide a runway to build out a large number of new nuclear plants. Utilities in many parts of the U.S. are phasing out coal as part of the clean energy transition, creating a supply gap in some regions because new generation is not being built fast enough.

Recently shuttered coal plants, those expected to retire, and currently operating plants with no estimated shutdown date yet could provide space for up to 174 gigawatts of new nuclear across 36 states, according to a Department of Energy study published earlier this month.

Coal plants already have transmission lines in place, allowing reactors at those sites to avoid the long process of siting new grid connections, Goff said. The plants also have people experienced in the energy industry who could transition to working at a nuclear facility, he said.

Advertisement

“We can actually get a significant cost reduction by building at a coal plant,” Goff said. “We can maybe get a 30% cost reduction compared to just going on a greenfield site.”

Cost overruns and long timelines are major hurdles for building new nuclear plants. The expansion of the Vogtle plant in Georgia with two new reactors, for example, cost more than $30 billion and took around seven years longer than expected.

Expanding operational nuclear plants and building at retired sites in the U.S. could create a pathway for up to 95 gigawatts worth of new reactors, according to the DOE study. Between coal and nuclear sites, the U.S. potentially has space for up to 269 gigawatts of additional nuclear power.

The potential capacity would depend on whether advanced, smaller reactors are built at the sites, or larger reactors with a gigawatt or more of power.

Advertisement

More electricity could potentially be generated if the smaller reactors were rolled out on a large scale because there is space for more them, according to the DOE study. Some of these smaller advanced designs, however, are still years away from commercialization.

But rising electricity demand from data centers, manufacturing and the electrification of the economy could provide a catalyst to build the larger plants as well, according to Goff. The Three Mile Island restart, for example, would bring back just under a gigawatt of power to meet Microsoft’s needs.

“That increased power demand, that will lead toward an additional push toward those gigawatt size reactors as well,” he said.

Restarts likely to secure greenlight

While reactor restarts aren’t a silver bullet, shoring up and maintaining the existing fleet is crucial, Goff said. The U.S. went through a decade-long period in which reactors were shutting down because they could not compete with cheap, abundant natural gas.

Advertisement

The economics are changing, however, with tax support from the Inflation Reduction Act and nuclear increasingly valued for its carbon-free attributes, Goff said.

“One of the issues with the economics, especially in the non-regulated utilities, was there was no value necessarily for clean, baseload electricity,” he said. “There is a lot more recognition of the need for that clean, firm, reliable baseload for nuclear”

Constellation’s decision to restart Three Mile Island follows in the footsteps of the Palisades nuclear plant in Michigan. The private owner, Holtec International, plants to restart Palisades in 2025. The two restarts are subject to review and approval by the Nuclear Regulatory Commission.

“They are an independent agency, but I expect if the safety cases are presented, they’re going to approve it,” Goff said of those potential restarts.

Advertisement

“Constellation obviously operated the Three Mile Island plant for years, and has a very large fleet of reactors that they’ve operated safely and efficiently,” he said. “They will continue to have a great expertise in moving those plants to continue their safe operation.”

But finding additional plants to restart could prove difficult, said Doug True, chief nuclear officer at the Nuclear Energy Institute.

“It gets harder and harder,” True previously told CNBC. “A lot of these plants have already started the deconstruction process that goes with decommissioning and the facility wasn’t as thoroughly laid up in a way that was intended to restart in any way.”



Source link

Advertisement

Continue Reading

Servers computers

Let's Check Out an Old Blade Server System with 32 CPUs!

Published

on

Let's Check Out an Old Blade Server System with 32 CPUs!



Many, many servers these days run as virtual machines — but there was a time when virtualization was still just catching on, and companies needed physical servers to be as dense as possible. So let’s look at a blade server system from around 2010 that packs 32 CPU sockets and weighs 500 pounds!

Image of C7000 chassis will full-height blades: https://commons.wikimedia.org/wiki/File:HP_BladeSystem_c7000_Enclosure.jpg#/media/File:HP_BladeSystem_c7000_Enclosure.jpg

—————————————-­————————————-

Please consider supporting my work on Patreon: https://www.patreon.com/thisdoesnotcompute

Follow me on Twitter and Instagram! @thisdoesnotcomp

—————————————-­————————————-

Music by
Epidemic Sound (http://www.epidemicsound.com).

source

Continue Reading

Technology

Governments bans on quantum computer exports have no basis in science

Published

on

Quantum computer isolated on black. Golden gear, quantum computing, quantum cryptography, steampunk, Q bits, parallel computing. 3D illustration, 3D render; Shutterstock ID 2218001269; purchase_order: -; job: -; client: -; other: -
Quantum computer isolated on black. Golden gear, quantum computing, quantum cryptography, steampunk, Q bits, parallel computing. 3D illustration, 3D render; Shutterstock ID 2218001269; purchase_order: -; job: -; client: -; other: -

Shutterstock/Marko Aliaksandr

Imagine if governments around the world announced restrictions on the sale of rulers that are 34 centimetres long. You would be pretty confused, given there doesn’t seem to be anything special about that length – and 34cm rulers don’t exist.

Such legislation would be ludicrous, but something similar has been enacted for quantum computers in several nations (see “Multiple nations enact mysterious export controls on quantum computers“). The restrictions – which limit the export of computers with 34 or more qubits, or quantum bits, and error rates below a certain threshold – are puzzling, as such devices have no practical use, according to all published research.

But the very specificity of the number suggests some thought behind it. Clearly, someone, somewhere, is worried about nefarious use of these devices – most likely their potential to break widely used encryption methods – and wants them restricted in the name of national security.

Advertisement

So what is going on? There are two possibilities here: either they are wrong, as scientific evidence suggests, and pointless legislation is now being cut and pasted across the world, or they are right and have now alerted their adversaries that this is a number worth paying attention to. Both possibilities seem counterproductive, but without access to the research behind these restrictions, it is impossible to say.

One of the great strengths of science is that it is an open endeavour. For all its faults, peer review allows us to harness minds around the world to scrutinise and improve research. Our approach to making policy should be no different.

The Intergovernmental Panel on Climate Change shows just how powerful this can be. By publicly synthesising research, it has allowed policy-makers to understand what needs to be done to tackle climate change – and allowed others to use that evidence base to analyse policy decisions. Equally, published research during the covid-19 pandemic allowed for a public discussion on rules being imposed on us. Simply plucking a number from the air, as seems to be the case with quantum computers, is no way to govern.

Topics:

Advertisement

Source link

Continue Reading

Technology

The best Nintendo Switch games for kids

Published

on

The best Nintendo Switch games for kids

Unlike the other major consoles on the market, Nintendo has worked hard to establish the Switch as a console for all ages. The majority of the best Switch games are made to appeal to gamers both young and old, with a great mix of big-budget games and some of the best indie games. With so many games on the market, and only more exciting upcoming Switch games, it can be difficult to tell which games are best for kids and which are better games for adults. Luckily, we have compiled a few of the most popular kids’ games for the Switch all in one place!






Source link

Continue Reading

Servers computers

Dell PowerEdge R820 Server: Complete Overview, Configuration & Benefits

Published

on

Dell PowerEdge R820 Server: Complete Overview, Configuration & Benefits



#BuyDellServer #BuyServerinUAE #BuyDellR820Refurb #Dell_R820_Server #ServerBasket

For Complete Information on Dell PowerEdge R820 Rack Server, Please Contact Us at:

Website: https://www.serverbasket.ae
Email: sales@serverbasket.ae
Contact No: +971 50520 3535
Whatsapp No: +971 50520 3535

Subscribe To Our Channel @ https://www.youtube.com/channel/UCXxaNw3P6OdkW29dMgTsv2A

Check out the Powerful Dell PowerEdge R720 Rack Server.

Refurbished Dell R820 Server: https://www.serverbasket.ae/refurbished-dell-poweredge-r820-server-8sff.html
Rental: https://www.serverbasket.ae/it-rental.html

Buy Refurbished Dell PowerEdge R820 Server from Server Basket as it is Optimal for Growing SME & Demanding Businesses. With High Memory Capacity & Huge Storage Ability, Dell R820 Server is an Ideal Server for Tech Startups & Booming SME Businesses.

Key Benefits:

-Save IT Investment With Refurbished Server
– Energy Optimized
– Manage data overload
– Accelerate the solution
– Virtualize more with the R820
– Highly Scalable Memory
– Better Storage Capabilites
– Easy To Use
– Best Price in Market
– Instant Delivery
– Quick Support
– Lowest Price
– Windows & Linux OS Supported

Dell PowerEdge R820 Server Specifications:

CPU Capacity:

– Supports 4 Processors
– Intel® Xeon® processor E5-4600 product family
– Single CPU = 8 Cores Max
– Quad CPU = 16 Cores Max
– Max VCPUs: 28 VCPUs

RAM Capacity:

– Inbuilt 248 DIMM Slots
– 32GB Max Memory Per DIMM Slot
– 1.5TB Maximum Memory Capacity
– Supported Technology: DDR3 Memory
– RAM Speed: 1600MT/s

Storage Capacity:

– Up to sixteen 2.5” hot-plug SAS, SATA, or SSD

– Max Potential: 16TB Storage

Raid Controller:

Internal controllers:

– PERC H310
– PERC H710
– PERC H710P

External HBAs (RAID):

PERC H810
External HBAs (non-RAID)
6Gbps SAS HBA

Power Supply:

– Platinum efficiency 750W or
– Platinum+ Efficiency 1100W Power Supplies

Remote Management:

– iDRAC7 Express (Default)
– iDRAC7 Enterprise (Upgrade option), both with Lifecycle Controller
– 8GB or 16GB vFlash media (Upgrade options)

Compatible Operating System With Refurbished Dell R910 Server:

– Microsoft® Windows Server® 2008 R2 SP1, x64
– SUSE® Linux® Enterprise Server
– Red Hat® Enterprise Linux

Virtualization Options:

– Citrix® XenServer®
– VMware vSphere®
– Hyper-V™ v2

Additional Features:

I/O Slots :
7 PCIe slots:
– Two x16 half-length, full-height
– One x8 half-length, full-height
– Three x8 half-length, half-height
– One x8 half-length, full-height for the RAID controller

Check out the Powerful Dell PowerEdge R820 Server from Server Basket. .

source

Continue Reading

Trending

Copyright © 2024 WordupNews.com