After a career counselor visited one of her classes earlier this year, Lily Hatch found herself asking a chatbot for guidance about college.

A junior at Wake Forest High School in North Carolina, Hatch had taken an in-class career quiz that recommended she pursue dermatology. She had finished quickly and so approached the counselor to find out how to explore that profession further. The counselor gave a couple of suggestions, before adding that Hatch could also play with a chatbot to explore her college options.

So, that’s what Hatch did.

But instead of returning information on which schools rank highly for dermatology, the chatbot — a general-purpose consumer product, rather than an edtech tool — veered off into offering information about climate, telling Hatch to consider the University of North Carolina in Wilmington because it’s near a beach.

It felt a little like a runaway train, with the bot dragging her down a pre-laid track. “I was looking for advice on what colleges would be ideal for me. And it switches into going more into what things in my life I would be looking for in the future, which was not what I was looking for,” Hatch says.

Today’s high school students — who spent years of their academic careers surfing disruptions and the challenges of returning to the classroom after the pandemic school closures — are preparing to enter a labor force and broader economic system that can seem confusing and unstable, as technologies like artificial intelligence are reshaping the career ladders that their parents climbed. Some national surveys show that Gen Z students feel more prepared for their futures now than they did in past years, but for those about to graduate, that’s not always the case. Many students describe a general pessimism about the future.

“There’s a lot of fear there,” says Matthew Tyson, CEO of Tapestry Public Charter School in DeKalb County, Georgia. Tyson notes that many of his students aren’t planning for college, or feel discouraged by the fast-changing nature of life around them.

Navigating these major shifts about starting a career requires both educators and young people to think flexibly, according to experts. Students need honest guidance, Tyson says, adding that adults should be transparent about the reality that they don’t have all the answers.

But new AI tools don’t have all the answers either, not even those purpose-built to offer career guidance. At least, some human counselors don’t think so.

“The AI stuff is kind of crazy to think about,” says Ian Trombulak, a school counselor in Vermont. “That’s not going to help us reverse the trend here of career readiness scores being low.”

Still, some say they are open to the possibility that offloading aspects of their work to AI may, ironically, free them up to offer better support to students contending with the disruptions AI is creating in the labor market.

A Tough Job

Career counseling is a demanding gig these days.

Youth unemployment rates are increasing, and recent assessments reveal dips in college readiness. And two-thirds of parents desire an increase in job skills and workforce training opportunities in high school, according to a figure from the Hunt Institute.

Yet counselors often have to make tough choices between giving academic and career advice or addressing students’ emotional crises, and many students seem to lack support systems, says Tyson, from the Georgia public charter school. Student traumas can spout up to the adults meant to give those students advice.

“A lot of times, there’s only so much water that can be taken out of a glass before the glass is empty,” Tyson says of counselors’ emotional states. Eager to assist students, counselors can burn out.

They also have to deal with staff shortages. Tapestry, Tyson’s public charter, doesn’t suffer from a shortage of counseling educators like some nearby schools. It has three counselors for 300 students, according to Tyson.

But across Georgia, there are 378 students for every school counselor, according to the latest data from the American School Counselor Association, which recommends one counselor for every 250 students. And that’s hardly the worst in the nation, with the ratios sitting at 573 students per counselor in Michigan and 645 per counselor in Arizona.

With human resources strained, schools are now considering how to use AI to create more opportunities to meaningfully advise students on how to approach the future.

Innovative uses of artificial intelligence can amplify the work of human college and career counselors, argues June Han, the CEO of EduPolaris AI, a company which offers Eddie, an AI counseling platform that includes counselor, student and parent portals licensed by schools. The company raised $1 million in early investments, and the company’s platform — which relies, at least in part, on third-party large language models — is being piloted in a handful of Title I high schools, the CEO told EdSurge.

School-support organizations, including the Homeschool Association of California, list the tool as a recommended AI resource, as does the White House.

Tapestry is one of the schools piloting Eddie. The platform has helped, according to Tyson, particularly because the dashboard lets Tyson see useful information such as how many students have completed their reference letters for college applications. From the dashboard, he can send a nudge to students, reminding them to finish. That feature cuts down on the number of meetings he has to take. The data collected by the platform also provides clues about what to focus on when he works with students, and where they need the most help, Tyson says.

The Davidson Institute, a nonprofit that provides educational opportunities to “profoundly gifted” students, uses the “Ask Eddie” chatbot function to counsel families in the Young Scholars Program for students ages of 5 through 18. Many of those students are on “nontraditional paths,” looking at early college, or coming from accelerated grades or homeschool backgrounds, says Megan Cannella, director of outreach.

More than 200 families in the program have used the tool since February 2025, according to Cannella. She says the big selling point is that it’s available 24/7 and in a number of languages. The nonprofit doesn’t offer traditional school counseling, so the AI tool boosts the limited support that staff provides. It’s proven particularly helpful for families just starting their college journey, and for homeschoolers, she adds.

Meanwhile, what students want from a career is also changing, in a way that makes it difficult for career counselors to keep up.

Shifting Interests

In northwest Missouri, students have become more interested in exploring non-college pathways after graduation, such as military service or vocational training, says Geoff Heckman, a school counselor at Platte County High School.

Apprenticeships, internships and alternative credentials feature more prominently in students’ plans these days because these options prepare them to step right into jobs when they leave high school, Heckman says. Indeed, around the country, students are skeptical about college, meaning that high school counselors can’t assume that pathway.

The students Heckman counsels at the public school outside of Kansas City are also starting to find postsecondary guidance resources on their own more often, using AI and social media, he adds.

There have been cultural shifts, sometimes away from the kinds of jobs the school’s infrastructure is set up to support. Not long ago, the career and technical school next door to Heckman’s school had a waiting list for its law enforcement opportunities. Now, there’s much less interest, Heckman reports.

Instead, some of the careers students now desire are hard for Heckman to understand. In the years since he’s become a counselor, students have found jobs as social media influencers and professional gamers. Indeed, the number of students who say their dream is to be a social media star has swelled.

“I want to support a student no matter how wild their dream may sound to me,” Heckman says.

It comes down to helping them construct a plan of attack, teaching them to research the industry of interest, to discern how strong their passion for this dream is and to reach out for mentorship, he adds. For example, last year a student came to Heckman and said she wanted to be a pilot. There was no program for that at the high school. But an effort from the district was able to create a new internship opportunity for the student through the local Air Guard, which has a flight school.

Similar situations occur in schools across the country, and many places are keen to build stronger career pathways.

For instance, Vermont switched over to proficiency-based grading requirements — beginning with the class of 2020 — and it has started to incorporate “self-direction skills” in the assessment of students. It’s a signal for schools to focus on skills that will be useful in a future where counselors can’t predict precisely what jobs students will be working, according to one school counselor in the state.

A lifelong Vermonter, Ian Trombulak came to career counseling after working in a group home after college. It sparked something, he says. After he left the emotionally tense work of a group home, he found himself pulled into schools where he could be the type of person who had helped him through high school.

Trombulak has worked in public education for nine years, and in that time, he’s seen “this continued drumbeat” where public educators are asked to do more with fewer resources, even as core components of education like curriculum have become swept up in political battles. Budgets are too tight to hire enough counselors, and counselors have too many students to feasibly advise, he admits.

“You know, we’re not superheroes,” he says. “At a certain point, you are constrained by the kind of resources that you have at your disposal, and public education is not working with a whole lot right now. Even in the best of times, it can be a struggle.”

Helping students steer through their uncertainty requires a deft approach. At the same time he’s helping ninth graders find their footing in the murky transition from middle school to high school, he’s also advising students on what could happen after graduation. On average, he meets about five to 10 students per day. Some meetings are pre-planned and some are drop-ins. A lot of his job happens outside of scheduled sessions, he says. While stopping in on a teacher, students will pull him aside to check in. There are about a dozen of those encounters a day.

Schools may be turning to AI out of desperation, Trombulak says. But he doubts it will advise students as well as human counselors.

EduPolaris leaders feel that the safeguards on Eddie, the AI counseling platform, position it to boost the human work of counselors. Han, the company’s CEO, argues that Eddie is so human-centric and school-specific that the tool amplifies the human counselor’s efforts, allowing for schools to provide personalized guidance even with limited resources.

Han argues that initial skepticism from counselors stems from a lack of AI literacy. Counselors and educators are afraid of losing control, she says.

Yet even if AI proves adept at providing accurate, useful career information and advice, that may miss the subtler value that can emerge when students sit down to chat with a trusted adult. That type of interaction is essential to building the “social capital” and interpersonal networks that actually help young people secure jobs, some researchers argue.

And much of Trombulak’s work is relational rather than transactional. Mostly gone are the days of relying on personality tests and career quizzes. Instead, Trombulak says, counselors hold open-ended conversations probing what students feel passionate about. It’s more self-exploratory and requires a more human touch. “I’m almost there as a mirror,” Trombulak says, or as a backboard to bounce ideas off.

Ultimately, a powerful lesson Trombulak believes he can teach students is how to find answers on their own. As students try on ideas, counselors teach them about what kind of path they would have to take to end up in a job. It means a lot of Googling with students. He goes through the process of how he, as a well-educated adult, would find answers.

Part of that process now is, yes, verifying information gathered from AI.

Unreliable Narrator

For students, what matters most is the quality of the advice they receive, whether it comes from a human or a bot.

After two or three weeks of back and forth with the chatbot, Hatch, the junior from North Carolina, didn’t return to the human career counselor.

But that doesn’t mean she found the AI useful.

The scraps of information she got could have been easily discovered by a quick Google search, she says. The experience contributed to her overall skepticism of AI, which she acts on as a student leader for her school’s chapter of Young People’s Alliance, which advocates for stronger AI regulations and more job training opportunities for young adults.

She doesn’t know yet where she wants to attend college, or even what she’ll study. Right now, instead of dermatology, Hatch is considering education as a career path.

So, what does she think about using AI for career counseling?

She wouldn’t recommend it. In fact, she’s not so keen on what she sees as an overreliance on technology in general. Students she knows use it to churn out passable school work, and in response, teachers even seem ready to give out good grades for subpar work when they feel it’s not AI-generated.

Students should really slow down, and rely on AI less, she says: “I feel like it overall is not as useful as people make it out to be.”

Researchers at Nvidia have developed a technique that can reduce the memory costs of large language model reasoning by up to eight times. Their technique, called dynamic memory sparsification (DMS), compresses the key value (KV) cache, the temporary memory LLMs generate and store as they process prompts and reason through problems and documents.

While researchers have proposed various methods to compress this cache before, most struggle to do so without degrading the model’s intelligence. Nvidia’s approach manages to discard much of the cache while maintaining (and in some cases improving) the model’s reasoning capabilities.

Experiments show that DMS enables LLMs to “think” longer and explore more solutions without the usual penalty in speed or memory costs.

The bottleneck of reasoning

LLMs improve their performance on complex tasks by generating “chain-of-thought” tokens, essentially writing out their reasoning steps before arriving at a final answer. Inference-time scaling techniques leverage this by giving the model a larger budget to generate these thinking tokens or to explore multiple potential reasoning paths in parallel.

However, this improved reasoning comes with a significant computational cost. As the model generates more tokens, it builds up a KV cache.

For real-world applications, the KV cache is a major bottleneck. As the reasoning chain grows, the cache grows linearly, consuming vast amounts of memory on GPUs. This forces the hardware to spend more time reading data from memory than actually computing, which slows down generation and increases latency. It also caps the number of users a system can serve simultaneously, as running out of VRAM causes the system to crash or slow to a crawl.

Nvidia researchers frame this not just as a technical hurdle, but as a fundamental economic one for the enterprise.

“The question isn’t just about hardware quantity; it’s about whether your infrastructure is processing 100 reasoning threads or 800 threads for the same cost,” Piotr Nawrot, Senior Deep Learning Engineer at Nvidia, told VentureBeat.

Previous attempts to solve this focused on heuristics-based approaches. These methods use rigid rules, such as a “sliding window” that only caches the most recent tokens and deletes the rest. While this reduces memory usage, it often forces the model to discard critical information required for solving the problem, degrading the accuracy of the output.

“Standard eviction methods attempt to select old and unused tokens for eviction using heuristics,” the researchers said. “They simplify the problem, hoping that if they approximate the model’s internal mechanics, the answer will remain correct.”

Other solutions use paging to offload the unused parts of the KV cache to slower memory, but the constant swapping of data introduces latency overhead that makes real-time applications sluggish.

Dynamic memory sparsification

DMS takes a different approach by “retrofitting” existing LLMs to intelligently manage their own memory. Rather than applying a fixed rule for what to delete, DMS trains the model to identify which tokens are essential for future reasoning and which are disposable.

“It doesn’t just guess importance; it learns a policy that explicitly preserves the model’s final output distribution,” Nawrot said.

The process transforms a standard, pre-trained LLM such as Llama 3 or Qwen 3 into a self-compressing model. Crucially, this does not require training the model from scratch, which would be prohibitively expensive. Instead, DMS repurposes existing neurons within the model’s attention layers to output a “keep” or “evict” signal for each token.

For teams worried about the complexity of retrofitting, the researchers noted that the process is designed to be lightweight. “To improve the efficiency of this process, the model’s weights can be frozen, which makes the process similar to Low-Rank Adaptation (LoRA),” Nawrot said. This means a standard enterprise model like Qwen3-8B “can be retrofitted with DMS within hours on a single DGX H100.”

One of the important parts of DMS is a mechanism called “delayed eviction.” In standard sparsification, if a token is deemed unimportant, it is deleted immediately. This is risky because the model might need a split second to integrate that token’s context into its current state.

DMS mitigates this by flagging a token for eviction but keeping it accessible for a short window of time (e.g., a few hundred steps). This delay allows the model to “extract” any remaining necessary information from the token and merge it into the current context before the token is wiped from the KV cache.

“The ‘delayed eviction’ mechanism is crucial because not all tokens are simply ‘important’ (keep forever) or ‘useless’ (delete immediately). Many fall in between — they carry some information, but not enough to justify occupying an entire slot in memory,” Nawrot said. “This is where the redundancy lies. By keeping these tokens in a local window for a short time before eviction, we allow the model to attend to them and redistribute their information into future tokens.”

The researchers found that this retrofitting process is highly efficient. They could equip a pre-trained LLM with DMS in just 1,000 training steps, a tiny fraction of the compute required for the original training. The resulting models use standard kernels and can drop directly into existing high-performance inference stacks without custom hardware or complex software rewriting.

DMS in action

To validate the technique, the researchers applied DMS to several reasoning models, including the Qwen-R1 series (distilled from DeepSeek R1) and Llama 3.2, and tested them on difficult benchmarks like AIME 24 (math), GPQA Diamond (science), and LiveCodeBench (coding).

The results show that DMS effectively moves the Pareto frontier, the optimal trade-off between cost and performance. On the AIME 24 math benchmark, a Qwen-R1 32B model equipped with DMS achieved a score 12.0 points higher than a standard model when constrained to the same memory bandwidth budget. By compressing the cache, the model could afford to “think” much deeper and wider than the standard model could for the same memory and compute budget.

Perhaps most surprisingly, DMS defied the common wisdom that compression hurts long-context understanding. In “needle-in-a-haystack” tests, which measure a model’s ability to find a specific piece of information buried in a large document, DMS variants actually outperformed the standard models. By actively managing its memory rather than passively accumulating noise, the model maintained a cleaner, more useful context.

For enterprise infrastructure, the efficiency gains translate directly to throughput and hardware savings. Because the memory cache is significantly smaller, the GPU spends less time fetching data, reducing the wait time for users. In tests with the Qwen3-8B model, DMS matched the accuracy of the vanilla model while delivering up to 5x higher throughput. This means a single server can handle five times as many customer queries per second without a drop in quality.

The future of memory

Nvidia has released DMS as part of its KVPress library. Regarding how enterprises can get started with DMS, Nawrot emphasized that the barrier to entry is low. “The ‘minimum viable infrastructure’ is standard Hugging Face pipelines — no custom CUDA kernels are required,” Nawrot said, noting that the code is fully compatible with standard FlashAttention. 

Looking ahead, the team views DMS as part of a larger shift where memory management becomes a distinct, intelligent layer of the AI stack. Nawrot also confirmed that DMS is “fully compatible” with newer architectures like the Multi-Head Latent Attention (MLA) used in DeepSeek’s models, suggesting that combining these approaches could yield even greater efficiency gains.

As enterprises move from simple chatbots to complex agentic systems that require extended reasoning, the cost of inference is becoming a primary concern. Techniques like DMS provide a path to scale these capabilities sustainably.

“We’ve barely scratched the surface of what is possible,” Nawrot said, “and we expect inference-time scaling to further evolve.”