Nahla Davies examines what constitutes an appropriate data integrity framework, and how inadequate frameworks damage data quality.

If you asked most companies whether they have a data integrity framework, they’d say yes without hesitation. They’d point you to a shared drive, maybe a Confluence page, possibly a colour-coded spreadsheet with tabs labelled ‘Validation Rules’ and ‘Ownership Matrix’. It looks official. It’s got a logo on it. Someone even added conditional formatting.

But here’s the thing: looking like a framework and actually functioning as one are two wildly different realities. Across industries, organisations are confusing documentation with governance, and the gap between those two things is where data quality quietly falls apart. The problem isn’t that teams don’t care. It’s that they’ve convinced themselves the spreadsheet is enough.

The spreadsheet trap is more common than anyone admits

There’s a pattern that plays out in nearly every mid-size org that’s undergone some kind of digital transformation push in the last five years. Someone in data engineering or analytics gets tasked with ‘building a data integrity framework’. They do their research, pull together some best practices, and create a document. Maybe it lives in Google Sheets, maybe it’s a Notion database, maybe it’s an actual PDF that got emailed around once and then forgotten about. Whatever form it takes, it checks a box. Leadership sees it and feels reassured.

The trouble starts when that document has to survive contact with reality. Data pipelines change. New sources get added. Team members rotate. And that spreadsheet? It doesn’t update itself. It doesn’t send alerts when a schema shifts or when a critical field starts returning nulls at twice the usual rate. It just sits there, frozen in the moment it was created, slowly becoming a historical artifact rather than an operational tool.

What’s worse is that people keep referencing it as though it’s still accurate. Decisions get made based on validation rules that haven’t been reviewed in months. Ownership columns list people who’ve left the company. It’s the organisational equivalent of navigating with a map from 2019 and wondering why you keep hitting dead ends.

And it’s not a niche problem. A 2023 Gartner survey found that poor data quality costs organisations an average of $12.9m per year. That number doesn’t come from dramatic, headline-grabbing breaches. It comes from the slow, invisible accumulation of bad records, missed anomalies, and unchecked assumptions that a static document simply can’t catch.

What a real framework actually looks like

So what separates a functioning data integrity framework from a well-formatted spreadsheet? It comes down to whether the thing can operate without someone manually babysitting it. A real framework is embedded in your infrastructure. It’s automated, observable and responsive.

That means validation checks run as part of your data pipelines, not as a quarterly audit someone remembers to do in the last week of the quarter. It means the data is correctly annotated and that there’s monitoring in place that flags anomalies in real time, whether that’s a sudden spike in null values or a mismatch between source and destination row counts. Tools like Great Expectations, Monte Carlo and dbt tests exist specifically to bring this kind of rigor into the workflow.

It also means ownership is enforced through tooling, not just documented in a tab. When a data asset has a registered owner in a data catalogue, and that catalogue integrates with your alerting system, accountability becomes structural. It stops being something you have to chase people about in Slack.

There’s a cultural component here, too. Organisations with mature data integrity practices treat data quality as a product concern and are better prepared to establish proper AI governance. Product managers care about it. Analysts flag issues proactively instead of working around them. Engineers write tests for data the same way they write tests for code. That kind of culture doesn’t emerge from a spreadsheet. It emerges from leadership, making it clear that data integrity is a priority, not a side project someone handles when things are slow.

The companies getting this right tend to share a few traits. They’ve invested in observability across their data stack. They treat schema changes as events that require review, not things that just happen silently. And they’ve moved past the idea that documentation alone equals governance.

Why it matters more now than it did five years ago

The stakes around data integrity have shifted significantly. Five years ago, a bad record in a reporting dashboard was annoying but manageable. Today, that same bad record might be feeding a machine learning model that’s making automated decisions about credit, hiring or patient care. The blast radius of poor data quality has expanded because the systems consuming that data have become more autonomous and more consequential.

Regulatory pressure is also mounting. Frameworks like the EU’s AI Act and evolving data privacy regulations are putting more scrutiny on how organisations manage the data that powers their products. It’s getting harder to shrug off data quality issues as ‘technical debt we’ll get to eventually’. Regulators want to see evidence of governance, and a spreadsheet with last year’s date on it won’t cut it.

There’s also the competitive angle. Companies that can trust their data move faster. They make decisions with more confidence. They spend less time reconciling conflicting reports and more time actually acting on insights. Data integrity isn’t glamorous, but it’s one of those foundational things that quietly determines whether an organisation can execute on its strategy or just talk about it.

Final thoughts

The uncomfortable truth is that most data integrity frameworks weren’t built to be frameworks at all. They were built to satisfy a request, to check a compliance box, or to give someone something to present in a meeting.

And that’s fine as a starting point. Every mature system started somewhere. But if your ‘framework’ is still a spreadsheet that no one’s touched in six months, it’s time to be honest about what you actually have.

Real integrity requires automation, observability and cultural buy-in. The spreadsheet was never the destination. Treat it as the rough draft it always was, and start building something that can actually keep up with your data.

By Nahla Davies

Nahla Davies is a software developer and tech writer. Before devoting her work full time to technical writing, she managed – among other intriguing things – to serve as a lead programmer at an Inc. 5,000 experiential branding organisation, where clients include Samsung, Time Warner, Netflix and Sony.

Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.

Elon Musk is still taking OpenAI to court over its transition to a for-profit company, but today he amended the complaint so that he won’t personally get any of the $150 billion in damages he’s pushing for. The Wall Street Journal reported that if Musk wins in his upcoming trial, he wants any damages should be awarded to the OpenAI nonprofit branch. He’s also seeking OpenAI CEO Sam Altman’s removal from the nonprofit’s board of directors if his suit succeeds.

Musk launched a lawsuit against OpenAI in 2024, claiming that the business had become a “closed-source de facto subsidiary” of Microsoft when it dropped its nonprofit designation. He claims that, as a co-chair of the OpenAI founding group, the change to a for-profit operation defrauded him as a donor. As a result, he’s now claiming that he, or apparently the remaining nonprofit side of OpenAI, deserve a portion of the company’s current valuation.

Considering the reputation Musk, Altman and their various business endeavors have for creating spicy PR situations, it seems likely that the exchanges between the two camps will get more heated as the trial date approaches.

• Storm enables session hijacking that bypasses passwords and multi-factor authentication
• Attackers can restore stolen sessions remotely without triggering standard security alerts
• Malware operates server-side to process encrypted browser credentials for stealthy exploitation

A new strain of infostealer malware dubbed Storm is changing how account compromise works, experts have warned.

New findings from Varonis Threat Labs have outlined how this strain moves away from passwords and focuses on session cookies that keep users logged in.

Japanese musicians commemorated Wikipedia’s 25th anniversary with a unique composition made up entirely of Wikipedia entries. Open Reel Ensemble produced the song as part of a virtual birthday celebration, and it’s a true journey because it’s totally made up of ancient reel-to-reel equipment that also function as instruments. Every sound is produced by physically moving the tape over the heads, with no artificial samples added after the fact.

The video shows the trio jamming at a table surrounded by recorders. Snippets of Wikipedia material appear on screen, and the lads are completely freestyling their way through them; grab an entry on how a term is defined, and the machines come to life. One of them is rewinding quickly to fit the description, producing a wonderful smooth swooshing sound across the speakers. Another one goes into fast-forward mode whenever the text flashes by, raising the pitch and adding a little of edge to the beat.

Each one flows seamlessly into the next, and the overall effect just seems natural Pitch control slows down for deeper tones and speeds up for brighter ones. Loops take small pieces and repeat them to create these steady rhythms below, while vibrato puts in some wavey portions by slowly changing the pace. Tremolo reduces the loudness in these rapid little pulses, and when they scratch the tape edges, they make these sharp little snappy noises. Then there are the wow effects, which are simply natural wobbles that go up and down in the same rhythm as your breathing.

The layers just develop as the devices interact with one another; definition after definition for reel-to-reel recording, tension, cut-up technique, and even magnetic punk all appear on screen, activating their corresponding action. The music remains techno and dance-friendly throughout, but it is all anchored in the mechanical slapping and hissing of the tape. The moniker “Cyklepedia” refers to the entire cycle of information that repeats itself through these physical rotations. Masaru Yoshida composed the song, Haruka Yoshida was in charge of the camera and editing, and the entire group collaborated to bring the performance to life, with even Wikipedia getting in on the action, with the anniversary event playing a role.
[Source]

At an event at Sony’s TV headquarters in Tokyo last month, we were treated to some one-on-one time with Sony’s upcoming RGB LED-backlit LCD TV, and I can say this TV is clearly something special. We got to see the new set, which Sony is calling “True RGB,” in its final form and with its LCD panel and screen removed, exposing the RGB backlight unit. Next to it was Sony’s current Mini LED flagship TV, the BRAVIA 9, also in complete form and also with its LCD panel and screen removed, exposing the Mini LED back light unit for comparison.

Compared to the BRAVIA 9, the True RGB TV exceeded the performance of that set in just about every measurable (and subjective) way, with wider color gamut, impressive peak brightness and freedom from artifacts like aliasing and color banding. It also had black levels and contrast that will give an OLED TV a run for its money. The new set offered excellent off-axis viewing with minimal dimming and color shift when viewing it from the sides. The upcoming set, which will be publicly unveiled later this spring, does all this while actually using less power than its predecessor, thanks to highly efficient power management and precise control over its RGB backlighting system.

Mini LED/LCD TVs like the BRAVIA 9 have a relatively easy job when it comes to color reproduction. The blue LED elements combine with a quantum dot layer to generate a white backlight. Each pixel on the LCD panel itself creates colors by adjusting the opaqueness of each LCD pixel’s red, green and blue subpixel. Because the backlight is uniform in color, the color filter process is entirely predictable and uniform from pixel to pixel.

With an RGB backlit TV, the image processor has to decide how to adjust both the intensity of each individual red, green and blue LED diode in each zone of the backlight unit and do further adjustment at the pixel level adjusting each of the red, green and blue LCD subpixels in order to create each pixel’s final color. This two-step process can lead to more accurate and more vivid color reproduction, wider color gamut and higher overall brightness, but at the expense of requiring more processing power. It is just this complexity that has led to Sony taking its time in releasing its first RGB-lit TV of the new era.

At Sony’s headquarters, we got to see the new True RGB set up against several RGB-backlit models from competitors. In this comparison, the Sony True RGB set was better able to remain in its true RGB backlighting mode, taking full advantage of its wide color gamut reproduction with independent control over its red, green and blue diodes, while at least one competitive model switched to a full white backlight whenever multiple contrasting colors were displayed on the screen concurrently. This caused the competing set to lose its RGB color advantage by reverting to a uniform white backlight. And this was evident in visible loss of color saturation.

We’ve seen some RGB backlit TVs struggle with reproduction of multiple colors onscreen at the same time, due to a condition called “color crosstalk.” This occurs when you have multiple colors on screen at a time, or white objects next to or surrounded by colored backgrounds. Some of that background color can bleed into the white due to less than perfect backlight and color filter management. The Sony True RGB set we saw in Japan exhibited none of these color crosstalk issues or color bleed.

Off-axis viewing and glare reduction were both exceptionally good on the True RGB TV, with the new TV able to maintain its colors and rich black levels, even when viewed from the sides in a fairly bright room. While there was occasionally some mild blooming on brightly colored images set against a black background, the use of RGB lighting elements made these faint artifacts nearly imperceptible. On traditional LED/LCD TVs, the bloom or halo around a bright object is typically white, while on a True RGB TV, the light bloom matches the color of the on-screen object, making it much less noticeable.

We viewed several challenging 4K/HDR clips highlighting HDR tone mapping and found that the new True RGB set outperformed the BRAVIA 9 MiniLED TV in both specular highlights and shadow detail. And the BRAVIA 9 is already a strong performer for tone mapping, so this was a pretty impressive feat.

The True RGB TV we spent time with in Japan was a 65-inch version, but, because these TVs use standard LCD “mother glass,” we can expect Sony’s True RGB tech to be available in much larger screen sizes. Certainly larger than OLED TVs which currently max out at 97 inches diagonally. More details will follow later this spring.

The Bottom Line

While Sony was the first TV maker to use RGB LED backlights in an LCD TV, with the Qualia 005 from 2004, they were not first to market with this new wave of RGB-backlit LCD TVs. Models from Samsung, TCL and Hisense were introduced last year, and second generation models are coming soon from these same manufacturers. LG also unveiled their own RGB-backlit LCD TVs this year, though they are still standing behind OLED technology for their flagship TVs.

Sony has been working on perfecting RGB backlighting in LCD TVs for several years. About a year ago, we saw Sony’s then current prototype RGB backlit TV, which was impressive, but this latest version is even more so. From what we can gather, the company wanted to make sure their version of RGB backlighting was truly ready for prime time before its release. And, from what we’ve seen so far, the wait will be worth it.

Stay tuned to eCoustics for more details on Sony True RGB TVs, including industrial design, model numbers, screen sizes, prices and more, coming later this spring.

Related Reading:

We’ve Seen the Future of Sony TVs in All its Red, Green and Blue Glory: Here’s What’s Coming

It’s Here! Check Out the 116-inch Hisense 116UX RGB Backlit TV

Sony, TCL Finalize Joint Venture for TVs and Audio; Say Hello to BRAVIA, Inc.

Arcee, a tiny 26-person U.S. startup that built a massive, 400B-parameter open source LLM on a $20 million shoestring budget, has released its new reasoning model. Arcee calls the model Trinity Large Thinking — and it’s the most capable open-weight model “ever released by a non-Chinese company,” claims CEO Mark McQuade to TechCrunch.

As that comment implies, Arcee has a goal that I can’t help but root for: It wants to give U.S. and Western companies a model that gives them no reason to use a Chinese-based one.

While Chinese models are extremely capable, they are perceived as risky, putting power, and perhaps data, into the hands of a government that doesn’t share all of the Western world’s ideals.

With Arcee, companies can download the model, train it to their own needs, and use it on premises. Companies can also use Arcee’s cloud-hosted version, accessible via API.

While Arcee’s models are not outperforming the closed source models from the big labs like Anthropic or OpenAI, they’re not being held hostage by the whims of those giants, either.

For instance, Claude, with its exceptional abilities to code, has been a popular choice for users of open source AI agent tool OpenClaw. But Anthropic pulled the rug out from them last week when it told users that their Anthropic subscriptions will no longer cover OpenClaw usage — they will have to pay additionally for that. (In February, OpenClaw creator Peter Steinberger said he was joining Anthropic’s biggest rival, OpenAI.)

In contrast, McQuade proudly points to data from OpenRouter that says it has become one of the top models used with OpenClaw.

So, how good is Trinity Large Thinking? It is comparable to some of the other top open source models, according to the benchmark results it shared with TechCrunch.

As we previously reported, it is not a head-to-head threat to the big cheese among U.S.-built open models: Meta’s Llama 4. But it also doesn’t have the odd, not-really open source license issues of Meta’s model. All of Arcee’s Trinity models are released under the gold standard for OS licenses, Apache 2.0.

Just to be clear, there are also countless other U.S. startups offering open source models and, as a fan of the ingenuity of startups, I’m rooting for them, too.