When AI says: “I checked”

A field test with five AI tools shows why live access, understanding of content and reliable judgement are three different things.

When a machine says it has checked, that has to be provable.

There is a question whose answer has become surprisingly hard in the age of language-based AI systems. The question is: what does it actually mean when a system declares it has checked a web page "live"?

Consider the underlying principle first. A web page is no abstract thing. At a given moment it has specific content, made up of headings, texts, images, metadata and technical structures. If you retrieve that content directly, you can check it in the way described. If a search index, a cache or older account data is used, an earlier version may appear instead. Both processes can be useful. But it is important to stress that these are different things.

This investigation examined that distinction using dimension-fx.com. The previous site, with a local text-to-speech tool, was replaced at the end of July 2026 by a new page about DFX SplatCore, a local application for 3D Gaussian splatting. The 3rd of August was defined as day one of the measurement series. The question was therefore not whether AI systems are intelligent in general. It was posed more precisely and more testably: which systems actually reproduce the current content of this specific website, and which confuse older or foreign information with a direct retrieval?

The method

The same prompt was used for several AI chat tools. First, an honest self-assessment of the kind of access was required: direct retrieval, search index, cache or training knowledge. The systems then had to quote verbatim the main heading, the subtitle immediately following it, and one question from the FAQ section. Finally they were to classify the content they found in terms of product and currency.

This approach is useful because statements about the content can be checked directly. The real page carried the main heading "Deterministic 3D digitisation", the subtitle "From photo and video to a web-ready 3D experience", and among other things the FAQ question "What is DFX SplatCore?". An answer that instead spoke of the "TTS Constructor" and of "orchestration instead of typing" was not describing the current page but an earlier state of the site.

The procedure resembles a scientific measurement. What counts is not the look of the instrument but obtaining the same verifiable value repeatedly. The verbatim quotations serve as a calibration sample: they make it possible to separate the claim of live access from actual knowledge of the content.

First finding

The result of the first round was mixed. Mistral and Claude.ai delivered the required elements of the current page correctly. ChatGPT, by contrast, stated openly that it could not reliably retrieve the page, and avoided replacing missing information with guesses. Gemini explained its lack of access with an alleged "Google-Extended opt-out", although the robots.txt as checked contained no such block. Perplexity initially showed older page content rather than the current live state. On further questioning it emerged that the search index and a previously attached file were the basis.

This leads to a first, important distinction. A system can fail at retrieval and still act reliably, if it names its limit. Another system can produce a detailed answer and still be unreliable, if the wealth of detail comes from a wrong or outdated source. Linguistic fluency is therefore no indicator of currency. It is merely a property of the presentation.

Second finding

The second test produced an even sharper insight. Mistral had quoted current content correctly in the first round. But in a comprehensive SEO, GEO and technical assessment of the same website, the system produced numerous claims of defects that proved false on local inspection. Of the twelve statements checked, two held: some meta descriptions were too long, and individual FAQ questions had no anchor IDs of their own.

The remaining alleged deficits — missing hreflang references, canonical URLs, Open Graph tags, JSON-LD, favicons, alt attributes and main headings — were mostly or entirely present in the local project. This does not mean that live retrieval is worthless. It has a more precise meaning: the ability to reach a page is no proof of the ability to analyse it correctly. Access and judgement are two different measures.

What that means

At this point a general property of AI-assisted work becomes clear. A model can retrieve data without checking its structure sufficiently. It is quite possible to give well-founded recommendations without knowing the individual case fully. It is even possible for a sensible technical proposal to meet only part of the actual aim. In this project that showed with the use of aria-hidden="true" on icon ligatures: the measure improved accessibility for screen readers but did not remove the ligature names from the actual document text that search systems and text-extracting systems can read.

This is no exotic exception but an interesting variable in every technical development. A check can be executed correctly and still be scoped too narrowly. A measurement can be highly precise, but only with respect to what was measured. A visible improvement can leave an invisible error untouched. In AI-assisted development, then, what matters is not which system answers most convincingly. What matters is that a procedure links claims reproducibly to files, line numbers, rendered results and counter-checks.

---

The machine as an instrument of measurement

The history of technology teaches a simple rule. A new tool is judged first by its visible performance. Only later comes the art of reflecting on oneself and one’s limits.

The steam engine could do its work before all the losses in boiler, valves and transmission were precisely calculated. Electricity was used early to light streets before the grids were reliably protected against overload, voltage drops and failures. In both cases progress did not lie in the machines becoming faultless. It lay in people learning to recognise errors systematically, to measure them and to build them into the design.

AI-assisted tools are in a comparable phase today. Their strengths include explaining texts, producing code, evaluating web pages and quickly ordering large amounts of information. That is a realistic and useful perspective. The output of a language model is no direct measurement of reality. It is, first of all, a claim about reality. The validity of that claim is not settled by linguistic sharpness, but by the possibility of checking it.

Two kinds of error

The test series identified at least two different kinds of error.

The first arises when a system does not reliably reach the current object. It may fall back on a search index, a cache, training knowledge or earlier conversation data. The result can sound plausible and still describe an older version of reality. In the case of dimension-fx.com, the old TTS Constructor content was technically not entirely invented. It is part of an earlier state of the project. Asked about the current website, it was the wrong answer.

The second kind is subtler. The system does have access to the object, but neither checks it thoroughly enough nor records it. It may capture a sitemap, list all subpages, and then output a general list of SEO defects that would fit many small business sites. The list can contain detailed grades, priorities and schedules. But a detailed form is not necessarily evidence of a careful investigation.

A useful formula can be set out:

Image created with AI

Access is not understanding. Understanding is not judgement.

An AI can reach a website without being able to read its content correctly. It can quote the content correctly without deriving a correct technical analysis from it. It can make a recommendation whose technical grounding is incomplete.

What the counter-check is worth

The decisive improvement therefore does not lie in a single model version. The method used for the analysis breaks every essential claim down into separate, checkable parts.

A statement such as "canonical URLs are missing" should not be treated as a final verdict. It is decisive that the question be put precisely. Could you tell me in which files the elements in question are missing? How many files were checked? Could you tell me which URL appears in each link rel=canonical element? Are there justified exceptions? Only then does a plausible claim become a technical finding.

It is not enough merely to establish that an HTML element exists in the source. What is decisive is how the document is generated. That became particularly clear with the icon spans carrying ligature names. Headings were present in the HTML. In the browser text, however, expressions such as "emoji_objectsWhy this work exists" appeared. Counting in the source therefore formally confirms the heading exists, but overlooks whether it is actually readable for text-extracting systems.

This is no small matter. HTML is the blueprint. The rendered text is the building in which users, screen readers, search engines and other systems actually work. Checking the blueprint alone can mean missing essential properties of the finished building.

Why automation alone is not enough

Scripts are made to carry out precisely defined tasks repeatably. Their tasks include counting files, checking attributes, comparing string lengths, finding patterns and logging deviations. Their strength is persistence. A script is consistent, it overlooks no file and does not change its criteria on a whim.

But every script carries an implicit assumption: that the query was formulated completely enough.

The first count of icon spans produced 250 hits. A later check produced 394. The difference arose not from an unreliable machine but from a search pattern that was too narrow. A CSS class variant with an additional class was not captured. The result was precise, but not complete.

This insight applies to all automated checks. How meaningful a number is depends on how precisely the thing being counted was defined. Automated processes therefore do not replace judgement by people. They shift it to an earlier point: from carrying out a task to deciding which task has to be carried out at all.

The economic question

The economic consideration matters here. Reducing computing time and cost by using AI tools is a reasonable approach. Not every task requires a large model or an extensive analysis. Generating identifiers, walking through a file inventory or replacing clearly defined strings are mechanical operations. If the rules are right, a smaller model or a local script can prepare or accompany that work reliably.

For judgement-dependent tasks a different approach is needed. The relevant questions are, for instance: which check is missing, does an explanation describe the actual effect, or does an apparently clean finding contain a blind spot? In such cases the cheapest solution is rarely the right one.

Because an initial saving can create rework. That can lead to additional costs for finding and fixing errors. A particularly serious aspect is the loss of trust in your own interim results. If an analysis has to be fully checked again, a significant part of its economic value is gone.

The sober rule is therefore not: always use the biggest model. It is: "Automate mechanical work precisely, but treat the conceptually difficult questions with an appropriate depth of checking."

Local tools, provable methods

This is where the difference between a local tool and a remote service becomes particularly clear.

That a local tool runs on your own computer is not sufficient reason to prefer it. It may contain errors, apply incomplete rules or suggest wrong conclusions. Its essential advantage is that its data paths, inputs, intermediate steps and results can in principle be followed in full.

In a local check it can be established which file was used at the moment in question. The number of elements a script changed can be determined by counting the 394. If a result seems implausible, the process can be run again — changing the rule and watching the difference.

Where an AI service runs remotely, gaps can appear in the visibility of the chain. The user sees the answer, perhaps a few source references and occasionally a summary of the working steps. But it often stays unclear whether the URL given was actually retrieved live, which cache layer was involved, whether earlier chat attachments played a part, or which parts of the page were read at all.

This is no moral charge against cloud services. It is a technical property of their architecture. When processing happens outside your own system, the demands on checking rise. For creative drafts or first research that may be acceptable. For robust business processes, technical audits and production-relevant decisions, traceability is central.

The role of people in this

The right role for people is not to execute every line themselves. That would waste the abilities of both sides.

The machine is excellent at comparing variants, repeating routine checks, taking stock of large holdings and identifying conspicuous places. People stay responsible for choosing the question, weighing conflicting aims and deciding whether a measured difference matters at all.

An empty alt field illustrates this well. A superficial check can mark it as missing information. In some cases using alt="" on purely decorative images is the right decision, because screen readers then skip the element. The machine can identify every empty attribute. Classifying it as an error, an intention or correct accessibility requires a look at the specific context.

The aim is therefore not human control over every single working step. What matters is human control over their meaning. The AI does the searching, counting and pre-structuring. People check the assumptions, judge the results and take the decision.

A robust checking protocol

A simple checking protocol can be derived from the findings so far.

First, the source has to be established unambiguously. Note whether an analysis works against the live domain, a search index, a cache, a project folder or an earlier conversation attachment. This distinction is no formality. It decides which evidence is sufficient for an answer.

Then comes taking stock. Before any assessment, the entire relevant holding is inventoried: number of files, language versions, routes, meta information, headings, images, structured data and external dependencies. An assessment without an inventory can be accurate, but it stays a sample as long as the scope is unknown.

Next, the presentation layer has to be checked. Source code, browser DOM, visible rendering, accessibility tree and extracted text can all give different results. For projects meant to be accessible to search engines, AI systems and people alike, it is essential to consider more than one of these layers.

Finally, every important finding requires a counter-check. That can be a second script, a different checking path, another model or a manual sample. What matters is not who contradicts. What matters is whether the contradiction leads to a more precise measurement.

And so to the real question:

The investigation began with a simple website and a clear instruction: "Open this page and describe the content." It led to a larger question.

Our aim is to build technical systems that are not only fast and convincing, but also checkable.

The answer will in all likelihood not consist in avoiding AI tools. Nor will it be possible to accept every generated analysis. The productive path lies between the two extremes: use AI as a powerful engine, but set it inside an architecture that knows its sources, logs its steps, measures its results and allows contradiction.

The AI then does not become an oracle. It becomes a tool whose performance can be understood, bounded and improved.

---