01First
Hypotheses on record
Four claims, stated and time-stamped before collection began. What would count as failure is written down too.
Research / pre-registered production studyLive · GDPR-anonymised
Mne
mo.
AI claims your auditors can verify — pre-registered, in production.
Most AI vendors show you a benchmark; mnemo shows you a method. The Memento-Skills learning loop is being replicated under real multi-tenant production traffic, with four hypotheses locked before the first byte of data flowed. When the study ends, the full replication package opens — so the results you buy on are results anyone can check.
Hypotheses locked before data4
Days, fixed in advance28
Evolving personas15
LLM providers under test8
GDPR-anonymised telemetry100 %
Replication package at study end1
Method / pre-registration
Locked before
the data flows.
Pre-registration removes the vendor’s favourite trick: deciding what success means after seeing the results. Here the questions, metrics and analysis were fixed first — publicly.
02Real traffic
Production, not a lab
The study runs on live multi-tenant workloads — the messy conditions your deployment will actually face.
03Anonymised
GDPR from the first byte
Telemetry is anonymised at the source. Evidence accumulates; personal data does not.
04Eight providers
No favourite model
Eight LLM providers under the same protocol. Results describe the method, not one vendor’s good week.
05Fifteen personas
Skills that evolve on record
Fifteen personas learn and change during the study — the growth curve is the measurement, not an anecdote.
06At the end
The replication package
Protocol, anonymised data and analysis code open at study end. Reproduce it, or hire someone who will.
The protocol
Four steps, in the open.
Register
Hypotheses, metrics and window locked and time-stamped.
Run
28 days of live production traffic, anonymised at source.
Analyse
Only the pre-registered analysis — no fishing expeditions.
Open
Replication package published: protocol, data, code.
What this buys you
Evidence, not adjectives.
Enterprise AI purchases fail on unverifiable promises. A pre-registered study is a promise with a receipt.
01
No moving goalposts.
Success criteria were fixed in advance — results cannot be reframed after the fact.
02
Procurement-ready.
A locked protocol and open package give due-diligence teams something to actually diligence.
03
Negative results included.
Hypotheses that fail are published with the ones that hold. That is what makes the ones that hold worth something.
04
Independently checkable.
Your data scientists can rerun the analysis themselves — no trust in the author required.
Not a benchmark.
Benchmarks measure a model on a Tuesday. This measures a method over 28 days of real work.
Not a demo dataset.
Live traffic, real tenants, genuine noise. The conditions are the point.
Not marketing research.
The protocol allows the study to fail publicly. Marketing departments do not sign up for that.
Audit the claims.
The study is live and the protocol is public. Briefings on method and interim status go through the main page.