Table of contents
When you ask a software vendor whether their system has an audit trail, the answer is almost always yes. When you ask a compliance officer whether that audit trail would hold up in an investigation, the answer is often no.
Both answers can be true at once. They describe the same product against two different bars: what helps an engineer debug, and what survives a regulator’s scrutiny.
Application logs are built for engineers. Compliance audit trails are built for regulators. They capture overlapping events, but they are designed for different audiences, different retention requirements, and different threat models. Treating one as if it were the other is the most common audit trail mistake regulated organizations make when they deploy AI.
Application logs and audit trails are not the same thing
Application logs are produced by software to help developers understand what the system is doing. They typically capture errors, performance metrics, API calls, and system events. They are written by the system itself, stored in infrastructure the system operator controls, and designed to be searched and deleted as needed.
An audit trail, in the regulatory sense, is a sequential record of events that can be used to reconstruct what happened, in what order, and who was responsible.[1] The key properties that distinguish it from application logs are:
| | Application logs | Compliance audit trail | | ---------------- | ---------------------------------------- | ----------------------------------------------------- | | Purpose | Help developers diagnose system behavior | Create a verifiable record for regulatory review | | Integrity | Can be modified or deleted by operators | Tamper-evident; alteration is detectable | | Completeness | Captures what the developer chose to log | Captures every relevant event by requirement | | Attribution | Tied to sessions or system processes | Tied to specific user identities and actions | | Retention | Typically 30–90 days; deleted as needed | Years, governed by regulation — 6 minimum under HIPAA |
Integrity. The record cannot be modified without detection. This typically means tamper-evident storage: hashing, digital signatures, or append-only write paths that make alteration visible.
Completeness. The record captures every relevant event, not just the events the developer decided to log. For AI systems, this means every prompt, every model response, every access event, and every configuration change.
Attribution. Every event is tied to a specific user identity, not just a session or IP address. “User ID 4821” is not sufficient. The audit trail needs to be able to answer “which licensed clinician saw this information, at what time, and what action did they take.”
Retention. The record is kept for a period that matches the applicable regulatory requirement. HIPAA requires covered entities to retain security-relevant documentation for at least six years from creation or last effective date.[2] Six is the floor, not a target — HASP retains the audit chain for seven years across every paid tier. Application logs, by contrast, are routinely deleted after 30 to 90 days.
Why AI makes the audit trail harder
An AI system is harder to audit than the database application it sits next to.
With a traditional database application, the audit trail captures reads, writes, and deletes on structured records. The events are discrete and well-defined. The schema of what happened is clear.
With an AI system, the “what happened” is harder to define. The relevant events include the prompt (what the user asked), the context injected into the prompt (what additional data the system added), the model response (what the AI said), and any downstream actions the AI took or recommended. None of these map neatly to a traditional database operation.
The prompt is particularly important. A prompt that contains PHI is itself a regulated record. If your AI system logs the prompt for debugging but doesn’t treat that log as PHI under HIPAA, you have a compliance gap - the PHI is in a system without the appropriate technical safeguards.
The impact of failing to monitor audit logs can be massive. In a landmark 2017 settlement, Memorial Healthcare System paid $5.5 million after it was discovered that a former employee’s credentials remained active for a year, allowing unauthorized access to 80,000 patient records that went undetected because audit logs were not reviewed.[3]
That risk scales with the sensitivity of the use case. An AI system handling clinical notes, diagnosis codes, or treatment decisions carries far more exposure than one handling general business communications.
What a compliance-grade AI audit trail is made of
A compliance-grade audit trail for an AI system has a few consistent elements.
Signed entries. Each log entry is cryptographically signed at write time. The signature covers the content of the entry and is produced by a key the logging system controls. If someone modifies the entry later, the signature fails verification.
A hash chain. Each entry includes the hash of the previous entry. This creates a chain where tampering with any single entry breaks the chain from that point forward - making selective modification visible even if individual signatures are somehow forged.
A trusted timestamp. Timestamps from the application server can be manipulated by anyone with access to the server. A compliance-grade timestamp comes from a third-party Time Stamp Authority (TSA) under RFC 3161 - an external anchor that provides a legally robust, non-repudiable record of when an event occurred.[4]
Export and verification capability. The audit trail is not useful if only the vendor can read it. A compliance officer or auditor needs to be able to export a subset of records and independently verify their integrity — without trusting the vendor’s own tooling. The verification recipe should be documented and runnable with standard tools.
Appropriate access controls. The audit trail itself needs to be access-controlled. Audit logs that any administrator can delete are not tamper-evident in any meaningful sense.
Ten questions to ask any AI vendor about its audit trail
When a vendor says “we have a complete audit trail,” these are the ten questions that separate a real one from a log file with a nice export button. Each is worth asking verbatim — they are written to be pasted into a security questionnaire or an RFP.
We have answered each one for HASP inline, because a checklist a vendor won’t answer about its own product isn’t a checklist, it’s marketing. Every answer below is true of the audit chain running in production today, and the last question tells you how to check the answers rather than believe them.
1. What events are captured, and can any request avoid being logged? The interesting half of this question is the second half. Ask where logging happens: if it’s a call the application code makes after the fact, someone can forget to make it. If it’s structurally part of the path a request already takes, it can’t be skipped.
HASP: every AI request runs through a single gateway, and writing the audit entry is part of that path — a prompt cannot reach a model without producing a record. Prompts, model responses, access events, configuration changes, and policy decisions (including denials) all land on the chain. There is no unlogged path and no per-organization setting that turns logging off.
2. Are prompts and responses recorded, or just summarized? “We log API calls” often means a row with a timestamp, a user ID, and an endpoint. That tells an investigator that something happened, not what.
HASP: the content of the request and the response is recorded, not a summary. Protected health information is deliberately not persisted into audit storage — the entry carries a redacted form plus metadata describing what kind of information was present, so the record is investigable without turning the audit log into a second copy of your PHI.
3. What prevents an administrator from deleting or modifying an entry? Ask whether the answer is a technical control or a policy control. “Our staff aren’t permitted to” is a policy control. Policy controls are not auditable; technical controls are.
HASP: the database itself refuses. Update and delete on a sealed audit row are blocked by database-level triggers, so it fails regardless of how the query is issued — through the application, through a console, through direct SQL. That’s the first layer; the next three questions cover what happens if someone gets underneath it.
4. Is every entry signed at the moment it’s written, or signed in bulk afterwards? A log signed nightly can be edited during the day. Signing at write time is what makes each entry individually accountable.
HASP: each entry is signed with an Ed25519 signature as it is written, and each entry also carries a fingerprint of the one before it. Altering any entry breaks its own signature and every fingerprint after it — you can see exactly where the break is.
5. Whose key signs it, and where is the public half published? This one catches a subtle forgery: a vendor can hand you an export that verifies perfectly against a key embedded in that same export. The check only means something if the key was published somewhere independent, before you asked.
HASP: each organization gets its own signing key, and the public half is published at a fixed public URL, separate from any export. Verification compares the two — an export signed with a key minted for the occasion fails.
6. Is the log anchored to an independent third party, and how often? Signatures prove who wrote a record. They do not prove when, because whoever holds the key can produce a correctly signed record at any later date. Only an outside party can fix a record in time.
HASP: the chain’s state is anchored daily to an independent RFC 3161 timestamp authority. The authority countersigns the state of the log at that moment, which means history up to that point cannot be rewritten or backdated afterward — not by us either.
7. What’s the retention period, and who pays for more? Match this against your own obligation before you look at the price.
HASP: seven years on every paid plan, included — not an add-on and not a tier upgrade. Enterprise agreements can extend it further.
8. Does the audit trail cover every part of the product, or just the flagship? Vendors frequently ship a strong audit trail on the main application and a thin one on the API, the integrations, or the automation layer — which is exactly where unattended activity happens.
HASP: one chain covers everything — chat, the app builder, the public API, and agent activity. An action taken by an autonomous agent is recorded the same way an action taken by a person is, including who delegated the authority for it.
9. Can you export your records, and can someone verify that export without the vendor’s software? Independence is the whole point. If verification requires a tool only the vendor can run, you’ve replaced trusting their log with trusting their tool.
HASP: exports are self-contained and verifiable with an open-source command-line tool, or by hand using nothing but standard tools — Python, openssl, jq, curl. No HASP code anywhere in the loop, and no HASP account required to check the published sample.
10. Can we verify a sample right now, on this call? The last question is the one that separates the answers from the evidence. Any vendor can answer questions one through nine well. Ask them to prove it on a real export while you’re watching.
HASP: yes — there’s a real export published on this site, and a one-command check that validates it end to end. Walk through it here; the whole thing takes about two minutes.
A vendor that answers all ten with specifics is worth a closer look. One that falls back on “we log everything” is not — and one that answers the first nine but has nothing to show for the tenth has told you which kind of audit trail it is.
Where this leaves a regulated team
For a healthcare organization, a legal team, or any regulated business deploying AI, the audit trail is not a nice-to-have. HIPAA’s Audit Controls standard is a required implementation specification under the Security Rule.[2] “We have logs” does not satisfy it. “We have tamper-evident, complete, access-controlled logs retained for six years with a documented verification procedure” does.
Compliant audit infrastructure is a solved engineering problem — the open question is whether your AI vendor has built it or left it to you. HASP’s audit architecture is documented for exactly this kind of evaluation.
Sources
National Institute of Standards and Technology. SP 800-92: Guide to Computer Security Log Management (2006).
csrc.nist.gov
U.S. Department of Health & Human Services. “HIPAA Security Rule: Administrative, Physical, and Technical Safeguards.” HHS.gov.
hhs.gov
U.S. Department of Health & Human Services OCR. “HHS Reaches $5.5 Million Settlement with Memorial Healthcare System” (2017).
hhs.gov
DigiCert. “What is an RFC 3161 Time-Stamp?” (2024). Technical guide on digital signature integrity.
digicert.com