AI in Auditing: PCAOB Rules, Bias, and the Black Box Problem

AI in auditing now covers four concrete jobs: spotting anomalies across every transaction in a ledger, reading unstructured documents at scale, matching records between systems, and scoring risk continuously rather than annually. The PCAOB finalized amendments to its core auditing standards in 2024 to address how auditors should handle this kind of technology-assisted analysis, and those amendments apply to audits of financial statements for fiscal years beginning on or after December 15, 2025.¹ The tools are in production. The rules governing them are catching up.

What AI Actually Does in an Audit

The applications fall into four buckets, each aimed at a specific pain point in traditional audit work.

Anomaly Detection Across Full Populations

Machine learning algorithms read entire ledgers and flag transactions that deviate statistically from normal patterns. A traditional audit selects a sample and extrapolates. AI examines every recorded transaction and surfaces every statistical outlier, so the risk of missing a problem because it fell outside the sample effectively disappears.² Flagged items may be errors, fraud, or unusual but legitimate business events. The algorithm surfaces them; the auditor still decides what they mean.³

Natural Language Processing for Document Review

Contracts, board minutes, legal correspondence, and regulatory filings sit in unstructured text that a human team can only skim. NLP tools read this material at scale, extracting things like renewal options, contingent liability language, and unusual termination clauses from thousands of documents at once. Research into NLP-assisted auditing has found that automated systems complete verification and extraction tasks roughly seven times faster than a human auditor working the same documents.⁴ The auditor’s time shifts from finding clauses to interpreting what they mean.

Automated Reconciliation and Data Matching

Matching general ledger entries to bank statements, vendor invoices, and customer receipts is foundational audit work. AI handles this across incompatible systems at volumes manual review cannot approach, flagging discrepancies for follow-up. The catch: automated matching is only as reliable as the data feeding it. Incomplete extracts, formatting inconsistencies, or misconfigured matching logic can produce false comfort. Confirming that the data is complete and configured correctly is part of the job, not an assumption.

Continuous Risk Scoring

Traditional risk assessment leans on prior-year findings and the engagement team’s institutional knowledge. AI models integrate real-time inputs including market conditions, news sentiment, regulatory developments, and internal control metrics to produce risk scores at the account, transaction, or business-unit level. Risk assessment becomes a continuously updated picture rather than a static annual exercise, which changes how audit hours get allocated. Models need ongoing calibration, though; without it, they lock in historical patterns and reproduce whatever biases were already there.

Related to this, some AI tools monitor client data streams continuously, flagging control failures within hours or days instead of at year-end. That level of coverage requires direct integration with the client’s systems, which not every client is willing or technically equipped to provide.

Full Population Testing and Why It Doesn’t Make Auditing Easier

The single biggest methodological change AI enables is the move from sampling to testing 100% of a population. PCAOB AS 2315 defines audit sampling as applying a procedure to less than 100% of items in an account balance, and the standard itself acknowledges that auditors accept sampling risk because examining all of the data was historically too costly and slow.⁵

AI removes that constraint. When you can process every transaction, the risk that your sample missed something goes away.² But the auditor’s work doesn’t shrink; it moves. The focus shifts from designing a sampling methodology to confirming that the AI ingested a complete and accurate data set. Full population testing on an extract missing a month of transactions or an excluded subsidiary produces 100% coverage of the wrong picture.

The PCAOB’s 2024 amendments to AS 2301 speak to this. When a test of details identifies items requiring further investigation, follow-up procedures must determine whether those items indicate misstatements or internal control deficiencies. And even after testing specific items, the auditor must assess whether the remaining untested items could contain a material misstatement, performing additional procedures if that possibility exists.⁶

The PCAOB Rules Now in Effect

The PCAOB’s amendments target two standards: AS 1105 (Audit Evidence) and AS 2301 (The Auditor’s Responses to the Risks of Material Misstatement). They apply to audits of financial statements for fiscal years beginning on or after December 15, 2025, which means they are in effect for calendar-year 2026 audits.¹ The concern behind them is that technology makes it easy to produce impressive-looking outputs, and impressive outputs are not automatically reliable evidence.

AS 1105: Information From External Sources

The amended AS 1105 added paragraph .10A, which governs what happens when a client provides the auditor with electronic information the client originally received from external sources. Before using that information as audit evidence, the auditor must understand where the data came from, how the client received and maintained it, and whether the client modified it before handing it over. The auditor must then either test whether the data was altered and evaluate the effect, or test the controls the client has over receiving, maintaining, and processing the information.⁷

Release 2025-004 provides limited relief. If the auditor’s risk assessment and initial procedures indicate no more than a remote possibility that the electronic information was modified in a way that would make it unreliable, the auditor will not be cited through inspection or enforcement for skipping the separate testing requirements under paragraph .10A(b). This is a risk-based concession, not a blanket exemption. The auditor still needs a documented basis for concluding the modification risk is remote.

AS 2301: Multi-Purpose Procedures

The amendments to AS 2301 clarify what auditors must do when technology-assisted analysis flags items for investigation. When a test of details identifies exceptions, the auditor must determine whether those items indicate misstatements requiring evaluation under AS 2810 or deficiencies in internal control over financial reporting. The standard also requires that when an auditor uses a procedure for multiple purposes, each objective must be independently achieved.⁶

The reason this matters: technology makes it tempting to run one broad analytical routine and treat it as satisfying multiple objectives at once. The PCAOB is saying a single procedure can serve multiple purposes, but each purpose has to actually be achieved. Running the routine doesn’t automatically check every box.

The Black Box Problem

Complex machine learning models, especially deep neural networks, often reach conclusions through reasoning that resists easy explanation. A model might correctly flag a transaction as suspicious but be unable to articulate why in terms a human can trace and verify. That opacity collides with the auditor’s obligation to understand and explain the basis for every conclusion supporting the opinion.

The auditor signs the opinion, not the AI system. That responsibility cannot be delegated. If a model surfaces a potential material misstatement and the engagement team cannot reconstruct how the finding was produced, they cannot rely on it as sole evidence. “The model said so” is not audit evidence.

The practical response has been movement toward more interpretable models in audit applications. Decision trees and logistic regression sacrifice some predictive power but produce transparent reasoning. Many firms are landing on hybrid workflows: use the more powerful models to identify areas of interest, then apply interpretable methods to validate and explain the findings. The NIST AI Risk Management Framework points in the same direction, calling for organizations to establish transparency policies documenting how AI systems reach their outputs and to set minimum performance thresholds as part of deployment approval.⁸ Data lineage tracking, which documents the origin, movement, and transformation of every data point, is essential to that traceability; ISO/IEC 38505-1 provides a governance framework for the use of data within organizations that has explicit relevance to auditors.⁹

Bias in AI Audit Models

A model trained on historical data reproduces whatever patterns exist in that history, including patterns reflecting past human errors or systemic blind spots. In audit work, this can show up as risk-scoring models that consistently over-flag certain transaction types or business units while under-flagging others, not because of genuine risk differences but because the training data reflected skewed historical attention. The audit plan looks data-driven while quietly perpetuating old assumptions.

Fairness auditing has developed specific quantitative methods for detecting this. Demographic parity measures whether the model produces similar outcomes across different groups. Equalized odds tests whether error rates are consistent regardless of group membership. Accuracy equality checks whether the model is equally accurate across subpopulations. These give the engagement team a measurable basis for judging whether outputs are skewed, rather than relying on intuition.

Models also drift. New data enters the system, patterns shift, and a model that tested clean at deployment can develop bias months later. Periodic recalibration and monitoring are essential, and both the testing performed and the results obtained need to sit in the workpapers.

Data Privacy, Vendors, and Insurance Gaps

AI-driven auditing processes large volumes of sensitive data, including personally identifiable information and proprietary business records. Because AI tools may ingest entire databases rather than selected samples, the consequences of any security failure scale accordingly. Firms handling data subject to the GDPR or the California Consumer Privacy Act face specific obligations around collection, processing, storage, and disposal.

The supply chain is where a lot of firms carry hidden risk. Most audit firms don’t build AI tools from scratch; they license platforms from technology vendors or integrate third-party models into their workflows. Standard SOC 2 reports and general risk questionnaires often lack the specificity to assess how a vendor uses AI, what data the model relies on, or whether adequate controls exist around bias mitigation and data lineage. AI-specific due diligence needs to push vendors to demonstrate controls over model development, training data sources, and auditability of outputs. Vendor contracts should require disclosure when AI is used in service delivery and address whether the vendor uses client data to train its own models.

Insurance has not caught up. As of early 2026, insurers are still working out how to classify AI-related incidents, with uncertainty about whether claims fall under professional liability, cyber coverage, or something else. Some insurers have begun treating AI failures as extensions of existing errors-and-omissions risk; others are exploring exclusions for autonomous AI decisions. Firms deploying AI tools without knowing how their professional liability coverage responds to an AI-driven audit failure are carrying risk they may not have priced.

Where Generative AI Fits Right Now

Generative AI is the newest frontier, and the profession is proceeding carefully. PCAOB staff outreach in 2024 found that current integration of generative AI at major audit firms is focused primarily on administrative and research activities, not on planning or performing core audit procedures. Firms acknowledged the potential for broader use while flagging significant limitations and the need for strong supervision.¹⁰

The caution is warranted. Generative AI introduces risks that traditional machine learning does not: data leakage of sensitive information into model training sets, hallucinated outputs that sound authoritative but are factually wrong, bias amplification through generated content, and intellectual property concerns when models produce outputs derived from proprietary training data. Any firm using generative AI near client data needs clear policies restricting the use of public large language models with regulated information.

One application looks genuinely useful. Generative models can create synthetic transaction data sets that preserve the statistical properties of real data without exposing sensitive information. Teams can use synthetic data to train and validate fraud detection models, test anomaly detection systems, and run scenario analyses without the privacy risks of using actual client data. Research on synthetic banking transaction data has shown these methods retain roughly 94% of downstream model performance while passing privacy compliance assessments.¹¹

International Rules That Reach US Firms

The regulatory picture extends beyond the PCAOB. The European Union’s AI Act, in force with a phased implementation schedule, is the most comprehensive AI legislation globally. As of February 2025, general provisions including AI literacy requirements and prohibited practices are already in effect. The rules for high-risk AI systems listed in Annex III, which include AI used for evaluating creditworthiness, take effect on August 2, 2026.¹²

Providers of high-risk AI systems under the EU AI Act must establish a risk management system spanning the AI system’s full lifecycle, conduct data governance to ensure training data is representative and free of errors, maintain technical documentation sufficient for authorities to assess compliance, design systems to enable human oversight, and achieve appropriate levels of accuracy, robustness, and cybersecurity.¹³ Audit firms with European clients or EU-regulated data need to determine whether their systems fall within these classifications.

The IAASB has proposed revisions to ISA 500 (Audit Evidence) to modernize the standard for an environment where both entities and auditors use technology, including automated tools and techniques.¹⁴ The direction matches the PCAOB’s: technology-generated audit evidence must meet the same reliability bar as evidence gathered through traditional procedures, with additional requirements around understanding and documenting how the technology works.

What the Auditor’s Job Looks Like Going Forward

As AI automates the mechanical work of data gathering, reconciliation, and initial pattern recognition, the auditor’s value moves toward judgment. Reviewing a flagged anomaly requires enough industry knowledge and regulatory context to decide whether it represents a misstatement, a control failure, or an unusual but legitimate business event. No model makes that call reliably.

Manual data extraction and tick-and-tie work becomes less central. Understanding how AI models function, knowing when to trust and when to challenge outputs, and applying professional skepticism to technology-generated evidence becomes essential. The AICPA’s Certified Information Technology Professional credential covers data management, data analysis and reporting, IT governance and strategy, and cybersecurity risk management, reflecting the technical grounding that audit professionals increasingly need.¹⁵

Smaller firms face particular pressure. Infrastructure investment, talent acquisition, and ongoing model maintenance create a real resource barrier. Large firms are building proprietary platforms and dedicating teams to AI development; smaller practices may struggle to match those capabilities. The technology gap between large and small firms is likely to widen, with consequences for competition, talent distribution, and the scope of services smaller firms can credibly offer.

1
Public Company Accounting Oversight Board. PCAOB Updates Its Standards To Clarify Auditor Responsibilities When Using Technology-Assisted Analysis
2
ScienceDirect. Audit Data Analytics, Machine Learning, and Full Population Testing
3
Springer Nature Link. Machine Learning for Anomaly Detection in Auditing and Financial Error Detection
4
National College of Ireland. Exploitation of Natural Language Processing for Financial Audits
5
Public Company Accounting Oversight Board. AS 2315 – Audit Sampling
6
Public Company Accounting Oversight Board. Amendments Related to Aspects of Designing and Performing Audit Procedures that Involve Technology-Assisted Analysis of Information in Electronic Form
7
Public Company Accounting Oversight Board. AS 1105 – Audit Evidence
8
NIST. Artificial Intelligence Risk Management Framework: Generative AI Profile
9
ISO. Information Technology – Governance of Data – Part 1: Application of ISO/IEC 38500 to the Governance of Data
10
Public Company Accounting Oversight Board. PCAOB Staff Shares Observations From Outreach on Use of Generative Artificial Intelligence in Audits and Financial Reporting
11
World Journal of Advanced Research and Reviews. Generative AI for Synthetic Data in Banking Transactions: Balancing Utility and Compliance
12
AI Act Service Desk. Timeline for the Implementation of the EU AI Act
13
EU Artificial Intelligence Act. High-Level Summary of the AI Act
14
IAASB. Proposed International Standard on Auditing 500 (Revised) – Audit Evidence
15
AICPA & CIMA. Certified Information Technology Professional (CITP)