What the PCAOB Has Been Telling Auditors About Audit Sampling: Five Years of Inspection Lessons

Audit sampling has been a fundamental part of auditing for decades, but recent Public Company Accounting Oversight Board (PCAOB) inspection reports demonstrate that sampling itself is rarely the root problem.

Instead, PCAOB inspectors have repeatedly found that audit firms often fail to design, execute, document, and evaluate sampling procedures in a manner that produces sufficient appropriate audit evidence. In many inspections, sampling deficiencies were symptoms of broader problems involving risk assessment, testing methodology, data reliability, and professional skepticism.

Over the past five years, inspection findings reveal a clear pattern:

The problem is usually not the sample. The problem is everything surrounding the sample.

For internal auditors, external auditors, government auditors, and quality reviewers, these inspection findings provide valuable lessons that extend well beyond PCAOB-regulated engagements.

Audit Quality Declined Before Beginning to Improve

Following the pandemic, PCAOB inspections showed a significant increase in audit deficiencies.

The PCAOB reported that the percentage of inspected audits containing at least one significant Part I.A deficiency increased from 29% in 2020, to 34% in 2021, to 40% in 2022, and 46% in 2023 before showing meaningful improvement in the 2024 inspection cycle.

Although sampling is only one component of these inspections, many cited deficiencies involved testing approaches that depended upon appropriate sample selection, execution, and evaluation.

Issue 1 — Testing the Wrong Population

One of the recurring inspection observations has been that auditors sometimes begin sampling before determining whether the population itself is complete and accurate.

If the population is incomplete, duplicated, filtered incorrectly, or excludes relevant transactions, even a statistically perfect sample produces unreliable conclusions.

Examples include:

• Incomplete journal-entry populations

• Missing transactions

• Incorrect system extracts

• Duplicate records

• Transactions excluded without justification

• Reports generated with improper parameters

The PCAOB has repeatedly cited firms for failing to sufficiently test the data and reports used during substantive testing. This issue appears across multiple inspection reports and has become one of the most frequently identified deficiencies.

Lesson

Never begin selecting samples until you have validated the completeness and accuracy of the population.

Issue 2 — Information Produced by the Entity (IPE)

Perhaps the fastest-growing inspection issue involves Information Produced by the Entity (IPE).

Auditors increasingly rely on:

• ERP reports

• Exception reports

• Aging schedules

• Inventory listings

• System-generated reconciliations

• Data analytics outputs

• Access listings

The PCAOB continues to find situations where firms relied upon system-generated reports without adequately testing whether those reports were complete and accurate.

Common mistakes include

• Assuming reports are correct

• Never testing report logic

• Ignoring report parameters

• Failing to verify system queries

• Accepting spreadsheets without source validation

Lesson

Sampling is only as reliable as the information from which the sample is selected.

Issue 3 — Sample Sizes Were Not Supported

Many auditors ask:

"How many items should we test?"

PCAOB inspections repeatedly suggest the better question is:

Why did you decide that this sample size was appropriate?

Inspectors frequently challenge engagements where workpapers contain little explanation supporting:

• sample size

• sampling method

• confidence level

• expected deviation rate

• tolerable deviation

• risk assessment

Professional judgment remains acceptable.

Unsupported judgment does not.

Lesson

Document not only what sample size you selected but why it provides sufficient audit evidence.

Issue 4 — Weak Risk Assessments Produce Weak Samples

Sampling begins long before selecting transactions.

It begins with understanding:

• significant risks

• relevant assertions

• fraud risks

• control environment

• materiality

• inherent risk

Recent PCAOB inspections continue to identify deficiencies where firms failed to appropriately respond to identified risks. When risk assessment is weak, the resulting sampling strategy often fails to focus on the areas of greatest audit risk.

Lesson

Sampling should follow risk assessment—not replace it.

Issue 5 — Overreliance on Internal Controls

Control testing frequently determines whether auditors may reduce substantive procedures.

The PCAOB continues finding deficiencies where firms did not obtain sufficient evidence regarding the design or operating effectiveness of controls.

Examples include:

• insufficient attribute testing

• inadequate reperformance

• incomplete walkthroughs

• weak control documentation

• failure to investigate deviations

Lesson

If control testing is weak, substantive testing—and often sample sizes—must increase accordingly.

Issue 6 — Failure to Investigate Exceptions

Finding an exception is only the beginning.

Inspectors continue identifying situations where firms:

• accepted management explanations

• failed to investigate root cause

• assumed isolated errors

• ignored patterns

• failed to expand testing

Professional skepticism requires auditors to determine:

• Was this random?

• Is the population affected?

• Is fraud possible?

• Does additional testing become necessary?

Lesson

One exception often raises more audit questions than it answers.

Issue 7 — Inadequate Professional Skepticism

Perhaps the most consistent PCAOB message over the last five years concerns professional skepticism.

Sampling requires judgment.

Judgment requires skepticism.

Auditors should question:

• unusual transactions

• unexpected relationships

• management explanations

• contradictory evidence

• incomplete documentation

Sampling should never become a mechanical exercise.

The PCAOB has repeatedly emphasized that audit deficiencies frequently stem from insufficient audit evidence and inadequate responses to identified risks rather than isolated procedural errors.

Issue 8 — Auditing Estimates Requires Different Sampling Thinking

Fair value measurements, loan losses, reserves, and goodwill do not always lend themselves to traditional transaction sampling.

Recent inspection reports consistently identify auditing estimates as one of the most frequent deficiency areas.

Auditors often need to evaluate:

• assumptions

• models

• management bias

• source data

• sensitivity analyses

Traditional random sampling alone may not provide sufficient evidence.

Issue 9 — Technology Has Changed Sampling

Today's audits increasingly include:

• full-population testing

• continuous auditing

• data analytics

• AI-assisted anomaly detection

• automated journal-entry analysis

Sampling has not disappeared.

Instead, auditors increasingly:

1. Analyze the full population.

2. Identify higher-risk items.

3. Apply targeted sampling.

4. Perform detailed testing.

The PCAOB has encouraged improvements in the testing of data, reports, and technology-enabled audit procedures as firms modernize their methodologies.

Issue 10 — Documentation Still Matters

Many PCAOB findings ultimately come down to documentation.

If reviewers cannot determine:

• why the sample was selected,

• how it was selected,

• what was tested,

• what exceptions occurred,

• why conclusions were reached,

then the audit evidence may be considered insufficient.

The workpaper should tell the audit story.

Common Sampling Weaknesses Seen in PCAOB Inspections

Across inspection reports, recurring weaknesses include:

• Failure to validate population completeness

• Insufficient testing of Information Produced by the Entity (IPE)

• Unsupported sample sizes

• Weak linkage between risk assessment and sampling

• Insufficient testing of controls

• Failure to investigate exceptions

• Weak documentation

• Inadequate professional skepticism

• Overreliance on management explanations

• Insufficient audit evidence supporting conclusions

Notice that only a few of these problems involve mathematical sampling techniques.

Most involve audit methodology.

What Internal Auditors Should Learn

Although PCAOB inspections apply to public-company audits, the lessons apply equally to Internal Audit.

Internal auditors should ask:

• Is my population complete?

• Is my sample risk-based?

• Have I validated system-generated reports?

• Do I understand why this sample size is appropriate?

• Have I investigated exceptions?

• Would another auditor reach the same conclusion?

• Does my documentation support my judgment?

These questions improve every audit.

The Future of Sampling

Artificial Intelligence and advanced analytics will continue changing sampling.

Instead of selecting transactions first, auditors increasingly will:

• analyze entire populations

• identify anomalies automatically

• prioritize high-risk items

• apply predictive analytics

• combine AI with professional judgment

However, one thing will not change.

Auditors must still obtain sufficient appropriate audit evidence.

Technology changes.

Professional judgment remains indispensable.

The Bottom Line

The last five years of PCAOB inspections provide a clear message.

The regulator is not criticizing auditors simply because they used samples.

The PCAOB is criticizing auditors because they sometimes:

• sampled from unreliable populations,

• relied on unverified reports,

• failed to respond to risk,

• inadequately tested controls,

• ignored exceptions,

• documented insufficient evidence, or

• exercised inadequate professional skepticism.

Audit sampling remains one of the profession's most powerful tools.

But a sample is only as strong as the audit methodology that surrounds it.

Learn More

Corporate Compliance Seminars' Audit Sampling Techniques course provides practical guidance on:

• Statistical and non-statistical sampling

• Sample-size determination

• Attribute and variable sampling

• Internal-control testing

• Substantive testing

• Sampling risk

• Documentation

• Evaluating sample results

• Applying sampling in today's technology-driven audit environment

For auditors seeking to improve audit quality and prepare for increasingly rigorous regulatory inspections, mastering audit sampling remains an essential professional skill.