Apex Test Quality: Why 75% Code Coverage Tells You Nothing

This article is part of the Apex Testing series, three articles on writing Apex tests that catch regressions instead of chasing a number. Articles are standalone; no need to read in order.

TL;DR

• Salesforce requires 75% org-wide Apex coverage to deploy to production, and every trigger needs some coverage. That is a deployment gate, not a quality bar.
• Coverage records which lines executed. It cannot tell whether anything was checked. A test method with zero assertions produces exactly the same coverage number as a thorough one.
• Use the Assert class (Assert.areEqual, Assert.isTrue, Assert.fail) rather than the older System.assertEquals family, and always pass the third message argument.
• Test.startTest() and Test.stopTest() do two specific things: they give the code between them a fresh set of governor limits, and they force queued asynchronous work to run before the assertions.
• Test with 200 records, not one. One record proves the code compiles. Two hundred proves it survives a data load.
• Test the paths you do not want: expected exceptions, validation failures, empty inputs, nulls.
• Use System.runAs with a real permission set so the test exercises the access model, not your System Administrator view.

What You'll Learn

• What the 75% rule actually requires, and what it deliberately does not
• The shape of an Apex test method that will still catch a bug in two years
• What Test.startTest() and @TestSetup really do, including the traps
• How to assert on exceptions without writing a test that silently passes
• The four anti-patterns that raise coverage while lowering confidence

The Problem

Every Salesforce team eventually has the same conversation. A change goes out, something breaks in production, and someone points at the pipeline and asks how it got through when the org sits at 84% coverage.

It got through because coverage and correctness are unrelated measurements. Coverage is a line counter. When the Apex runtime executes a line inside a test context, that line is marked covered. Nothing anywhere asks whether the test looked at the result.

This is not a loophole anyone has to exploit deliberately. It happens by drift. A developer under deadline writes a test that inserts a record and calls the method. It goes green, the class shows 92%, the review passes. Three years later that class has been rewritten twice, and the test still inserts a record and calls the method, and it has never once failed.

Common questions this article answers:

• What does Salesforce actually require for coverage, and can it be changed?
• How do I tell a useful Apex test from one that only produces coverage?
• Why do my tests pass locally and then fail after a data load?

Quick Answer

Salesforce requires 75% org-wide Apex code coverage to deploy to production, plus some coverage on every trigger. The number is a gate against completely untested code, not a measure of test quality, because coverage only records which lines the runtime executed and never inspects whether the test verified anything. To get real value, write each test method in three visible parts: arrange the data, act on the code under test, then assert on observable outcomes with Assert.areEqual(expected, actual, 'message explaining what broke'). Wrap the call in Test.startTest() and Test.stopTest() so it gets clean governor limits and so any queued asynchronous work completes before you assert. Build your data at 200 records rather than one, because the trigger batch size is 200 and bulk failures do not appear at a batch of one. Exercise the failure paths explicitly, asserting both that an exception was thrown and what it said. Finally, wrap the call in System.runAs(testUser) with the permission set the real user will hold, so the test reflects the access model rather than the developer's own administrator profile.

What the 75% rule actually requires

Three separate things are worth keeping apart.

To deploy Apex to production, the org needs at least 75% coverage across all Apex, and every trigger needs some coverage. The 75% is computed org-wide, so a completely untested class can hide behind a well-tested one. There is no per-class minimum for a normal deployment.

Coverage is computed from the last test run, not continuously. A class showing 90% in the setup UI may be showing a stale number from a run weeks ago against different code. Re-run the tests before you trust the figure.

What counts as a covered line is narrower than people expect. Comments, blank lines, method signatures, and curly braces do not count. System.debug statements do not count. That means a class padded with debug statements reports differently from one that is not, which is one reason coverage percentages move for no apparent reason.

None of the three says anything about whether the behaviour is right.

The shape of a test that earns its keep

Three parts, visibly separated. Arrange the world, act on one thing, assert on what should now be true.

@IsTest
private class InvoiceTotalCalculatorTest {

    @IsTest
    static void totalExcludesCancelledLines() {
        // Arrange
        Account customer = new Account(Name = 'Northwind Trading');
        insert customer;

        Invoice__c invoice = new Invoice__c(Account__c = customer.Id);
        insert invoice;

        insert new List<Invoice_Line__c>{
            new Invoice_Line__c(Invoice__c = invoice.Id, Amount__c = 100, Status__c = 'Active'),
            new Invoice_Line__c(Invoice__c = invoice.Id, Amount__c = 250, Status__c = 'Active'),
            new Invoice_Line__c(Invoice__c = invoice.Id, Amount__c = 999, Status__c = 'Cancelled')
        };

        // Act
        Test.startTest();
        InvoiceTotalCalculator.recalculate(new Set<Id>{ invoice.Id });
        Test.stopTest();

        // Assert
        Invoice__c result = [SELECT Total__c FROM Invoice__c WHERE Id = :invoice.Id];
        Assert.areEqual(
            350,
            result.Total__c,
            'Cancelled lines should be excluded from the invoice total'
        );
    }
}

Read the assertion on its own. It states a rule of the business: cancelled lines do not count. If someone changes the calculator so cancelled lines are included, this test fails and the message tells them why. That is the entire job.

Compare it to the version that produces identical coverage:

@IsTest
static void testRecalculate() {
    // ... same setup ...
    Test.startTest();
    InvoiceTotalCalculator.recalculate(new Set<Id>{ invoice.Id });
    Test.stopTest();
}

Same lines executed. Same percentage. Zero protection.

One behaviour per method. A test method named totalExcludesCancelledLines tells you what broke the moment it goes red. A method named testInvoice that asserts eleven unrelated things tells you only that something, somewhere, is wrong, and it stops at the first failed assertion so you never see the other ten.

Use the Assert class, and use the message argument

Since Winter '23 (API 56.0) Apex has a proper Assert class. It reads better and it covers cases the old methods did not.

Assert.areEqual(350, result.Total__c, 'Cancelled lines should be excluded');
Assert.areNotEqual(null, result.Id, 'Record should have been inserted');
Assert.isTrue(result.Total__c > 0, 'Total should be positive after recalculation');
Assert.isFalse(result.Locked__c, 'Recalculation should not lock the invoice');
Assert.isNull(orphan, 'Orphaned line should have been deleted');
Assert.isNotNull(summary, 'Summary should be built for every invoice');
Assert.isInstanceOfType(handler, InvoiceHandler.class, 'Factory returned the wrong handler');
Assert.fail('Expected a DmlException for a negative amount');

The older System.assertEquals(expected, actual, message) still works and is not deprecated, so there is no urgency to rewrite existing tests. New code should use Assert.

Two details cost people real time.

Argument order is (expected, actual). Reversing it does not fail the test, it fails the failure message, which then says the opposite of what happened and sends the next person looking in the wrong place.

Always pass the third argument. Assert.areEqual(350, result.Total__c) produces "Expected: 350, Actual: 250" with no indication of which rule was violated. In a class with twenty assertions that is a scavenger hunt.

What Test.startTest and Test.stopTest actually do

They are not decoration, and they are not a timer. They do two concrete things.

They reset governor limits. Everything between startTest() and stopTest() gets a fresh allocation: 100 SOQL queries, 150 DML statements, and the rest. This matters because your setup data usually costs a lot of DML, and without the reset that spend counts against the code you are trying to test. Put data creation before startTest() and the code under test between the pair.

They force asynchronous work to complete. @future methods, Queueables, and Batch jobs enqueued inside the block do not run when you call them. They sit in a queue. At stopTest() they execute synchronously, before the next line. That is why the assertions must come after stopTest(), not before it:

Test.startTest();
AccountEnrichmentQueueable job = new AccountEnrichmentQueueable(accountIds);
System.enqueueJob(job);
Test.stopTest();   // the queueable actually runs here

// Only now is there anything to assert on
List<Account> enriched = [SELECT Industry FROM Account WHERE Id IN :accountIds];
for (Account a : enriched) {
    Assert.areEqual('Technology', a.Industry, 'Enrichment should set Industry from the match service');
}

Assert before stopTest() and the test passes while proving nothing, because the job has not run yet. This is one of the most common false-green tests in Apex.

You get one pair per test method. A second Test.startTest() in the same method throws. If you need two independent limit windows, you need two test methods.

@TestSetup, and the three things it does not do

@TestSetup runs once per test class and the records it creates are available to every test method, with any changes rolled back between methods. On a class with fifteen test methods it turns fifteen data builds into one.

@TestSetup
static void makeData() {
    Account customer = new Account(Name = 'Northwind Trading');
    insert customer;

    List<Invoice__c> invoices = new List<Invoice__c>();
    for (Integer i = 0; i < 200; i++) {
        invoices.add(new Invoice__c(Account__c = customer.Id, Reference__c = 'INV-' + i));
    }
    insert invoices;
}

Three traps come with it.

Records do not arrive in memory. Each test method has to query for what it needs. There is no shared variable, because each method runs in its own transaction against the rolled-back state.

A failure in setup fails the whole class. Every method reports as failed, usually with an error pointing at the setup method rather than at the test, which makes a single bad validation rule look like fifteen broken tests.

It is incompatible with SeeAllData=true. If the class carries that annotation, the setup method is simply not supported. That is a feature, not a limitation.

Test at 200, not at 1

The trigger batch size is 200. Code that works on one record and fails on two hundred is the most common Apex defect, and a test that inserts one record cannot see it.

@IsTest
static void recalculationHandlesAFullBatch() {
    List<Invoice__c> invoices = [SELECT Id FROM Invoice__c LIMIT 200];
    Assert.areEqual(200, invoices.size(), 'Test setup should provide a full batch');

    Test.startTest();
    InvoiceTotalCalculator.recalculate(new Map<Id, Invoice__c>(invoices).keySet());
    Test.stopTest();

    Integer unprocessed = [SELECT COUNT() FROM Invoice__c WHERE Total__c = NULL];
    Assert.areEqual(0, unprocessed, 'Every invoice in the batch should have been totalled');
}

This is the test that catches a SOQL query inside a loop before your users do. If it fails with Too many SOQL queries: 101, the design is wrong, and the fix belongs in the code, not in the test.

Note the assertion on the setup itself (Assert.areEqual(200, invoices.size(), ...)). Without it, a setup method that quietly produced three records would leave you with a bulk test that passes because it never ran in bulk.

Test the paths you do not want

Most real defects live in the branches nobody exercised. Three categories are worth deliberate effort.

Expected exceptions. The pattern needs a fail() call, or the test passes when no exception is thrown at all:

@IsTest
static void negativeAmountIsRejected() {
    Invoice_Line__c bad = new Invoice_Line__c(Amount__c = -50);

    Test.startTest();
    try {
        insert bad;
        Assert.fail('Expected a DmlException for a negative amount');
    } catch (DmlException e) {
        Assert.isTrue(
            e.getMessage().contains('Amount must be zero or greater'),
            'Wrong validation fired: ' + e.getMessage()
        );
    }
    Test.stopTest();
}

Leave out Assert.fail(...) and the test is worthless: if the validation rule is deactivated tomorrow, the insert succeeds, the catch block never runs, and the test still reports green.

Partial failures. Database.insert(records, false) returns a result per record. Assert on the successes and the failures separately, because the interesting bug is usually that one bad record silently took nineteen good ones with it.

Empty and null inputs. Call the method with an empty list and a null. Most methods should do nothing gracefully. Many throw a NullPointerException that nobody found because no test ever passed nothing.

Run as the user who will run the code

A test that runs as you runs as a System Administrator, which means it never sees the access model your users live inside.

@IsTest
static void serviceAgentCannotSeeOtherRegions() {
    User agent = TestUsers.serviceAgent();   // created with the real permission set

    System.runAs(agent) {
        Test.startTest();
        List<Case> visible = CaseFinder.forCurrentUser();
        Test.stopTest();

        Assert.areEqual(3, visible.size(), 'Agent should only see cases in their own region');
    }
}

System.runAs changes the running user, so sharing rules apply to classes declared with sharing, and the test exercises the record access design rather than your view of it.

One caveat that catches people: runAs does not make Apex enforce object and field permissions on its own. Apex still runs in system mode for CRUD and field-level security unless you ask for user mode explicitly with WITH USER_MODE on a query or as user on DML. If you want a test to prove a field is inaccessible, the code under test has to be asking in user mode in the first place. Secure by default behaviour tightened this in recent releases, which is worth checking against your own classes.

Four anti-patterns that pass the gate

SeeAllData=true. It gives the test access to real org data, which makes it pass or fail depending on what someone did in the org that morning. It is occasionally unavoidable (some standard objects cannot be created in a test), but each use should be a decision with a comment, not a default.

Test.isRunningTest() in production code. A branch that only exists during tests means the tested path is not the shipped path. It is usually a sign that something needs to be injected rather than detected, which is what Part 2 of this series is about.

Assertion-free coverage padding. A test method that loops over every method in a utility class to lift the percentage. It is fast to write, adds a maintenance burden forever, and protects nothing.

Asserting that the platform works. insert acc; Assert.isNotNull(acc.Id); tests Salesforce, not your code. Assert on the thing your code was supposed to do to the record.

Frequently Asked Questions

Q: Can Salesforce raise the 75% coverage requirement for my org?

A: No. The 75% org-wide threshold for production deployments is a platform rule and support cannot change it. You can make the gate stricter yourself, either in your pipeline by failing a build below a higher number, or per deployment by using a test level such as RunSpecifiedTests with your own rules about what must pass. Raising the number is much less valuable than raising the assertion quality behind it, since 90% coverage with no assertions protects less than 76% with good ones.

Q: Why does my test pass in a sandbox and fail in production?

A: Almost always data or configuration, not code. The usual causes are a test relying on SeeAllData=true and finding different records, a validation rule or required field that exists in production but not in the sandbox, a trigger or flow from a managed package that is installed in only one of them, or a record type that differs. Build every record the test needs inside the test itself, and the difference disappears.

Q: Do I need Test.startTest and Test.stopTest in every test method?

A: Not strictly, but it costs nothing and you need it more often than you think. It is required whenever the method under test enqueues asynchronous work, since without it the job never runs. It is required in practice whenever your setup data costs meaningful DML, since without it that spend counts against the code under test. Using it consistently means you never have to work out which case you are in.

Q: How many assertions should one test method have?

A: As many as it takes to describe one behaviour, and no more. If the assertions are all facets of a single outcome, such as checking four fields on one record after a calculation, they belong together. If they describe different rules, split them into separate methods so that a failure names the rule that broke. Remember that execution stops at the first failed assertion, so a long method hides the assertions after the failure.

Q: Does @IsTest(isParallel=true) change what my tests prove?

A: No, it changes how they run. Test classes marked isParallel=true are exempt from some concurrency limits and run alongside each other, which can cut a long suite's runtime substantially. It does not change any assertion. The catch is that it is only safe when the class does not depend on org-wide setup data or anything else a concurrent test could be mutating, so classes touching setup objects such as users or permission sets are usually left out.

Key Takeaways

• Coverage is a gate, not a measure: 75% org-wide is the minimum to deploy, and it says nothing about whether any behaviour is verified.
• The assertion is the test: if a change to the code cannot make the method go red, the method is not testing anything.
• Test.stopTest() is where async happens: assert after it, never before, or the job has not run yet.
• Two hundred records, not one: bulk defects are invisible at a batch of one and obvious at a full batch.
• Failure paths deserve tests too: expected exceptions need an Assert.fail() or they pass when the exception stops being thrown.
• System.runAs tests the access model: but Apex still needs WITH USER_MODE or as user for object and field permissions to be enforced.

What's Next?

Recommended reading:

• Mocking in Apex: Stub API, HttpCalloutMock and ApexMocks Compared, the next article in this series
• Apex Test Data and CI Tooling: What to Use and What It Costs
• System.LimitException: Too many SOQL queries: 101
• Mixed DML operations when provisioning users

Action items:

1. Pick your three largest Apex classes and count the assertions in their test classes. Divide by the number of test methods. If the answer is below one, coverage is all you have.
2. Search your codebase for Test.isRunningTest() and SeeAllData=true. Every hit is either a deliberate decision with a comment, or debt.
3. Add one full batch test, at 200 records, to the trigger handler that touches the most data.
4. Convert one test class to the Assert class with messages, and use it as the reference the next person copies.

Resources & References

• Apex Developer Guide: Testing Apex
• Assert Class reference
• Using Test.startTest and Test.stopTest
• Testing as a different user with System.runAs
• Code coverage requirements for deployment
• Trailhead: Apex Testing module