Artifact §03 · Test pack

The UAT pack, in miniature

FIN-014 for the third and last time, at the point where the build is done and the business has to decide whether to accept it. This is the pack I would put in front of the controller: what gets tested, by whom, on what data, what failure looks like, and what has to be true before anyone signs.

User acceptance testing is not a second round of QA. System integration testing, which comes before it, proves the system does what the specification says. UAT proves the specification said the right thing. The testers are the people who will live with the result, and the pack is written for them, not for the delivery team.

As with the rest of the set, this is illustrative. The defects are the kind I have actually logged. The data is made up.

Scope and approach

Traceability

Every test case traces to a criterion in the requirement, and every criterion has at least one test. When a test cannot be traced, one of two things is true: the test is unnecessary, or the requirement is missing something. The last two rows here were the second kind.

Four of five criteria failed first time. That is not a bad build. It is what a first UAT on real data looks like, and a pack where everything passes on day one usually means the data was too clean or the testers were too polite.

Six of the ten test cases, written out

1. TC-01

   Every disagreeing line is listed

   traces to AC-01

   Precondition

   June 2026 loaded in both ledgers, both closed. Three differences seeded by the analyst: account 4100, GL higher by 1,250.00. Account 5230, production higher by 80.40. Account 6010, present in the GL only, 312.00.

   Steps

   1. Run the reconciliation for June 2026.
   2. Open the result.
   3. Compare the listed lines with the three seeded differences.

   Expected

   Exactly three lines, each showing the account code, the GL value, the production value, and the difference. No other lines.

   Result

   Pass. Three lines, values correct. The analyst asked that the difference column always be signed the same way, GL minus production. It is. Written into the one page guide so nobody has to ask twice.

2. TC-03

   The differences explain the whole variance

   traces to AC-02

   Precondition

   July 2026 loaded, both closed, nothing seeded. This is the real month.

   Steps

   1. Run the reconciliation for July.
   2. Add up the difference column.
   3. Compare with GL total minus production total, calculated independently. The query is further down.

   Expected

   The two figures are equal to the cent.

   Result

   Fail. Sum of listed differences 14,862.37. Ledger totals differ by 14,862.39. A residual of two cents, which is either nothing or the whole problem. Logged as D-02. Retest after the fix: pass.

3. TC-05

   Names every open ledger, not just the first

   traces to AC-03

   Precondition

   July 2026, both ledgers reopened for the test by their owners.

   Steps

   1. Attempt to run.
   2. Read the message.

   Expected

   No result is produced. The message names both ledgers.

   Result

   Fail. The message named the general ledger only, because the check stopped at the first open ledger it found. Severity 3: the controller would have reopened one, run again, and only then been told about the other. Logged as D-03. Retest: pass.

4. TC-06

   The export opens cleanly and keeps account codes as text

   traces to AC-04

   Precondition

   A completed July reconciliation.

   Steps

   1. Export to XLSX.
   2. Open it in Excel on the controller's machine, not mine.
   3. Check for a conversion or repair prompt.
   4. Find accounts 0410 and 0042, then sort the account code column.

   Expected

   The file opens with no prompt. The codes display as 0410 and 0042, left aligned, and the column sorts as text.

   Result

   Fail. The file opened cleanly. Account 0410 displayed as 410: the column had been written as numbers. Severity 2, because a downstream lookup against the chart of accounts would silently miss every zero led code. Logged as D-01. Retest: pass.

5. TC-08

   Runs unattended by the end of day two

   traces to AC-05

   Precondition

   Schedule set for 18:00 Pacific on the second working day. Both ledgers closed by 15:00 that day.

   Steps

   1. Nobody touches anything.
   2. The next morning, check the run log and the timestamp on the result.

   Expected

   One run, started at 18:00, completed, result available.

   Result

   Fail. The run started at 11:00, because the schedule was stored in UTC and nobody had converted it. Both ledgers happened to be closed by then, so the result was correct, which is the dangerous kind of pass. Logged as D-04. Retest the following day: pass.

6. TC-09

   The analyst can run and read, and cannot change a ledger

   no criterion yet. Proposed AC-06

   Precondition

   Logged in as the analyst's UAT account, not an administrator.

   Steps

   1. Run the reconciliation.
   2. Open and export the result.
   3. Attempt to post an adjustment to either ledger from the result screen.

   Expected

   Steps 1 and 2 succeed. Step 3 is not available.

   Result

   Pass. The controller asked for this test herself, which is usually how a missing criterion gets found. It goes into the requirement before anyone reuses it.

Defect log

Severity is how bad it is. Priority is how soon it gets fixed. They are not the same thing, and most of the arguments in triage come from treating them as if they were. D-02 is the example: severity 2, because a residual means the reconciliation cannot be fully trusted, but priority 2, because it was a known, explained two cents, and the controller would rather have the export fix first. Her call, not mine, and the log says so.

The query I ran myself, for AC-02

I do not execute the tests, but I do check the numbers the tests depend on. This is the independent figure TC-03 compares against. PostgreSQL, which is what the ERP copy ran on.

-- AC-02: the listed differences must explain the whole variance.
-- Two numbers that should agree to the cent, computed two different ways.
WITH gl AS (
  SELECT account_code, SUM(amount) AS gl_amount
  FROM gl_balances
  WHERE period = '2026-07'
  GROUP BY account_code
),
prod AS (
  SELECT account_code, SUM(amount) AS prod_amount
  FROM production_balances
  WHERE period = '2026-07'
  GROUP BY account_code
),
lines AS (
  SELECT
    COALESCE(gl.account_code, prod.account_code)        AS account_code,
    COALESCE(gl.gl_amount, 0)                          AS gl_amount,
    COALESCE(prod.prod_amount, 0)                      AS prod_amount,
    COALESCE(gl.gl_amount, 0) - COALESCE(prod.prod_amount, 0) AS difference
  FROM gl
  FULL OUTER JOIN prod ON prod.account_code = gl.account_code
)
SELECT
  SUM(difference)                                      AS sum_of_listed_differences,
  (SELECT SUM(amount) FROM gl_balances         WHERE period = '2026-07')
  - (SELECT SUM(amount) FROM production_balances WHERE period = '2026-07')
                                                       AS total_variance,
  COUNT(*) FILTER (WHERE difference <> 0)              AS lines_listed
FROM lines;

The full outer join is the point of the query. An inner join would silently drop any account that exists in only one ledger, the residual would look fine, and it would be wrong. That is the exact failure the as is process had for years, and I wanted a check that could not repeat it.

Cutover, and the first two closes

1. Training is one session, not a course.

   The controller and the analyst, forty minutes, on their own machines, with the July data. Then a one page guide: how to read the list, what the rounding line means, who to call. Nobody reads a twenty page manual in the middle of a close.

2. Go live is the August close, and I am in the room for it.

   I sit with the analyst on day two and day three of the first two closes. Not to run anything. To be there when the first real question comes up, because it will, and how that question gets answered decides whether the system gets used or worked around.

3. We measure the publish day for three closes and write it down.

   If it is not day three by the third close, the requirement was wrong somewhere, and I would rather find out from the measurement than from the controller six months later.

Sign off

The conditions line is the part people skip. A sign off with unwritten conditions is a sign off that gets relitigated the first time something goes wrong. Two lines of text, agreed in the room, and the conversation in October is about the system instead of about what everyone remembers agreeing to.

What I left out, on purpose

No performance test. I would have had to invent a target, and a criterion nobody can fail is decoration. The same reasoning is in the requirement, and it still holds here.

No regression pack for the ledgers themselves. Their teams own that, and pretending otherwise would put my name on a result I cannot stand behind.

No automated UAT. Automation belongs in SIT, and the delivery team had plenty of it. UAT is the business touching the thing. If they do not touch it, it is not acceptance, whatever the report says.

Written by Nasmaan Ibrahim for nasmaan.com. Illustrative, not drawn from any employer's systems or data.
Back to the record · 01 One requirement, written out in full · 02 The process, drawn · Resume, PDF