Payroll ran short and nobody found out until Friday
The run reported success and the file went to the bank. The people it missed found out on payday, and by then the fix was a hardship payment rather than a correction.
What actually happens
A process that fails completely gets attention. A process that succeeds for ninety-eight percent of its population gets a green tick.
The payroll run executes on schedule. It reads employee records, calculates gross to net, applies deductions, and produces a payment file for the bank. It reports success.
It processed everyone it could see. What it could not see was a group of employees added since the last cycle whose records were still awaiting a cost-centre assignment, so the extract that feeds payroll silently excluded them.
Nothing errors. The extract returned rows, payroll processed rows, and the file contains payments. Every check in the chain is a check for failure, and this was not a failure, it was a smaller success.
The payment file goes to the bank on Wednesday. The people who were missed find out on Friday morning, in the way people always find out about payroll problems, which is by looking at their account.
What follows is not a technical incident. It is hardship payments raised individually, a set of very difficult conversations, and a trust cost with the affected employees that lasts considerably longer than the correction does.
Payroll did not fail. It succeeded for everyone it could see, which is a different thing, and no check in the chain was asking that question.
The same run, two ways
The window here is not minutes. It is the gap between the file being produced and the file being irreversible.
Illustrative, not measured. The times below model a scenario built from the patterns we see in production estates. They are not timings recorded at a named customer. The point is the shape of the clock, not the totals: check it against your own last ten incidents.
Today, checking for failure
- Tue 22:00Payroll run executes. Extract silently omits employees without a cost centre.
- Tue 22:40WaitingRun reports success. Record count not compared against the expected population.
- Wed 09:15WaitingPayment file released to the bank. Now irreversible for this cycle.
- Wed ↓ FriWaitingNo signal. Nobody is looking for people who are absent from a file.
- Fri 08:20Affected employees discover they have not been paid. Escalation to HR and finance.
- Fri 16:00Hardship payments raised individually. Root cause found in the extract filter.
~3 days · and the correction was the expensive part
With the population checked, not just the run
- Tue 22:40Run reports success. Sentinel compares the processed population against the expected headcount from the HR system.
- Tue 22:42Variance detected. Missing employees identified by name, along with what they have in common.
- Tue 22:45Extract filter, recent joiner records and cost-centre assignment status correlated. Cause identified.
- Tue 22:46Payment file release held automatically. This is reversible; releasing it is not.
- Tue 22:47Action Ticket raised with the affected employee list, the proposed fix and the rollback path. Payroll owner notified.
- Wed 07:30Approved. Cost centres assigned, extract re-run, file regenerated and released on time. Nobody is short.
Caught before release · the file that went out was correct
There is no MTTR figure here worth quoting, and inventing one would miss the point. The value is that a payment file which cannot be recalled was never released in the wrong state.
The cost in the first column is not engineering time. It is hardship payments, finance and HR effort spent one employee at a time, and a trust cost that does not appear on any incident report.
The mechanism is checking the population, not the process. A run that reports success tells you the code executed. Comparing the processed headcount against the expected headcount from the source of record tells you whether it executed on everyone, and those are different questions.
Holding the file is the reversible half and Sentinel does it immediately. Regenerating and releasing is not reversible, so that waits for the payroll owner. That asymmetry is the same one that governs every action on this platform.
Why the number is what it is
There is no MTTR figure here worth quoting, and inventing one would miss the point. The value is that a payment file which cannot be recalled was never released in the wrong state.
The cost in the first column is not engineering time. It is hardship payments, finance and HR effort spent one employee at a time, and a trust cost that does not appear on any incident report.
The mechanism is checking the population, not the process. A run that reports success tells you the code executed. Comparing the processed headcount against the expected headcount from the source of record tells you whether it executed on everyone, and those are different questions.
Holding the file is the reversible half and Sentinel does it immediately. Regenerating and releasing is not reversible, so that waits for the payroll owner. That asymmetry is the same one that governs every action on this platform.
The mechanism is checking the population, not the process.
Who decides to press go
Releasing a payment file is one of the least reversible actions in a business, so nothing about it is automated.
Holding the file, comparing populations, identifying the affected employees and assembling the evidence all run under policy. All of it is reversible and none of it moves money.
The Action Ticket carries the affected employee list, the proposed correction, the expected record count and the rollback path. The payroll owner approves before any file is regenerated or released.
Holding a payment file costs you hours if the hold turns out to be unnecessary. Releasing a wrong one costs you a great deal more, and it cannot be undone. The gate is asymmetric because the risk is.
Payroll run reports success. Payment file generated. No error raised anywhere in the chain. Processed population not compared against expected headcount.
Processed population compared against the HR system of record. Missing employees identified along with their common attribute. Extract filter, joiner records and cost-centre status correlated.
Payment file release held automatically. Action Ticket raised with the affected employee list and proposed correction, held for the payroll owner.
Population variance promoted to a first-class check on every run. Cost-centre assignment added as a joiner completion gate so the extract cannot silently exclude anyone again.
This is a platform capability, not a published customer deployment. The mechanism, which is treating a business process outcome as a signal, correlating across the systems that implement it, and then executing a governed MOP with pre-check, post-check, rollback and approval gating, is running in production today; see governed day-2 operations across 2,000+ nodes and closed-loop network automation. The back-office scenario above applies that same mechanism to a business process. The timings shown are modelled, not measured at a named customer.
If the action carries no service impact
Holding the file, comparing populations, identifying the affected employees and assembling the evidence all run under policy. All of it is reversible and none of it moves money.
If it releases, regenerates or moves money
The Action Ticket carries the affected employee list, the proposed correction, the expected record count and the rollback path. The payroll owner approves before any file is regenerated or released.
What Sentinel did, step by step
- ObservePayroll run reports success. Payment file generated. No error raised anywhere in the chain. Processed population not compared against expected headcount.
- InvestigateProcessed population compared against the HR system of record. Missing employees identified along with their common attribute. Extract filter, joiner records and cost-centre status correlated.
- ActPayment file release held automatically. Action Ticket raised with the affected employee list and proposed correction, held for the payroll owner.
- OptimizePopulation variance promoted to a first-class check on every run. Cost-centre assignment added as a joiner completion gate so the extract cannot silently exclude anyone again.
Bring us a payroll cycle that went wrong
We will show you at what point the file could still have been held.