COBOL Module 11 – Errors, debugging and testing
Refactoring legacy COBOL with regression tests
Refactor legacy COBOL without changing a line of its output: golden-file regression tests, seeded synthetic data, small steps and compiler warnings.
What you will learn
- Record a program's output as a golden file and compare every later run with it
- Generate seeded synthetic test data that meets every rule on both sides of its limit
- Refactor GO TO, PERFORM THRU and nested IF logic in steps that keep the output identical
- Find risky code with -Wall, the listing's cross-reference and the production dialect
Before you start
What you’ll learn · Before you start
What you will learn
- Record a program's output as a golden file and compare every later run with it
- Generate seeded synthetic test data that meets every rule on both sides of its limit
- Refactor GO TO, PERFORM THRU and nested IF logic in steps that keep the output identical
- Find risky code with -Wall, the listing's cross-reference and the production dialect
Before you start
On this page
INTCALC has worked out the month’s interest on savings accounts and the charge on overdrawn current accounts in the same 72 lines for years: three GO TOs, a PERFORM THRU, two NEXT SENTENCEs and its rates written as literals. A new account type has to be added, and nobody can say for certain what every line does.
The safe order is to record exactly what the program prints today, and then to change it in steps small enough that any difference points at one edit. The record is a golden file, and the comparison is a regression test.
A golden-file test compares every run with today’s output
INTCALC reads accounts, prints one line for each savings or current account with the month’s amount, skips any other type as rejected, and prints two totals. Its rules are teaching rules: 3 % a year on savings balances below 10,000.00, 3.5 % from 10,000.00, and 18 % a year on overdrawn current accounts:
IDENTIFICATION DIVISION.
PROGRAM-ID. INTCALC.
* Monthly interest on savings accounts and the monthly charge
* on overdrawn current accounts, in the style of an old program.
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT ACCT-FILE ASSIGN TO "ACCTIN"
ORGANIZATION IS LINE SEQUENTIAL
FILE STATUS IS ACCT-STATUS.
DATA DIVISION.
FILE SECTION.
FD ACCT-FILE.
01 ACCT-REC.
05 AC-ID PIC X(6).
05 AC-TYPE PIC X.
05 AC-BALANCE PIC S9(7)V99 SIGN LEADING SEPARATE.
WORKING-STORAGE SECTION.
01 ACCT-STATUS PIC XX.
01 WS-AMOUNT PIC S9(5)V99.
01 WS-TOTAL PIC S9(7)V99 VALUE 0.
01 WS-COUNT PIC 9(4) VALUE 0.
01 WS-REJECTS PIC 9(4) VALUE 0.
01 WS-OLD-RATE PIC V999 VALUE 0.025.
01 OUT-LINE.
05 OL-ID PIC X(6).
05 FILLER PIC X VALUE SPACE.
05 OL-TYPE PIC X.
05 FILLER PIC X VALUE SPACE.
05 OL-AMOUNT PIC -(6)9.99.
PROCEDURE DIVISION.
000-MAIN.
OPEN INPUT ACCT-FILE.
010-READ.
READ ACCT-FILE AT END GO TO 090-END.
ADD 1 TO WS-COUNT.
PERFORM 100-CALC THRU 100-EXIT.
GO TO 010-READ.
090-END.
DISPLAY "ACCOUNTS " WS-COUNT " REJECTED " WS-REJECTS.
DISPLAY "TOTAL " WS-TOTAL.
CLOSE ACCT-FILE.
STOP RUN.
100-CALC.
MOVE 0 TO WS-AMOUNT.
IF AC-TYPE = "S"
IF AC-BALANCE > 0
IF AC-BALANCE < 10000
COMPUTE WS-AMOUNT ROUNDED =
AC-BALANCE * 0.03 / 12
ELSE
COMPUTE WS-AMOUNT ROUNDED =
AC-BALANCE * 0.035 / 12
ELSE
NEXT SENTENCE
ELSE
IF AC-TYPE = "C"
IF AC-BALANCE < 0
COMPUTE WS-AMOUNT ROUNDED =
AC-BALANCE * 0.18 / 12
ELSE
NEXT SENTENCE
ELSE
ADD 1 TO WS-REJECTS
GO TO 100-EXIT.
ADD WS-AMOUNT TO WS-TOTAL.
MOVE AC-ID TO OL-ID.
MOVE AC-TYPE TO OL-TYPE.
MOVE WS-AMOUNT TO OL-AMOUNT.
DISPLAY OUT-LINE.
100-EXIT.
EXIT.AC0001S+000250000
AC0002S+000999999
AC0003S+001000000
AC0004S+001250050
AC0005S+000000000
AC0006S-000005000
AC0007C+000150000
AC0008C-000040000
AC0009C-000000001
AC0010X+000100000A golden file: The output of a program on a fixed input, saved so that every later version of the program can be run on the same input and compared with it line by line. is that output, saved: the session runs the legacy program on accounts.txt and keeps what it prints as golden.txt. intcalc-after.cob is the same program restructured, with one READ loop, an EVALUATE and named rates, and its output is compared with golden.txt line by line. Then comes intcalc-slip.cob, the restructured program with one operator changed:
from session import files, run
# The legacy INTCALC run on ten accounts makes the golden file. The restructured version is compared with it, then a
# version of the restructured program with one operator changed.
files("intcalc-before.cob", "intcalc-after.cob", "intcalc-slip.cob", "accounts.txt")
run("""
cobc -x -o before intcalc-before.cob && DD_ACCTIN=accounts.txt ./before > golden.txt
cat golden.txt
cobc -x -o after intcalc-after.cob && DD_ACCTIN=accounts.txt ./after > after.txt
diff golden.txt after.txt && echo "no differences"
!diff intcalc-after.cob intcalc-slip.cob
!cobc -x -o slip intcalc-slip.cob && DD_ACCTIN=accounts.txt ./slip > slip.txt && diff golden.txt slip.txt
""")Output
$ cobc -x -o before intcalc-before.cob && DD_ACCTIN=accounts.txt ./before > golden.txt $ cat golden.txt AC0001 S 6.25 AC0002 S 25.00 AC0003 S 29.17 AC0004 S 36.46 AC0005 S 0.00 AC0006 S 0.00 AC0007 C 0.00 AC0008 C -6.00 AC0009 C 0.00 ACCOUNTS 0010 REJECTED 0001 TOTAL +0000090.88 $ cobc -x -o after intcalc-after.cob && DD_ACCTIN=accounts.txt ./after > after.txt $ diff golden.txt after.txt && echo "no differences" no differences $ diff intcalc-after.cob intcalc-slip.cob 70c70 < WHEN AC-BALANCE < SAVINGS-TIER --- > WHEN AC-BALANCE NOT > SAVINGS-TIER [exit status 1] $ cobc -x -o slip intcalc-slip.cob && DD_ACCTIN=accounts.txt ./slip > slip.txt && diff golden.txt slip.txt 3c3 < AC0003 S 29.17 --- > AC0003 S 25.00 11c11 < TOTAL +0000090.88 --- > TOTAL +0000086.71 [exit status 1]
Recorded with Python 3.14.8 on macOS 26 arm64. To run it yourself: mise exec python@3.14.8 -- python3 golden_session.py
The restructured program printed exactly what the legacy one prints, so diff found no differences. The slip, NOT >
where < belongs, gave AC0003, an account of exactly 10,000.00, the low rate: 25.00 instead of 29.17, and the total
moved from 90.88 to 86.71. A test of this kind is a characterisation test: A test that records what code does today, right or wrong, so that a change meant to keep its behaviour can be checked against the record. A golden-file test is one.: it
checks that behaviour stays the same, not that it is right. Step through the loop it puts around every change:
The drawing shows the whole scene; its 6 steps are listed below.
Every step as text
- The input. accounts.txt holds ten synthetic accounts, chosen so that each rule of INTCALC is met on both sides of its limit, 10,000.00 among them.
- The golden file. The legacy program, as it runs today, prints one line for each account it reports and two totals. Saved as golden.txt, that output is what every later version must print.
- One change. The restructured program replaces the GO TOs, the THRU range and the NEXT SENTENCEs. In practice that is several small steps, each run and compared on its own.
- The same input again. The changed program reads the same accounts.txt and its output goes to a file of its own, after.txt.
- No differences. diff found no line that differs, so this version prints exactly what the legacy program prints, for every case in the input.
- A difference. The slipped version, with NOT > where < belongs, changed two lines, AC0003 at exactly 10,000.00 and the total. A difference sends you back to the change, to explain it or undo it.
Key idea
Record what the program prints today before you change a line, and compare every changed version with that record: a difference points at the last edit.
Checkpoint
Check: Golden-file regression tests
Five habits keep a refactoring from changing results
The golden file is the first of five habits, and each one closes a gap the others leave:
| Habit | What it protects | In one sentence |
|---|---|---|
| Golden files | today’s output | the legacy program’s output, kept and compared after every change |
| Seeded synthetic data | real accounts, and the rules’ limits | generated accounts that meet every limit, the same file on every run |
| Small steps | the link between a change and its effect | one kind of edit at a time, each compared before the next |
| Warnings and the cross-reference | code that looks unused or risky | cobc -Wall and -Xref point at what to read first |
| The production dialect | outputs that differ by compiler settings | build the golden file the way production builds the program |
Seeded synthetic data means test accounts that a program generates instead of copying real ones, from a fixed seed, so that every run writes the same file. Compiler warnings come from the compiler itself: cobc -Wall names constructs that GnuCOBOL treats as archaic or risky, and the cross-reference of the listing marks items that no statement uses.
A behaviour that looks wrong stays as it is during a refactoring. INTCALC prints a line of 0.00 for a savings account in debit, which may be a fault, but a refactoring that changes it can no longer be checked against the golden file; write it down and change it later, in a change of its own, with a test that says the new behaviour is wanted.
Checkpoint
Check: The five habits of a safe refactoring
Practise: compare before you change
The questions below come from the golden-file runs and the five habits. Decide each before you choose.
Practice that adapts to you
Practise this lesson
5 questions about 5 ideas of this lesson. Each answer is explained at once, and what you get right is kept in this browser.
Seeded synthetic data that meets every rule at its limits Pro: opens with a Pro, Premium or Ultimate pass
Ten hand-written accounts cover INTCALC’s rules, and a real batch needs thousands. This part generates them from a seed, shows why the seed matters in GnuCOBOL 3.2, and measures what random accounts alone would have missed.
In this part:
- A generator with a seed, the limits first, and no real accounts
- What random accounts alone would have missed
3 runnable examples (COB, Python) · about 4 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
GO TO, THRU and nested IFs restructured in five steps, compared after each Pro: opens with a Pro, Premium or Ultimate pass
Each kind of edit in a refactoring gets a step of its own, and each step is compared with the golden file before the next begins. This part takes INTCALC from three GO TOs to none in five steps, and shows the one slip on the way that only the comparison found.
In this part:
- Five steps from intcalc-before.cob to intcalc-after.cob
- A period removed: the slip only the comparison saw
1 diagram · 3 runnable examples (COB, Python) · about 5 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Warnings, cross-references and dialects that find risky code first Pro: opens with a Pro, Premium or Ultimate pass
Before a refactoring starts, the compiler can say where to look and the build can be made to match production. This part reads cobc’s warnings and cross-reference for INTCALC and builds it for the IBM dialect.
In this part:
- What the warnings, the cross-reference and GnuCOBOL 3.2’s checks point at
- The golden file must be built the way production builds the program
1 runnable example (Python) · about 4 minutes
Opening this part with your pass…
This part opens with any MySmartCoPilot pass.
- Pro pass · 1 week₹79$2.99
- 30 days₹179$5.99
- 1 year₹1,699$45
One payment; a pass never renews. The price shown is the total you pay: no fees or charges are added at checkout.
Passes can’t be bought right now, so this part can’t be opened at the moment. The free part above stands on its own. If you have a pass, enter your licence key.
It also opens every other in-depth part in Learn and every Pro tool. Sales are final, except where the law gives you a right to withdraw, for example in the EU and the UK (refund policy).
Interview questions
One warm-up, answered here, then three questions a refactoring raises: a difference nobody planned, the test data and the plan itself:
-
Fresher to mid · Warm-up
What is a golden-file test, and what does it prove?
A golden-file test runs the program as it is today on a fixed input and saves what it prints as the golden file. Every later version runs on the same input, and its output is compared with the golden file line by line, with diff for example. No differences means the change kept the behaviour for every case in the input, and a difference points at the last change. It proves sameness, not correctness: it is a characterisation test, so a line that looks wrong stays as it is during a refactoring and is changed later, on its own. In the lesson the restructured INTCALC matched its golden file, and a slip of one operator changed two lines, the account at exactly 10,000.00 and the total, which is why the input holds a value on each limit.
-
Mid to senior
You restructured a legacy interest program. The golden-file comparison shows 3 of 400 accounts with interest 0.01 higher than before, and the business says the new figures are the right ones. What do you do?
-
Fresher to mid
A colleague proposes copying last month's production accounts to use as regression test data for a refactoring, because they are realistic. What do you say, and what do you do instead?
-
Senior to staff
Plan the refactoring of a 2,000-line batch program full of GO TO, PERFORM THRU and NEXT SENTENCE, so that nothing it prints changes. What are your steps, and what do you check after each one?
Organisations named on this page
MySmartCoPilot is not connected with IBM or the GnuCOBOL project. IBM’s reference is quoted for NEXT SENTENCE, GnuCOBOL’s NEWS and configuration files for the compiler, and every run here used GnuCOBOL 3.2.
Flashcards · 7 cards
Recall it, then check
Say the answer to yourself before you look: recalling it is what makes it stick.
-
What is a golden file?
The output the legacy program prints today on a fixed input, kept. Every changed version runs on the same input, and its output is compared with the golden file line by line, with diff for example.
-
What does a characterisation test check?
That behaviour stays the same, not that it is right: it records what the program does now, so a refactoring that changes any result is caught.
-
Why must the input of a golden-file test hold a value on each limit of the rules?
A change at a limit shows only where the input meets it: the slip at exactly 10,000.00 changed one account's interest and the total, and nothing else.
-
Why refactor legacy code in small steps?
Make one kind of edit at a time and rerun the golden-file comparison after each one, so any difference points at the edit just made.
-
What is seeded synthetic data?
Test accounts that a program generates from a fixed seed instead of copying real ones, meeting every limit of the rules and the same on every run.
-
What do you do with behaviour that looks wrong while you refactor?
Keep it and write it down, then change it later in a change of its own, with a test that says the new behaviour is wanted. Changed during the refactoring, it could no longer be checked against the golden file.
-
What can the compiler point you at before a refactoring starts?
cobc -Wall names constructs GnuCOBOL treats as archaic or risky, and the listing's cross-reference marks items no statement uses: the places to read first. Build the golden file the way production builds the program.
Key takeaways
- A golden-file test records what the legacy program prints on a fixed input and compares every changed version with it, line by line; the slip at exactly 10,000.00 changed two lines and the comparison found both.
- The input of a golden-file test needs a value on each limit of the rules, because a change at a limit shows only where the input meets it.
- Golden files, seeded synthetic data, small steps, warnings and the production dialect each close a gap the others leave, and behaviour that looks wrong is recorded and changed later, in a change of its own.
Terms in this lesson
- characterisation test
- A test that records what code does today, right or wrong, so that a change meant to keep its behaviour can be checked against the record. A golden-file test is one.
- golden file
- The output of a program on a fixed input, saved so that every later version of the program can be run on the same input and compared with it line by line.
References
- GnuCOBOL 3.2 NEWS (FUNCTION RANDOM, -Wgoto-different-section, -Wsuspicious-perform-thru) (GnuCOBOL project (GNU))
- GnuCOBOL 3.2 Programmer's Guide (GnuCOBOL project (GNU))
- Enterprise COBOL for z/OS 6.5: RANDOM function (IBM)
- Enterprise COBOL for z/OS 6.5: GO TO statement (IBM)
- Enterprise COBOL for z/OS 6.5: CONTINUE statement (IBM)
- Enterprise COBOL for z/OS 6.5: IF statement (NEXT SENTENCE) (IBM)
- GnuCOBOL 3.2 ibm-strict.conf: the IBM dialect, included by ibm.conf (GnuCOBOL project (GNU))
Related tools
Report a problem with this lesson
Kept only in this browser. Your Learn progress