The eight turns, as the tester typed them and the agent answered.
Copied from the session trace. Every answer is the agent of 2026-11-01, the day before the climb, so four of the eight are refusals and two of those rest on limits the data does not have. The Field notes say what each turn became.
Retirement eligibility by occupation, projected over the next several years, and a Groups/Divisions/Branches breakdown are not in the data. The records carry years of federal service but no age or birth date, so eligibility cannot be computed, and the data has no organizational hierarchy below the component and occupation series. The staffing plan and postings align to component and occupation only, so the hiring pipeline can be matched at that level and no lower.
As a proxy, CBP as a whole had 43,890 of 70,843 staff (62.0%) with 10 or more years of service in the August 2026 snapshot, including 20,476 (28.9%) with 20 or more. Long service indicates retirement exposure but does not establish eligibility, and the data has no forward projection. The nearest answerable variants are the tenure profile by occupation series for CBP (for example CBP Officers, 1895, and Border Patrol agents, 1896), or the retirement-category separation counts by occupation for recent months.
I read "all areas in CPB" as all CBP occupations as of the August 2026 snapshot; if you meant CBP Officers only, or a different component, say so.
CBP's workforce as of the August 2026 snapshot was 70,843, and 28.9% had 20 or more years of federal service (62.0% had 10 or more). That puts CBP second only to ICE on the 20-year share and second only to CISA on the 10-year share. Years of service is a proxy for retirement exposure, not retirement eligibility, which depends on age and retirement coverage, and the data has no forward projection.
CBP occupations, August 2026 (top 10 by headcount; the other 105 series are summed below the table)
| Series | Occupation | Headcount | With 20+ years | 20+ share |
|---|---|---|---|---|
| 1895 | CBP Officers | 27,650 | 8,732 | 31.6% |
| 1896 | Border Patrol agents | 22,464 | 5,513 | 24.5% |
| 0401 | Natural resources and biological sciences | 2,833 | 852 | 30.1% |
| 1801 | Inspection, investigation, enforcement and compliance | 2,247 | 910 | 40.5% |
| 0301 | Miscellaneous administration and program | 2,017 | 642 | 31.8% |
| 1802 | Compliance inspection and support (screeners) | 1,958 | 223 | 11.4% |
| 0343 | Management and program analysis | 1,446 | 478 | 33.1% |
| 2210 | IT management | 1,235 | 228 | 18.5% |
| 1889 | Import compliance | 1,120 | 405 | 36.2% |
| 0340 | Program management | 951 | 429 | 45.1% |
| All other 105 series | 6,922 | 2,064 | 29.8% |
I added the rows: the top 10 sum to 63,921 staff, and the remaining 105 series add 6,922, giving the 70,843 total.
CBP against the other components (all occupations, August 2026)
| Component | Headcount | 20+ years | 10+ years |
|---|---|---|---|
| CBP | 70,843 | 28.9% | 62.0% |
| ICE | 28,674 | 29.6% | 57.5% |
| TSA | 59,342 | 21.2% | 46.4% |
| USCIS | 20,327 | 21.1% | 59.8% |
| CISA | 2,204 | 22.9% | 65.8% |
| FEMA | 20,208 | 10.4% | 33.9% |
I read "CPB" as CBP, all occupations, as of August 2026, and the comparison as years of service across the six components; if you meant CBP Officers only, that is the 1895 row, and if you want age-based eligibility, it is not in the data.
WITH series_counts AS (
SELECT
workforce_snapshot.occupational_series_code,
workforce_snapshot.occupational_series,
COUNT(*) AS headcount,
SUM(CASE WHEN workforce_snapshot.length_of_service_years >= 20 THEN 1 ELSE 0 END) AS headcount_20_plus_years
FROM workforce.semantic.workforce_snapshot
WHERE workforce_snapshot.snapshot_month = DATE '2026-08-01'
AND workforce_snapshot.agency_subelement_code = 'HSBD'
AND workforce_snapshot.occupational_series_code IS NOT NULL
AND workforce_snapshot.occupational_series IS NOT NULL
GROUP BY
workforce_snapshot.occupational_series_code,
workforce_snapshot.occupational_series
)
SELECT
series_counts.occupational_series_code,
series_counts.occupational_series,
series_counts.headcount,
series_counts.headcount_20_plus_years,
SUM(series_counts.headcount) OVER () AS total_headcount_all_series
FROM series_counts
ORDER BY series_counts.headcount DESC, series_counts.occupational_series_code, series_counts.occupational_seriesDraft for OPM:
CBP Officers recorded 475 accessions against 594 planned hires in FY2026 Q1 (October to December 2025), a variance of -119 (-20.0%). Two findings bear on the gap: over the twelve months to August 2026 the cell separated 954 people, a rate of 0.0351 that is the same with and without the 2025 deferred resignation wave, and 586 of those separations (61%) were retirements; and only eight announcements with stated openings were open on August 1, 2026, totaling eight openings (three more say "Many"), against those 954 separations, a coverage of 0.8%. The quarter is not closed, so late OPM files will add accessions and the variance will narrow, and separation counts will also grow as late reports arrive. We request that CBP open enough CBP Officer announcements with stated openings to cover the 586 retirements in the trailing twelve months, and that the agency report the updated FY2026 Q1 accession count once later files are in.
I read this as the open cell, CBP Officers in FY2026 Q1, with attrition over the twelve months to August 2026; if you meant another cell or a different attrition window, say so.
The data cannot say why CBP Officer recruitment is thin or why announcements carry the openings they do. The records hold separation categories and posting openings, not reasons, and budget and appropriations are outside this review.
The figures already in this review show what "thin" means here. On August 1, 2026, eight CBP Officer announcements with stated openings were open, totaling eight openings, against 954 separations over the twelve months to August 2026, a coverage of 0.8%. Three more announcements say "Many" and are not counted, so the eight is a floor. Of the 954 separations, 586 were retirements.
Nearest answerable variants: the coverage ratio for CBP Officers at earlier dates, to show whether advertising has thinned over the year, or the separation categories by month, to show whether retirements are rising. I read "thin" as stated openings against separations; if you meant the number of announcements instead, say so.
On August 1, 2026, 11 CBP Officer announcements were open (the 1895 series, including supervisory and program-manager titles). Eight state a number of openings, totaling eight, and three say "Many." Five give no state and two list Virginia. Because the count is an upper bound, announcements filled or cancelled before their closing dates are not visible. I read this as announcements open on August 1, 2026; if you meant the number opened over the trailing twelve months, say so.
"Many" is the posting's own wording for an unstated number of openings, and the data holds only that word, so it cannot tell you whether a posting means two openings or two hundred. The three CBP Officer announcements that say "Many" carry control numbers 878058100 (grades 5 to 7), 878058600 (grade 9) and 875468500 (grade 13), and the openings figure is on each posting on USAJOBS. Until those are read, the eight stated openings are the only countable figure, and the 0.8% coverage is a floor.
The review's data has no applicant records, so it cannot show which positions applicants skip, and it has no grade field for staff to test whether higher grades are harder to fill. Posting counts do not measure difficulty either: an announcement can be open for weeks and still draw applicants.
The nearest answerable view is the grades on the 11 CBP Officer announcements open on August 1, 2026. Eight state openings and sit at grades 12 to 14 (three at 14, three at 13, two at 12). The three "Many" announcements sit at grades 5 to 7, 9 and 13, so the only lower-grade announcements open that day have no stated openings. I read "hard to fill" as openings advertised without applicant data; if you want the advertised grade mix over time, say so.
From September 2025 to October 2026, 463 CBP postings listed the CBP Officer series: 337 (73%) at grades 13 to 14, 94 (20%) at grades 5 to 12, and 32 (7%) at grade 15 and above. The 13 to 14 band was the largest in every month, from 58% in March 2026 to 86% in December 2025. Stated openings were 708 in total, but postings that say "Many" count as zero, so openings are understated.
| Period (months) | Postings | Grades 5 to 12 | Grades 13 to 14 | Grade 15 and above | Stated openings |
|---|---|---|---|---|---|
| September 2025 | 37 | 11 | 25 | 1 | 107 |
| October to December 2025 (FY2026 Q1) | 86 | 14 | 71 | 1 | 132 |
| January to March 2026 (FY2026 Q2) | 78 | 18 | 53 | 7 | 113 |
| April to June 2026 (FY2026 Q3) | 104 | 21 | 71 | 12 | 156 |
| July to August 2026 | 79 | 18 | 54 | 7 | 101 |
| September 2026 | 45 | 4 | 38 | 3 | 62 |
| October 2026 | 34 | 8 | 25 | 1 | 37 |
The data shows what was advertised and at which grades. It does not show why, and it cannot answer the Director's other questions. There are no applicant records, no performance ratings, no training or competency records, and no separation reasons beyond the four categories. Comparison with agencies outside DHS is outside this review. The closest in-scope comparison is the other DHS components, which I can set against CBP on years of service. To close the gaps, the analyst could ask OPM and the agency for applicant flow by posting (applicants, referred, selected), grade on staff records so staff and posting grade mix can be compared, performance rating distributions and training completion by series and grade, and reasons recorded at separation.
I read "grade mix over time" as the grade band of CBP postings for the CBP Officer series, by month of opening; if you meant the grade mix of staff, the review's calculations do not report staff by grade.
WITH `filtered` AS (
SELECT
DATE_TRUNC('MONTH', `open_date`) AS `open_month`,
CASE
WHEN CAST(`minimum_grade` AS INT) BETWEEN 5 AND 9 THEN '5 to 9'
WHEN CAST(`minimum_grade` AS INT) BETWEEN 10 AND 12 THEN '10 to 12'
WHEN CAST(`minimum_grade` AS INT) BETWEEN 13 AND 14 THEN '13 to 14'
WHEN CAST(`minimum_grade` AS INT) >= 15 THEN '15 and above'
ELSE NULL
END AS `minimum_grade_band`,
COALESCE(`total_openings`, 0) AS `stated_openings`
FROM `workforce`.`semantic`.`job_posting`
WHERE `agency_subelement_code` = 'HSBD'
AND `open_date` >= DATE '2025-09-01'
AND `open_date` < DATE '2026-11-01'
AND `open_date` IS NOT NULL
AND `minimum_grade` IS NOT NULL
AND array_contains(`occupational_series_codes`, '1895')
), `grouped` AS (
SELECT
`open_month`,
`minimum_grade_band`,
COUNT(*) AS `postings`,
SUM(`stated_openings`) AS `total_openings`
FROM `filtered`
WHERE `minimum_grade_band` IS NOT NULL
GROUP BY `open_month`, `minimum_grade_band`
)
SELECT
`open_month`,
`minimum_grade_band`,
`postings`,
`total_openings`,
SUM(`postings`) OVER (PARTITION BY `open_month`) AS `month_total_postings`
FROM `grouped`
ORDER BY
`open_month`,
CASE `minimum_grade_band`
WHEN '5 to 9' THEN 1
WHEN '10 to 12' THEN 2
WHEN '13 to 14' THEN 3
WHEN '15 and above' THEN 4
ELSE 5
ENDchallenge/eval/sessions/2026-11-01-tester-a/trace.jsonl · label, tools, seconds and cost per turn are the trace fields
The audio, in order
Eighty minutes on a webcam microphone, transcribed with speaker labels and redacted to roles. Long silences are the tester working alone in the app. Timestamps are minutes into the recording.
| Time | What happened | Heard |
|---|---|---|
| 00:00 | Setup. The facilitator asks for think-aloud and promises to stay quiet. | "I learned from what I get wrong." |
| 03:00 | Task 1, the brief. Reads 475 onboarded as under goal. Calls the attrition high against a remembered norm of about 4%. The first questions are why people quit and whether there was an exit process. Jumps ahead to the recommended cells; redirected twice. Has not heard of deferred resignation. | "The hiring pipeline is working because you're still growing." Wants retirement eligibility by the end of next fiscal year and five years out. Notices the brief says exposure, not eligibility. |
| 12:00 | The assistant. Types the retirement question (A01). Works mostly in silence. Spoken questions on clearances and whether hiring correlates with retirement; clearance data is not in scope. Asks whether FEMA was included. | 21:45, on the component comparison (A02): "This is actually great data." Needed to be told to scroll up to find the answer. |
| 28:20 | Task 2, three cells for the review. Spoken, by size. | CBP, USCIS, TSA. |
| 30:20 | Task 3, the paragraph. The facilitator suggests the AI draft (A03). The tester asks why hiring is thin instead (A04), then types alone for most of thirty minutes. Reads the applicant-data refusal (A07) and asks for grade mix over time (A08). | "I wouldn't want to just put in here what my thoughts are. I would want to know why there's such thin hiring." |
| 59:14 | Reads the paragraph written by hand: the numbers are the what, the director wants the why. Wants comparisons with like-sized organizations and more OPM detail. Then reads the grade-mix answer and finds it useful. | "The numbers are the what. The why is what the director will be interested in." On A08: "This would be helpful." "I like AI." |
| 64:00 | Task 4, the recommendations. Accepts the hiring-committee item, likes the eligibility projections, argues retirement is the bigger issue and prior years could anticipate departures. Skips the optional "add what you know." | "Am I saying accept?" |
| 66:20 | Debrief. Trusted least: "many," because it is not specific. Trust but verify; would want to sit and ask more. Not technical; the SQL under a Verified answer meant nothing. A director would ask where the numbers came from, where the problems are, how long a fix takes, and about skills mix. | "I didn't trust many." "I'm kind of a trust but verify person." |
| 71:43 | What to show a panel: look at the problem systematically; the numbers indicate, an organizational assessment confirms; is OPM collecting the right data for models like this. | "I love the way they answer the questions and they tell you what you can't get." "It helps you understand the questions that you need to ask." |
| 74:00 | Asked to be brutal, the tester turns it on themselves. Then the real criticism: HR directors can already drill down internally; OPM is the big picture; the useful version takes the agency's own numbers and answers whether a gap is skills mix or recruitment. | "It's my issue, not the system's." "I could plug my numbers in." Facilitator, closing: "We need more data." |
wiki/sources/2026-11-01-tester-a-session-transcript.md, redacted · wiki/analyses/tester-a-session-findings.md