Repository navigation
Conversation
Add a single combined runner that generates the daily partner files and retries pending delivery/archive steps. Define a suspended-by-default Vancouver CronJob at 19:00 through 23:00 and allow environments to activate it with FLAT_FILE_JOB_SUSPEND=false. Add an S3 connectivity test command and unit coverage for runner sequencing and error handling.
Prefix flat-file archives and the S3 connectivity-test object with address_change/ so both are grouped under the requested bucket folder. Update tests and docs to match.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The job pods lack database ingress access when run in disposable or PR environments; they need a distinct worker label allowed by the database policy to avoid routing backend HTTP traffic to them.
1 open issue
1 file not reviewed
backend/package-lock.json: Generated file
What changed in this PR
Adds scheduled partner flat-file processing and an S3 connectivity check, using the backend’s existing generation, delivery, and archival services.
Changes:
- Adds a combined generation-and-retry runner with unit coverage.
- Defines a suspended-by-default CronJob running hourly from 19:00 through 23:00 Vancouver time, controlled through deployment configuration.
- Adds an S3 upload/download/cleanup check, documents its use, and updates the archive prefix.
| File | Description |
|---|---|
docs/src/content/docs/flat-file-s3-test.mdx |
Documents S3 verification and required configuration. |
docs/astro.config.mjs |
Adds the S3 guide to navigation. |
backend/src/flat-file/test-s3-upload.ts |
Implements S3 round-trip verification and cleanup. |
backend/src/flat-file/test-s3-upload.spec.ts |
Tests verification and failure handling. |
backend/src/flat-file/s3.transport.ts |
Adds download/delete operations and updates the object prefix. |
backend/src/flat-file/run-flat-files.ts |
Adds the combined runner’s application entry point. |
backend/src/flat-file/flat-file.service.spec.ts |
Updates expected S3 object paths. |
backend/src/flat-file/flat-file.job.ts |
Sequences generation and pending transfers. |
backend/src/flat-file/flat-file.job.spec.ts |
Tests runner sequencing and error handling. |
backend/package.json |
Adds runner and S3 verification commands. |
backend/package-lock.json |
Refreshes dependency classification metadata. |
backend/openshift.deploy.yml |
Adds the CronJob and shared, Secret-backed environment configuration. |
.github/workflows/reusable-deploy.yml |
Passes the environment’s job suspension setting. |
Review effort: Balanced
Label scheduled batch pods separately from HTTP backends and allow same-preview workers through the disposable database ingress policy. Add offline selector tests for worker access, cross-preview isolation, and exclusion from backend Service routing.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Add a single combined runner that generates the daily partner files and retries pending delivery/archive steps. Define a suspended-by-default Vancouver CronJob at 19:00 through 23:00 and allow environments to activate it with FLAT_FILE_JOB_SUSPEND=false.
Add an S3 connectivity test command and unit coverage for runner sequencing and error handling.
Description
Please provide a summary of the change and the issue fixed. Please include relevant context. List dependency changes.
Fixes # (issue)
Type of change
How Has This Been Tested?
Checklist
Further comments