Spring Batch

Spring Batch is a framework for building robust batch processing applications in Java.

Executing large volumes of data tasks without user interaction, typically scheduled/offline.

ETL, file processing, reporting, reconciliation, billing, data migration.

Top-level container representing an entire batch process.

A phase within a job that performs a specific processing task.

Logical job run identified by job name + identifying JobParameters.

A single attempt to run a JobInstance.

Execution metadata for one step attempt within a job execution.

Input parameters used to launch and identify job instances.

They control job identity, scheduling semantics, and restart behavior.

Component used to start jobs programmatically.

Persistent store for batch metadata (executions, statuses, contexts).

To track progress, failures, and enable restartability.

Read-only API to inspect job/step execution metadata.

Higher-level API for starting/stopping/restarting jobs.

Read-process-write pattern in chunks within transactional boundaries.

ItemReader, ItemProcessor, ItemWriter.

Reads one item at a time from source.

Transforms/validates/filter items between read and write.

Writes processed items to target system.

Number of items processed/written per transaction.

Affects memory usage, throughput, and rollback scope.

A step executing custom logic once (or repeat loop) rather than chunk pipeline.

For simple operations: cleanup, file move, trigger task, pre/post checks.

Builder API to configure step behavior.

Builder API to define jobs and step flow.

Step/job outcome status used in flow decisions.

Execution state lifecycle (STARTING, STARTED, COMPLETED, FAILED, etc.).

Persistent key-value state storage for job/step executions.

Supports restart from last checkpoint and state sharing.

Stored progress marker allowing restart near failure point.

Ability to rerun failed job from saved state rather than from scratch.

Only if designed/configured for restart and state consistency.

Utility adding/incrementing run.id parameter to create new instances.

Prevent accidental “job instance already complete” conflicts.

Reader for line-based flat files (CSV/fixed width, etc.).

Writer for flat files with configurable formatting.

Reads DB rows via cursor.

Reads DB rows page by page for scalable large datasets.

Cursor can hold long connection; paging offers chunked retrieval/control.

JPA-based paging reader for entities.

Reader backed by Spring Data repository methods.

Chains multiple processors in sequence.

Delegates writes to multiple writers.

Returning null to skip writing an item (semantic filtering).

Ignoring specific recoverable item-level failures.

Re-attempting failed item processing/writing for transient errors.

Max number of skippable exceptions allowed before step fails.

Max retry attempts before treating as failure.

Hook interface for lifecycle callbacks (job/step/chunk/read/process/write).

Auditing, metrics, custom logging, notifications, resource handling.

Callback before/after job execution.

Callback before/after step execution.

Callbacks around chunk processing boundaries.

Each chunk typically processed in one transaction.

On failure, current chunk transaction rolls back.

Safe reprocessing without duplicate harmful side effects.

Putting all logic in one huge step/job without clear boundaries.

Track job start/end, status, counts, and failure reason.

Design small restartable steps with clear inputs/outputs.

Conditional transitions between steps based on exit status.

Use transition rules (on/to) with statuses.

Custom flow decision component based on runtime state.

Parallel execution of independent step flows.

Embedding a flow as a single step in a larger job.

Launching another job as a step.

Splitting one step into multiple parallel partitions over data ranges.

Manager step distributes partition work to remote workers.

Reader/processor/writer responsibilities distributed via messaging.

Partitioning splits data domain; multithreaded step parallelizes within one step instance.

Executor enabling asynchronous/multi-threaded step processing.

Controls concurrency level for parallel processing.

Avoid DB/resource saturation and contention.

Component with open/update/close for stateful checkpointing support.

Persist reader/writer state for restartability.

Controls whether reader/writer stores restart state.

Stateless/idempotent scenarios or when metadata overhead is unnecessary.

Promotes step context values to job context for later steps.

Defers bean creation so step/job parameters can be injected at runtime.

Bean scope tied to step execution lifecycle.

Bean scope tied to job execution lifecycle.

Enable parameterized/stateful components per execution.

Enables skip/retry and related fault-handling policies.

Custom logic deciding whether exception should be skipped.

Custom logic controlling retry eligibility and attempts.

Controls delay strategy between retries.

Prevents hammering unstable dependencies.

Exceptions that should not trigger transaction rollback.

Can compromise consistency if misapplied.

Callback when item is skipped during read/process/write.

Auditability and later reconciliation/replay.

Store permanently failed items for later inspection/reprocessing.

Validate input and reject/filter invalid items early.

Routes items to different writers based on classification logic.

Reads from multiple files/resources in sequence.

Item carrying source resource metadata for diagnostics/routing.

Wrapper improving thread safety for non-thread-safe readers.

They keep mutable cursor/state internally.

Only failed/incomplete steps rerun depending job configuration and instance state.

Allows step rerun even if previously completed.

Limits number of times a step can start.

Prevent endless retries on repeatedly failing steps.

Determines whether parameter contributes to JobInstance identity.

Pass runtime hints without creating new logical instance.

Set of BATCH_* tables storing job/step execution info and contexts.

Yes, with proper naming/prefix/management strategy.

Changing default BATCH_ prefix for metadata tables.

Controls transaction isolation when creating job execution records.

Concurrent launch attempts for same JobInstance parameters.

Unique job parameters + repository constraints + scheduler coordination.

Use cron/scheduler/orchestrator to trigger jobs.

Central visibility, retries, dependency management, calendars.

Unit test processors + integration test full step/job with sample datasets.

Force mid-step failure, rerun, verify resumed progress and correctness.

Write metadata lines or validate file boundaries.

Special handling when reading from transactional resources like JMS.

Ignoring item-level metrics and only tracking final job status.

Track read/process/write/skip/retry counts, chunk durations, and error taxonomy.

Clarifies expected completion window and operational alerts.

Input schema/quality changes over time can silently break processing.

Validation steps, schema checks, anomaly metrics, canary runs.

Tune chunk size, fetch size, commit interval, and parallelism carefully.

Idempotent writes + checkpointed restart + bounded retry/skip policies.

Team can recover failed jobs predictably without manual data corruption.

Design for restart, observability, and controlled fault tolerance from day one.

Separate ingestion, processing, and output concerns with explicit backpressure controls.

Primary goal is total completion throughput; latency matters per SLA checkpoints.

Balance transaction overhead against rollback cost and memory footprint.

Large rollback scope, memory pressure, long lock duration.

Excess transaction overhead and lower throughput.

Choose evenly distributed stable keys minimizing skew/hot partitions.

Uneven data distribution causing straggler partitions and poor parallel efficiency.

Dynamic partitioning, range rebalancing, or workload-aware partition keys.

Retries/restarts can duplicate writes unless idempotency/dedup applied.

Upsert keys, unique constraints, checksum tables, processed-item ledgers.

Follow-up job verifying source/target counts and data correctness.

Detects silent partial failures and data quality issues.

Updating DB and external system separately can diverge on failure.

Transactional outbox or staged writes with replay mechanisms.

Track last processed timestamp/id/version to process only new changes.

Clock skew/out-of-order events can miss or duplicate records.

Use overlap window + deduplication.

Reprocess windows or correction jobs for delayed records.

Processing historical data to populate or correct datasets.

Competes with daily jobs for resources and can breach SLAs.

Throttle, isolate resources, and schedule during low-load windows.

Ensure tenant data/process failures remain isolated.

Separate queues/partitions/schemas/resources and per-tenant limits.

Invalid persisted state causing incorrect restart behavior.

Versioned context schema, validation, and safe reset/restart procedures.

High job concurrency can overload Batch metadata repository.

DB tuning, indexing, cleanup policies, and controlled job launch concurrency.

Archiving/purging old execution records to maintain performance.

Retain enough history for audits/debugging before deletion.

Per-step metrics, structured logs, traceable item errors, SLA dashboards, alerting.

Throughput/sec, chunk duration, retry/skip rates, lag, completion time percentile.

Difference between expected processing point and actual completed point.

Detect unusual volume/error/runtime changes automatically.

Inject DB/network/file failures to verify restart and fault policies.

Operational teams need proven recovery runbooks before incidents.

Avoid duplicate concurrent execution of same logical job instance.

Leader election/locks/scheduler control/idempotent job parameters.

Coordinated cutover so only one environment triggers jobs.

Input/output schema changes can break readers/writers/processors.

Versioned contracts, compatibility layers, and staged rollout.

Least privilege DB/file access, secret rotation, encrypted transport/storage.

Mask/redact sensitive fields and enforce retention/access controls.

Autoscaling workers and right-sizing resources per job window.

Lower cost but higher interruption risk; requires robust restartability.

Treating restart/idempotency as optional instead of core design requirements.

Deterministic, restartable, observable pipelines with controlled failure recovery.

Benchmark with realistic volumes and tune chunking/parallelism empirically.

Assume every dependency can fail; design retries/skips/checkpoints intentionally.

Automate runbooks, alerts, and reconciliation—manual heroics do not scale.

Spring Batch excellence means predictable correctness at scale, not just job completion.