Mock Mode

Mock mode lets you test pipelines that use bulker crates without a container runtime. Instead of running Docker or Apptainer, mock mode replays pre-recorded command outputs — stdout, stderr, return codes, and even newly created files. This makes it possible to run CI pipelines on machines where Docker is unavailable or impractical.

How it works

Mock mode has two phases:

1. Record — run your pipeline once with real containers. Bulker intercepts each containerized command, executes it normally, and captures its outputs to a JSON file.
2. Run — replay the recorded outputs without containers. Bulker creates lightweight Python shims that read the JSON file and reproduce the recorded behavior.

The recording shims sit in front of the real container commands on PATH. When a command runs, the shim delegates to the real container, captures stdout/stderr/returncode plus any new files created in the working directory, and appends the result to an outputs.json file. On replay, the shims match incoming arguments against recorded patterns and reproduce the captured output.

Recording outputs

Run your pipeline command through bulker mock record, specifying the crate and an output file:

bulker mock record databio/pepatac:1.0.13 outputs.json -- python pipeline.py sample1

This does the following:

1. Loads the manifest for databio/pepatac:1.0.13
2. Creates recording shims for every command in the crate
3. Prepends the shim directory to PATH
4. Runs python pipeline.py sample1 in that environment
5. Each time a crate command is invoked, the shim runs the real container command and appends the result to outputs.json

The outputs.json file is created automatically if it does not exist. If it already exists, new recordings are appended (not overwritten), so you can record multiple pipeline runs into the same file.

What gets recorded

For each command invocation, the recording captures:

Field	Description
args_pattern	Regex matching the arguments (escaped from the actual args)
stdout	Full standard output
stderr	Full standard error
returncode	Exit code
creates_files	Map of relative paths to file contents for any new files created in the working directory

Binary files that cannot be read as text are stored as "PLACEHOLDER_BINARY" and replayed as a single null byte.

outputs.json format

The file is a JSON object keyed by command name, where each value is an array of invocation records:

{
  "samtools": [
    {
      "args_pattern": "view\\ \\-h\\ input\\.bam",
      "stdout": "@HD\tVN:1.6\tSO:coordinate\n...",
      "stderr": "",
      "returncode": 0,
      "creates_files": {}
    }
  ],
  "bowtie2": [
    {
      "args_pattern": "\\-x\\ /data/genomes/hg38\\ .*",
      "stdout": "",
      "stderr": "10000 reads; of these:\n...",
      "returncode": 0,
      "creates_files": {
        "aligned.sam": "@HD\tVN:1.0\n..."
      }
    }
  ]
}

Replaying recorded outputs

Once you have an outputs.json, use bulker mock run to activate a mock environment:

bulker mock run databio/pepatac:1.0.13 outputs.json

This opens an interactive shell where every crate command is a Python shim that reads from outputs.json instead of running a container. You can then run your pipeline normally:

python pipeline.py sample1
# All containerized commands replay their recorded outputs
exit

Echo mode

For scripts and CI, use --echo to print the environment variables instead of launching a shell:

eval "$(bulker mock run databio/pepatac:1.0.13 outputs.json --echo)"
python pipeline.py sample1

This prints three export statements:

• PATH — prepended with the mock shim directory
• BULKER_MOCK_OUTPUTS — absolute path to the outputs.json file
• BULKERCRATE — set to mock:<crate>

Strict mode

The -s flag makes only mock commands available on PATH (no host commands):

bulker mock run -s databio/pepatac:1.0.13 outputs.json

This is useful for verifying that your pipeline does not depend on any commands outside the crate.

CI integration example

A typical CI workflow:

# .github/workflows/test.yml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install bulker
        run: curl -sL https://bulker.io/install.sh | bash

      - name: Run pipeline with mock outputs
        run: |
          eval "$(bulker mock run databio/pepatac:1.0.13 tests/outputs.json --echo)"
          python pipeline.py sample1

The outputs.json file is checked into the repository alongside tests. No Docker needed on the CI runner.

When to use mock mode

Good use cases:

• CI environments where Docker is unavailable or slow to start
• Testing pipeline logic (branching, file handling, error paths) without waiting for real containers
• Verifying that a pipeline invokes the expected commands with the expected arguments
• Fast local iteration on pipeline code

Not suitable for:

• Testing the actual container behavior or tool output
• Commands whose output varies based on input data not captured in the recording
• Pipelines that depend on container side effects beyond stdout/stderr/files (e.g., network calls, databases)

Limitations

• Argument matching is regex-based. The recording escapes the literal arguments into a regex pattern. If your pipeline calls the same command with different arguments, each invocation needs a separate recording. The first matching pattern wins.
• File creation is relative to cwd. Only new files created under the working directory are captured. Files written to absolute paths outside cwd are not recorded.
• Binary files are placeholders. Binary files are recorded as "PLACEHOLDER_BINARY" and replayed as a single null byte. They will not match the original binary content.
• No stdin capture. Interactive commands that read from stdin are not supported by the recording shims.
• Requires Python 3. The mock shims are Python scripts. Python 3 must be available on the replay machine.
• Recording requires real containers. You need a working container runtime to create the initial recording.
• Host commands are also mocked. Both commands and host_commands from the manifest get mock shims during replay.