feat: publish emulator image directly from testing library

[SilanHe]

Issue #, if available:

Description of changes: Create emulator image from the python testing library. For each release, we should update the version in the pyproject.poml file in the emulator folder. The image tag will be the aws-durable-sdk-python-testing version with v prefixing it, e.g. v1.1.1.

Dependencies

If this PR requires testing against a specific branch of the Python Language SDK (e.g., for unreleased changes), uncomment and specify the branch below. Otherwise, leave commented to use the main branch.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[bchampp]

🤖 Emulator PR Created

A draft PR has been created with locked dependencies:

➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401

The emulator will build binaries using the exact testing SDK commit locked in uv.lock.

[bchampp]

🔄 Emulator PR Updated

The emulator PR has been updated with locked dependencies:

➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401

[bchampp]

🔄 Emulator PR Updated

The emulator PR has been updated with locked dependencies:

➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401

[bchampp]

🔄 Emulator PR Updated

The emulator PR has been updated with locked dependencies:

➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401

[bchampp]

🔄 Emulator PR Updated

The emulator PR has been updated with locked dependencies:

➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401

[bchampp]

🔄 Emulator PR Updated

The emulator PR has been updated with locked dependencies:

➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401

[bchampp]

🔄 Emulator PR Updated

The emulator PR has been updated with locked dependencies:

➡️ https://github.com/aws/aws-durable-execution-emulator/pull/401

[yaythomas]

suggestion: could simplify by putting emulator under the src, so that the source lives here:

src/emulator/

this then goes into the pyproject.toml in root:

[project.scripts]
dex-local-runner = "aws_durable_execution_sdk_python_testing.cli:main"
dex-emulator = "aws_durable_execution_sdk_python_testing.emulator.server:main"

however, all of that said, does the emulator server + factory + config actually add anything that the existing cli code doesn't already do?

so, you can start the local runner like this, and I think this is equivalent, or am I missing something:

dex-local-runner start-server \
  --host 0.0.0.0 \
  --port 5000 \
  --log-level INFO \
  --lambda-endpoint http://localhost:3001 \
  --store-type sqlite \
  --store-path ./durable-executions.db

Alternatively, these $envs - these are not the same as in emulator config though, which just has HOST, PORT, LOG, EXECUTION_STORE_TYPE etc:

# Set these once:
export AWS_DEX_HOST=0.0.0.0
export AWS_DEX_PORT=5000
export AWS_DEX_LOG_LEVEL=INFO
export AWS_DEX_LAMBDA_ENDPOINT=http://localhost:3001
export AWS_DEX_STORE_TYPE=sqlite
export AWS_DEX_STORE_PATH=./durable-executions.db

# Then just run:
dex-local-runner start-server

These emulator entries do not look like they're used outside of the emulator itself, though, because https://github.com/aws/aws-sam-cli/blob/f466a449eb5137dafa002b68e97ed0b03a8feafa/samcli/local/docker/durable_functions_emulator_container.py#L39-L75

Port: User can override via DURABLE_EXECUTIONS_EMULATOR_PORT env var
Store type: User can override via DURABLE_EXECUTIONS_STORE_TYPE env var
Host: HARDCODED to 0.0.0.0 (no user override)
Data dir: HARDCODED to .durable-executions-local in cwd (no user override)

The Dockerfile would then look something like this:

FROM python:3.13-slim

# Copy and install the wheel
COPY dist/*.whl /tmp/
RUN pip install --no-cache-dir /tmp/*.whl && rm -rf /tmp/*.whl

# AWS credentials (required for boto3)
ENV AWS_ACCESS_KEY_ID=foo \
    AWS_SECRET_ACCESS_KEY=bar \
    AWS_DEFAULT_REGION=us-east-1

EXPOSE 9014

HEALTHCHECK --interval=10s --timeout=3s --start-period=5s \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:9014/health').read()" || exit 1

CMD ["dex-local-runner", "start-server", \
     "--host", "0.0.0.0", \
     "--port", "9014", \
     "--log-level", "DEBUG", \
     "--lambda-endpoint", "http://host.docker.internal:3001", \
     "--store-type", "sqlite", \
     "--store-path", "/tmp/.durable-executions-local/durable-executions.db"]

The version number of the emulator container then becomes the SAME as the testing lib version number:

name: Build and Push Docker Image

on:
  release:
    types: [published]

jobs:
  docker:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.13'
      
      - name: Install hatch
        run: pip install hatch
      
      - name: Build wheel with hatch
        run: hatch build --target wheel
      
      - name: Get version from __about__.py
        id: version
        run: |
          VERSION=$(python -c "from src.aws_durable_execution_sdk_python_testing.__about__ import __version__; print(__version__)")
          echo "VERSION=${VERSION}" >> $GITHUB_OUTPUT
      
      - name: Build and push Docker image
        run: |
          VERSION=${{ steps.version.outputs.VERSION }}
          docker build -t public.ecr.aws/your-repo/dex-local-runner:${VERSION} .
          docker tag public.ecr.aws/your-repo/dex-local-runner:${VERSION} public.ecr.aws/your-repo/dex-local-runner:latest
          docker push public.ecr.aws/your-repo/dex-local-runner:${VERSION}
          docker push public.ecr.aws/your-repo/dex-local-runner:latest

in summary: you could simplify this by getting rid of emulator directory entirely and just injecting the appropriate args via the Dockerfile.

[zhongkechen]

nit: not the latest.

[zhongkechen]

nit: rm wouldn't reduce image size

[yaythomas]

you could use a bind mount

RUN --mount=type=bind,source=dist,target=/tmp/dist pip install --no-cache-dir /tmp/dist/*.whl

(then you don't need to COPY before)

[yaythomas]

for posterity, which bug in hatch? so we know when/what to fix in future

[SilanHe]

I couldn't easily find any mention of this but I added this/experienced this on Feb 25th where you can see, there were multiple updates to virtualenv version to try to fix this: https://pypi.org/project/virtualenv/#history. I will see if the tests work without the newest version released on Feb 26.

[yaythomas]

why? we're only interested in the whl hatch builds in the previous step?

if it's to extract the version number, can do that without installing like this:

VERSION=$(python -c "import sys; sys.path.insert(0, 'src'); from aws_durable_execution_sdk_python_testing.__about__ import __version__; print(__version__)")

or even just grep it:

VERSION=$(grep "^__version__" src/aws_durable_execution_sdk_python_testing/__about__.py | cut -d'"' -f2)

[SilanHe]

yeah you're right, it was only to fetch the the version.

[yaythomas]

without

EXPOSE 9014

consumers will have to specify -p at run docker run -p 9014:9014 arb/img-name

[SilanHe]

I can include EXPOSE 9014.

[yaythomas]

could be a nice addition, but maybe not essential:

HEALTHCHECK --interval=10s --timeout=3s --start-period=5s \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:9014/health').read()" || exit 1

[SilanHe]

I did not include this since the SAM CLI already does this healthcheck. In addition, this was causing the image to fail when running through the SAM CLI. As it's not essential, I opted to not use my time there.

[yaythomas]

maybe standardize to a v in front?

PER_ARCH_IMAGE_TAG: "v${{ steps.version.outputs.VERSION }}-${{ matrix.arch }}"

so
Standard format is v1.2.3-arm64 not arm64v1.2.3.

[SilanHe]

yeah I can do this

[yaythomas]

Dockerfile rather than DockerFile.
happens to work because docker -f is not case sensitive.

[SilanHe]

I think it is case sensitive. DockerFile is the name of the docker file.

[yaythomas]

https://docs.docker.com/build/concepts/dockerfile/

standard is Dockerfile

"The default filename to use for a Dockerfile is Dockerfile, without a file extension. Using the default name allows you to run the docker build command without having to specify additional command flags."

[yaythomas]

https://docs.docker.com/build/concepts/dockerfile/

standard is Dockerfile

"The default filename to use for a Dockerfile is Dockerfile, without a file extension. Using the default name allows you to run the docker build command without having to specify additional command flags."