There are a handful of files that are required for a Klio job, which the klio job create command automatically generates for you.
klio job create
$ tree . ├── Dockerfile ├── MANIFEST.in ├── README.md ├── __init__.py ├── job-requirements.txt ├── klio-job.yaml ├── run.py ├── setup.py ├── test_transforms.py └── transforms.py 0 directories, 8 files
run.py and transforms.py jointly define the logic of your job (the pink box of “your code” in the job overview diagram).
run.py
transforms.py
test_transforms.py gives you a place to test the logic of your job.
test_transforms.py
__init__.py helps the Python executable find the path where the other Python files are.
__init__.py
setup.py file defines how the job should be packaged so the configured runner can appropriately install it. This is also where job-specific system-level dependencies should be declared. See setup.py for more information.
setup.py
job-requirements.txt is a standard pip requirements file where you declare the Python packages needed in your Klio job.
job-requirements.txt
pip
Dockerfile describes a Docker image that will be used for launching a Klio job, regardless of the configured runner. It is setup to install the packages you specify in job-requirements.txt.
Dockerfile
klio-job.yaml contains configuration that tells a runner how to run your job. It also stores Klio job-level information like event and data I/O configuration, metrics, etc. Read more about a job’s configuration here.
klio-job.yaml
MANIFEST.in declares files needed for job installation (i.e. job-requirements.txt) but not needed at runtime. See MANIFEST.in for more information.
MANIFEST.in
README.md is initially populated with setup instructions on how to get a job running, but should be used however needed.
README.md