Observation Execution Tool

Project description

The ska-oso-oet project contains the code for the Observation Execution Tool (OET), the application which provides on-demand Python script execution for the SKA.

Overview

The core of the OET is a script execution engine which runs a requested script in a child Python process. The engine supervises script execution, in that it can terminate the script at any time when requested, and captures the output and/or errors generated by the script for inspection by a (remote) client.

A REST layer makes the Python API for the script execution engine available via REST over HTTP. This project also contains a command line client to allow users to submit script execution requests to a remote OET backend.

The REST layer is made up of two components that work together to provide the remote script execution functionality:

  • The OET REST server maintains a list of the scripts that have been loaded and their current state. The server implements the interface specified by the OET Module View: REST API.

  • The OET OET command line tool provides a Command Line Interface (CLI) to the OET backend.

More details on the OET architecture can be found in C&C view: OET client and OET backend.

Note

SKA control scripts are not packaged as part of this project. The repository of observing scripts executed by the OET can be found in the ska-oso-scripting project.

Quickstart

Build a new OET image:

make oci-build

Execute the test suite and lint the project with:

make python-test
make python-lint

Format and lint on commit

We recommend you use pre-commit to automatically format and lint your commits. The commands below should be enough to get you up and running. Reference the official documentation for full installation details.

Pre-commit installation on Linux

# install pre-commit
sudo pip3 install pre-commit

# install git hook scripts
pre-commit install

# uninstall git hook scripts
pre-commit uninstall

Pre-commit installation on MacOS

The commands below were tested on MacOS 10.15.

# install pre-commit
pip3 install --user pre-commit

# install git hook scripts
~/Library/Python/3.8/bin/pre-commit install

# uninstall git hook scripts
~/Library/Python/3.8/bin/pre-commit uninstall

Makefile targets

This project extends the standard SKA Make targets with a few additional Make targets that can be useful for developers. These targets are:

Makefile target

Description

dev-up

deploy the OET using the current developer image, exposing REST ingress on the host

dev-down

tear down the developer OET deployment

rest

start the OET backend in a Docker container

diagrams

recreate PlantUML diagrams whose source has been modified

k8s-chart-test

run helm chart unit tests (note: requires helm unittest plugin: https://github.com/quintush/helm-unittest )

help

show a summary of the makefile targets above

Local development with k8s

OET REST server can be deployed locally using Helm and Kubernetes and OET CLI OET command line tool can be used to communicate with the server. OET CLI is installed as part of the Poetry virtual environment (see README) or can be used inside a running OET container/pod.

If using OET CLI within Poetry virtual environment these steps are needed for the CLI to access the REST server:

  • set rest.ingress.enabled to true in charts/ska-oso-oet/values.yaml

  • set OET_URL environment variable with export OET_URL=http://<minikube IP>/<kube namespace>/oet/api/v<OET major version>

To deploy OET REST server run

make k8s-chart-install && make k8s-wait

Feature flags

OET feature flags are configured via environment variables and configuration files. The configuration file, ska_oso_oet.ini, can be located either in the user’s home directory, or the root of the installation folder.

Feature flags are read in this order:

  1. environment variable;

  2. ska_oso_oet.ini configuration file;

  3. default flag value as specified in OET code.

No feature flags are available at this time.