An executor for the DAGs based on python backend.
Module Contents
-
plynx.plugins.executors.python.dag.POOL_SIZE = 3[source]
-
plynx.plugins.executors.python.dag.worker_main(job_run_queue: queue.Queue, job_complete_queue: queue.Queue)[source]
Main threaded function that serves Operations.
-
class
plynx.plugins.executors.python.dag.DAGParallel(node: Node)[source]
Bases: plynx.plugins.executors.dag.DAG
Python DAG scheduler.
- Args:
- node_dict (dict)
-
IS_GRAPH = True[source]
-
GRAPH_ITERATION_SLEEP = 0[source]
-
pop_jobs(self)[source]
Get a set of nodes with satisfied dependencies
-
_execute_node(self, node: Node)[source]
-
kill(self)[source]
Force to kill the process.
The reason can be the fact it was working too long or parent exectuter canceled it.
-
finished(self)[source]
Return True or False depending on the running status of the DAG.
-
class
plynx.plugins.executors.python.dag.DAG(node: Node)[source]
Bases: plynx.plugins.executors.dag.DAG
Base Executor class
-
IS_GRAPH :bool = True[source]
-
kill(self)[source]
Force to kill the process.
The reason can be the fact it was working too long or parent exectuter canceled it.
-
init_executor(self)[source]
Initialize environment for the executor
-
clean_up_executor(self)[source]
Clean up the environment created by executor
-
_apply_inputs(self, node)[source]
-
run(self, preview: bool = False)[source]
Main execution function.
-
class
plynx.plugins.executors.python.dag.ExecutorWithWebWorkerServer[source]
Bases: plynx.plugins.executors.python.dag.DAG
This executor is used for testing purposes only.
-
static
request_task(url, data)[source]
Send a request to the server
-
static
fire_and_forget(url, json)[source]
Fire and forget
-
launch(self)[source]
Launch the executor