import inspect from collections import namedtuple from typing import Any, Dict, List import networkx as nx from exceptions import NodeException from .block import BaseBlock from .rule import ( EndpointExist, ExactlyOneInputAndOutput, GraphIsDAG, PortTypeMatch, RequiredInPortIsFit, Rule, TypeHasStrMethod, ) Edge = namedtuple("Edge", ["source", "sink", "port", "case"]) class Graph: def __init__( self, nodes: Dict[str, BaseBlock], edges: List[Edge], skip_validation: bool = False, ): super(Graph, self).__init__() self.g = nx.DiGraph() self.nodes = nodes for e in edges: self.g.add_edge(e.source, e.sink, port=e.port, case=e.case) if not skip_validation: self._validate( [ EndpointExist(), GraphIsDAG(), RequiredInPortIsFit(), ExactlyOneInputAndOutput(), PortTypeMatch(), TypeHasStrMethod(), ] ) def _reset(self): for n in self.g.nodes: if "data" in self.g.nodes[n]: del self.g.nodes[n]["data"] def _validate(self, rules: List[Rule]): for r in rules: r.check(self.g, self.nodes) def run_node(self, node_id) -> Any: node = self.nodes[node_id] in_edges = self.g.in_edges(node_id, data=True) if len(in_edges) == 0: try: return node() except Exception as e: raise NodeException(node_id) from e signature = inspect.signature(node.__call__) node_params = {} # fill default values for k, p in signature.parameters.items(): if p.default != inspect.Parameter.empty: node_params[k] = p.default # run upstreams and fill values for source_node, _, properties in in_edges: port = properties["port"] if "data" not in self.g.nodes[source_node]: self.g.nodes[source_node]["data"] = self.run_node(source_node) if port is None or port not in signature.parameters: # the None port and unknown port is special, it's required if self.g.nodes[source_node]["data"] is None: return None # if the edge has a filter, check if the value is match with the filter if ( properties["case"] is not None and properties["case"] != self.g.nodes[source_node]["data"] ): return None if port is not None: node_params[port] = self.g.nodes[source_node]["data"] elif self.g.nodes[source_node]["data"] is not None: if ( properties["case"] is not None and properties["case"] != self.g.nodes[source_node]["data"] ): continue node_params[port] = self.g.nodes[source_node]["data"] # check if required params are filled leak_params = set( [k for k, p in signature.parameters.items() if p.kind != p.VAR_KEYWORD] ) leak_params -= set(node_params.keys()) if len(leak_params) > 0: # some params leaked return None try: node_output = node(**node_params) if hasattr(node, "usage"): self.g.nodes[node_id]["usage"] = node.usage return node_output except Exception as e: raise NodeException(node_id) from e def input_type(self) -> type: input_nodes = [node for node in self.nodes.values() if node.is_input] assert len(input_nodes) == 1, "exactly one input node is required" signature = inspect.signature(input_nodes[0].input) return list(signature.parameters.values())[0].annotation def run(self, input: Any) -> str: self._reset() input_nodes = [node for node in self.nodes.values() if node.is_input] assert len(input_nodes) == 1, "exactly one input node is required" output_nodes = [ node_id for node_id, node in self.nodes.items() if node.is_output ] assert len(output_nodes) == 1, "exactly one output node is required" input_nodes[0].input(input) self.g.nodes[output_nodes[0]]["data"] = self.run_node(output_nodes[0]) return self.g.nodes[output_nodes[0]]["data"] @property def data(self): return dict( (node_id, node.get("data")) for node_id, node in self.g.nodes.items() ) @property def usage(self): return dict( (node_id, node.get("usage")) for node_id, node in self.g.nodes.items() if node.get("usage") )