Extension: LimitedParallel State

The LimitedParallel class allows concurrent processing, across arbitrary invocations of the same branch, but limited by a maximum number of concurrent executions.

This allows finer control over AWS Lambda invocations to avoid unexpected throttling by AWS Lambda (and associated timeouts, due to exceeding the maximum number of Lambda concurrent executions per second.

For more details on Parallel, see the AWS documentation.

class awssl.ext.LimitedParallel(Name=None, Comment='', InputPath='$', OutputPath='$', NextState=None, EndState=None, ResultPath='$', RetryList=None, CatcherList=None, BranchState=None, Iterations=0, MaxConcurrency=1, IteratorPath='$.iteration')

Limited Parallel allows a throttled amount of concurrent processing, constrained by the value of MaxConcurrent.

Each branch executed is the same, starting at BranchState, and the number of branch executions is specified by Iterations.

The value of the iterator is passed to each branch, so that Task``s in the branch can process appropriately.  The location of the iterator is specified by ``IteratorPath.

Either:

  • EndState is True and NextState must be None
  • EndState is False and NextState must be a valid instance of a class derived from StateBase.

Output is returned as a list of the outputs from each branch.

Parameters:
  • Name (str) – [Required] The name of the state within the branch of the state machine
  • Comment (str) – [Optional] A comment describing the intent of this pass state
  • InputPath (str) – [Optional] Filter on the Input information to be passed to the Pass state. Default is “$”, signifying that all the Input information will be provided
  • OutputPath (str) – [Optional] Filter on the Output information to be returned from the Pass state. Default is “$”, signifying that all the result information will be provided
  • EndState (bool) – [Optional] Flag indicating if this state terminates a branch of the state machine. Defaults to False
  • NextState (instance of class derived from StateBase) – [Optional] Next state to be invoked within this branch. Must not be None unless EndState is True
  • ResultPath (str) – [Optional] JSONPath indicating where results should be added to the Input. Defaults to “$”, indicating results replace the Input entirely.
  • RetryList – [Optional] list of Retrier instances corresponding to error states that cause the entire set of branches to be retried
  • CatcherList – [Optional] list of Catcher instances corresponding to error states that can be caught and handled by further states being executed in the StateMachine.
  • BranchState – [Required] StateBase instance, providing the starting state for each branch to be run concurrently
Type:

RetryList: list of Retrier
Type:

CatcherList: list of Catcher
Type:

BranchState: StateBase
Param:

Iterations: [Required] The total number of branches to be executed. Must be larger than zero
Type:

Iterations: int
Param:

MaxConcurrency: [Required] The maximum number of branches that are to executed concurrently. Must be larger than zero
Type:

MaxConcurrency: int
Param:

IteratorPath: [Required] The JSONPath in which to inject the iterator value into the Input passed to the branch
Type:

IteratorPath: str
clone(NameFormatString='{}')

Returns a clone of this instance, with the clone named per the NameFormatString, to avoid state name clashes.

If this instance is not an end state, then the next state will also be cloned, to establish a complete clone of the branch form this instance onwards.

Parameters:NameFormatString (str) – [Required] The naming template to be applied to generate the name of the new instance.
Returns:LimitedParallel – A new instance of this instance and any other instances in its branch.
get_branch_state()

Returns the initial state for the branch processing

Returns:StateBase – The initial state of the branch
get_iterations()

Returns the total number of branch executions to be performed

Returns:int – The total number of branch executions to perform
get_iterator_path()

Returns the injection JSONPath to be used to add the iterator value into the Input for a branch

Returns:str – The JSONPath for iterator value injection
get_max_concurrency()

Returns the maxiumum number of concurrent branch executions

Returns:int – The maximum number of concurrent branch executions
set_branch_state(BranchState=None)

Set the initial state for the branch processing

Parameters:BranchState – [Required] StateBase instance, providing the starting state for each branch to be run concurrently
Type:BranchState: StateBase
set_iterations(Iterations=0)

Sets the total number of branch executions to be performed. Must be larger than zero.

Param:Iterations: [Required] The total number of branches to be executed. Must be larger than zero
Type:Iterations: int
set_iterator_path(IteratorPath='$.iteration')

Sets the injection JSONPath to use to add the iterator value into the Input for a branch

Param:IteratorPath: [Required] The JSONPath in which to inject the iterator value into the Input passed to the branch
Type:IteratorPath: str
set_max_concurrency(MaxConcurrency=1)

Sets the maximum number of concurrent branch executions, and the supplied value must be greater than zero. Default is sequential processing.

Param:MaxConcurrency: [Required] The maximum number of branches that are to executed concurrently. Must be larger than zero
Type:MaxConcurrency: int
to_json()

Returns the JSON representation of this instance.

Returns:dict – The JSON representation
validate()

Validates this instance is correctly specified.

Raises Exception with details of the error, if the state is incorrectly defined.