Extension: LimitedParallel State¶

The LimitedParallel class allows concurrent processing, across arbitrary invocations of the same branch, but limited by a maximum number of concurrent executions.

This allows finer control over AWS Lambda invocations to avoid unexpected throttling by AWS Lambda (and associated timeouts, due to exceeding the maximum number of Lambda concurrent executions per second.

For more details on Parallel, see the AWS documentation.

class awssl.ext.LimitedParallel(Name=None, Comment='', InputPath='$', OutputPath='$', NextState=None, EndState=None, ResultPath='$', RetryList=None, CatcherList=None, BranchState=None, BranchRetryList=None, Iterations=0, MaxConcurrency=1, IteratorPath='$.iteration')¶

Limited Parallel allows a throttled amount of concurrent processing, constrained by the value of MaxConcurrent.

Each branch executed is the same, starting at BranchState, and the number of branch executions is specified by Iterations.

The value of the iterator is passed to each branch, so that Task``s in the branch can process appropriately. The location of the iterator is specified by ``IteratorPath.

Branch executions have optional Retrier lists, which allow individual executions to be retried. In addition, the state also supports retries and catches, but this will result in all branches being re-executed.

Either:

EndState is True and NextState must be None
EndState is False and NextState must be a valid instance of a class derived from StateBase.

Output is returned as a list of the outputs from each branch.

Parameters:	Name (str) – [Required] The name of the state within the branch of the state machine Comment (str) – [Optional] A comment describing the intent of this pass state InputPath (str) – [Optional] Filter on the Input information to be passed to the Pass state. Default is “$”, signifying that all the Input information will be provided OutputPath (str) – [Optional] Filter on the Output information to be returned from the Pass state. Default is “$”, signifying that all the result information will be provided EndState (bool) – [Optional] Flag indicating if this state terminates a branch of the state machine. Defaults to `False` NextState (instance of class derived from `StateBase`) – [Optional] Next state to be invoked within this branch. Must not be `None` unless `EndState` is `True` ResultPath (str) – [Optional] JSONPath indicating where results should be added to the Input. Defaults to “$”, indicating results replace the Input entirely. RetryList – [Optional] `list` of `Retrier` instances corresponding to error states that cause the entire set of branches to be retried CatcherList – [Optional] `list` of `Catcher` instances corresponding to error states that can be caught and handled by further states being executed in the `StateMachine`. BranchState – [Required] `StateBase` instance, providing the starting state for each branch to be run concurrently BranchRetryList (list of `Retrier`) – [Optional] `list` of `Retrier` instances corresponding to error states that cause the `For` loop iteration to be retried. This will occur until the number of retries has been exhausted for this iteration, afterwhich state level `Retrier` will be triggered if specified
Type:	RetryList: list of `Retrier`
Type:	CatcherList: list of `Catcher`
Type:	BranchState: `StateBase`
Param:	Iterations: [Required] The total number of branches to be executed. Must be larger than zero
Type:	Iterations: int
Param:	MaxConcurrency: [Required] The maximum number of branches that are to executed concurrently. Must be larger than zero
Type:	MaxConcurrency: int
Param:	IteratorPath: [Required] The JSONPath in which to inject the iterator value into the Input passed to the branch
Type:	IteratorPath: str

clone(NameFormatString='{}')¶

Returns a clone of this instance, with the clone named per the NameFormatString, to avoid state name clashes.

If this instance is not an end state, then the next state will also be cloned, to establish a complete clone of the branch form this instance onwards.

Parameters:	NameFormatString (str) – [Required] The naming template to be applied to generate the name of the new instance.
Returns:	`LimitedParallel` – A new instance of this instance and any other instances in its branch.

get_branch_retry_list()¶

Returns the list of Retrier instances that will be applied separately to each branch execution, allowing failure in one branch iteration during the LimitedParallel execution to be retried.

Returns:	`list` of `Retrier` instances

get_branch_state()¶

Returns the initial state for the branch processing

Returns:	`StateBase` – The initial state of the branch

get_iterations()¶

Returns the total number of branch executions to be performed

Returns:	int – The total number of branch executions to perform

get_iterator_path()¶

Returns the injection JSONPath to be used to add the iterator value into the Input for a branch

Returns:	str – The JSONPath for iterator value injection

get_max_concurrency()¶

Returns the maxiumum number of concurrent branch executions

Returns:	`int` – The maximum number of concurrent branch executions

set_branch_retry_list(BranchRetryList=None)¶

Sets the list of Retrier instance to be applied to each of the branch execution in the LimitedParallel.

If none are specified, then LimitedParallel will retry at the state level (if Retrier are specified)

Parameters:	BranchRetryList – [Optional] `list` of `Retrier` instances corresponding to error states that can be retried for each branch execution
Type:	BranchRetryList: list of `StateBase`

set_branch_state(BranchState=None)¶

Set the initial state for the branch processing

Parameters:	BranchState – [Required] `StateBase` instance, providing the starting state for each branch to be run concurrently
Type:	BranchState: `StateBase`

set_iterations(Iterations=0)¶

Sets the total number of branch executions to be performed. Must be larger than zero.

Param:	Iterations: [Required] The total number of branches to be executed. Must be larger than zero
Type:	Iterations: int

set_iterator_path(IteratorPath='$.iteration')¶

Sets the injection JSONPath to use to add the iterator value into the Input for a branch

Param:	IteratorPath: [Required] The JSONPath in which to inject the iterator value into the Input passed to the branch
Type:	IteratorPath: str

set_max_concurrency(MaxConcurrency=1)¶

Sets the maximum number of concurrent branch executions, and the supplied value must be greater than zero. Default is sequential processing.

Param:	MaxConcurrency: [Required] The maximum number of branches that are to executed concurrently. Must be larger than zero
Type:	MaxConcurrency: int

to_json()¶

Returns the JSON representation of this instance.

Returns:	dict – The JSON representation

validate()¶

Validates this instance is correctly specified.

Raises Exception with details of the error, if the state is incorrectly defined.