User-provided matrixed steps for a shared pipeline definition

[justinabrahms]

Feature request

Steps should be able to matrix with objects as inputs.

Use case

I have a shared pipeline. It handles communication with various stakeholders via slack, etc, as well as notifying various services that a deploy has happened.

I'd like the users of this pipeline to have the ability to add custom steps in the pipeline. They need parallelism due to potentially long running test suites. My thought was that the user could provide an object as an input to the pipeline and we could do a matrix build across those values.

# [...pipelinerun..]
params:
 - name: post-dev-validation-steps
   value:
   - name: "always-pass"
     image: "alpine:3.14"
     script: "echo Hello from dev validation && exit 0"
   - name: "something that takes a long time"
     image: "ubuntu:20.04"
     script: "sleep 180"

An underlying step would look something like..

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: custom-script
  namespace: productivity-team
spec:
  params:
    - name: validation
      type: object
      properties:
        name:
          type: string
        image:
          type: string
        script:
          type: string
  workspaces:
    - name: source
      description: The workspace containing the source code
  steps:
    - name: custom-validation
      image: $(params.validation.image)
      script: |
        #!/usr/bin/env bash
        set -xe
        echo "Running script: $(params.validation.name)"
        $(params.validation.script)