Run A JobSet

Run a Kueue scheduled JobSet.

This document explains how you can use Kueue’s scheduling and resource management functionality when running JobSet Operator JobSets.

This guide is for batch users that have a basic understanding of Kueue. For more information, see Kueue’s overview.

Before you begin

  1. Check Administer cluster quotas for details on the initial Kueue setup.

  2. See JobSet Installation for installation and configuration details of JobSet Operator.

JobSet definition

When running JobSets on Kueue, take into consideration the following aspects:

a. Queue selection

The target local queue should be specified in the metadata.labels section of the JobSet configuration.

metadata:
  labels:
    kueue.x-k8s.io/queue-name: user-queue

b. Configure the resource needs

The resource needs of the workload can be configured in the spec.replicatedJobs. Should also be taken into account that number of replicas, parallelism and completions affect the resource calculations.

    - replicas: 1
      template:
        spec:
          completions: 2
          parallelism: 2
          template:
            spec:
              containers:
                - resources:
                    requests:
                      cpu: 1

c. Jobs prioritisation

The first PriorityClassName of spec.replicatedJobs that is not empty will be used as the priority.

    - template:
        spec:
          template:
            spec:
              priorityClassName: high-priority

Example JobSet

The JobSet looks like the following:

# jobset-sample.yaml
apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
  generateName: sleep-job-
  labels:
    kueue.x-k8s.io/queue-name: user-queue
spec:
  network:
    enableDNSHostnames: false
    subdomain: some-subdomain
  replicatedJobs:
    - name: workers
      replicas: 2
      template:
        spec:
          parallelism: 4
          completions: 4
          backoffLimit: 0
          template:
            spec:
              containers:
                - name: sleep
                  image: busybox
                  resources:
                    requests:
                      cpu: 1
                      memory: "200Mi"
                  command:
                    - sleep
                  args:
                    - 100s
    - name: driver
      template:
        spec:
          parallelism: 1
          completions: 1
          backoffLimit: 0
          template:
            spec:
              containers:
                - name: sleep
                  image: busybox
                  resources:
                    requests:
                      cpu: 2
                      memory: "200Mi"
                  command:
                    - sleep
                  args:
                    - 100s

You can run this JobSet with the following commands:

# To monitor the queue and admission of the jobs, you can run this example multiple times:
kubectl create -f jobset-sample.yaml

Last modified March 28, 2024: docs: Fix broken link in Kubernetes Jobs guide (#1923) (e98a935)