Guided Exercise: Inspect Kubernetes Resources

Jul 12, 2023·

12 min read

Verify the state of an OpenShift cluster by querying its recognized resource types, their schemas, and extracting information from Kubernetes resources that are related to to OpenShift cluster services.

Outcomes

  • List and explain the supported API resources for a cluster.

  • Identify resources from specific API groups.

  • Format command outputs in the YAML and JSON formats.

  • Use filters to parse command outputs.

  • Use JSONPath and custom columns to extract information from resources.

As the student user on the workstation machine, use the lab command to prepare your system for this exercise. This command ensures that the cluster is accessible and that all resources are available for this exercise. It also creates a myapp application in the cli-resources project.

[student@workstation ~]$ lab start cli-resources

Procedure 2.2. Instructions

  1. Log in to the OpenShift cluster as the developer user with the developer password. Select the cli-resources project.

    1. Log in to the OpenShift cluster.

       [student@workstation ~]$ oc login -u developer -p developer \
         https://api.ocp4.example.com:6443
       Login successful.
       ...output omitted...
      
    2. Set the cli-resources project as the active project.

       [student@workstation ~]$ oc project cli-resources
       ...output omitted...
      
  2. List the available cluster resource types with the api-resources command. Then, use filters to list namespaced and non-namespaced resources.

    1. List the available resource types with the api-resources command.

       [student@workstation ~]$ oc api-resources
       NAME                    SHORTNAMES   APIVERSION   NAMESPACED   KIND
       bindings                             v1           true         Binding
       componentstatuses       cs           v1           false        ComponentStatus
       configmaps              cm           v1           true         ConfigMap
       endpoints               ep           v1           true         Endpoints
       events                  ev           v1           true         Event
       limitranges             limits       v1           true         LimitRange
       namespaces              ns           v1           false        Namespace
       nodes                   no           v1           false        Node
       persistentvolumeclaims  pvc          v1           true         PersistentVolumeClaim
       persistentvolumes       pv           v1           false        PersistentVolume
       pods                    po           v1           true         Pod
       ...output omitted...
      

      The api-resources command prints the supported API resources, including resource names, available shortnames, and the API versions.

    2. Use the --namespaced option to limit the output of the api-resources command to namespaced resources.

      Then, determine the number of available namespaced resources. Use the -o name option to list the resource names, and then pipe the output to the wc -l command.

       [student@workstation ~]$ oc api-resources --namespaced
       NAME                    SHORTNAMES   APIVERSION   NAMESPACED   KIND
       bindings                             v1           true         Binding
       configmaps              cm           v1           true         ConfigMap
       endpoints               ep           v1           true         Endpoints
       events                  ev           v1           true         Event
       limitranges             limits       v1           true         LimitRange
       persistentvolumeclaims  pvc          v1           true         PersistentVolumeClaim
       pods                    po           v1           true         Pod
       podtemplates                         v1           true         PodTemplate
       replicationcontrollers  rc           v1           true         ReplicationController
       resourcequotas          quota        v1           true         ResourceQuota
       secrets                              v1           true         Secret
       ...output omitted...
       [student@workstation ~]$ oc api-resources --namespaced -o name | wc -l
       108
      

      The cluster has 108 namespaced cluster resource types, such as the pods, deployments, and services resources.

    3. Limit the output of the api-resources command to non-namespaced resources.

      Then, determine the number of available non-namespaced resources. To list the resource names, use the -o name option and then pipe the output to the wc -l command.

       [student@workstation ~]$ oc api-resources --namespaced=false
       NAME                             SHORTNAMES   APIVERSION                       ...
       componentstatuses                cs           v1                               ...
       namespaces                       ns           v1                               ...
       nodes                            no           v1                               ...
       persistentvolumes                pv           v1                               ...
       mutatingwebhookconfigurations                 admissionregistration.k8s.io/v1  ...
       validatingwebhookconfigurations               admissionregistration.k8s.io/v1  ...
       customresourcedefinitions        crd,crds     apiextensions.k8s.io/v1          ...
       ...output omitted...
       [student@workstation ~]$ oc api-resources --namespaced=false -o name | wc -l
       114
      

      The cluster has 114 non-namespaced cluster resource types, such as the nodes, images, and project resources.

  3. Identify and explain the available cluster resource types that the core API group provides. Then, describe a resource from the core API group in the cli-resources project.

    1. List the available resource types with the api-resources command.

       [student@workstation ~]$ oc api-resources
       NAME                    SHORTNAMES   APIVERSION   NAMESPACED   KIND
       bindings                             v1           true         Binding
       componentstatuses       cs           v1           false        ComponentStatus
       configmaps              cm           v1           true         ConfigMap
       endpoints               ep           v1           true         Endpoints
       events                  ev           v1           true         Event
       limitranges             limits       v1           true         LimitRange
       namespaces              ns           v1           false        Namespace
       nodes                   no           v1           false        Node
       persistentvolumeclaims  pvc          v1           true         PersistentVolumeClaim
       persistentvolumes       pv           v1           false        PersistentVolume
       pods                    po           v1           true         Pod
       ...output omitted...
       controllerrevisions                  apps/v1      true         ControllerRevision
       daemonsets              ds           apps/v1      true         DaemonSet
       ...output omitted...
       cronjobs                cj           batch/v1     true         CronJob
       jobs                                 batch/v1     true         Job
       ...output omitted...
      

      You can use the APIVERSIONS field to determine which API group provides the resource. The field lists the group followed by the API version of the resource. For example, the jobs resource type is provided by the batch API group, and v1 is the API version of the resource.

    2. Filter the output of the api-resources command to only show resources from the core API group. Use the --api-group option and set '' as the value.

       [student@workstation ~]$ oc api-resources --api-group ''
       NAME                     SHORTNAMES   APIVERSION   NAMESPACED   KIND
       bindings                              v1           true         Binding
       componentstatuses        cs           v1           false        ComponentStatus
       configmaps               cm           v1           true         ConfigMap
       endpoints                ep           v1           true         Endpoints
       events                   ev           v1           true         Event
       limitranges              limits       v1           true         LimitRange
       namespaces               ns           v1           false        Namespace
       nodes                    no           v1           false        Node
       persistentvolumeclaims   pvc          v1           true         PersistentVolumeClaim
       persistentvolumes        pv           v1           false        PersistentVolume
       pods                     po           v1           true         Pod
       podtemplates                          v1           true         PodTemplate
       replicationcontrollers   rc           v1           true         ReplicationController
       resourcequotas           quota        v1           true         ResourceQuota
       secrets                               v1           true         Secret
       serviceaccounts          sa           v1           true         ServiceAccount
       services                 svc          v1           true         Service
      

      The core API group provides many resource types, such as nodes, events, and pods.

    3. Use the explain command to list a description and the available fields for the pods resource type.

       [student@workstation ~]$ oc explain pods
       KIND:     Pod
       VERSION:  v1
      
       DESCRIPTION:
            Pod is a collection of containers that can run on a host. This resource is
            created by clients and scheduled onto hosts.
      
       FIELDS:
          apiVersion    <string>
            APIVersion defines the versioned schema of this representation of an
            object. Servers should convert recognized schemas to the latest internal
            value, and may reject unrecognized values
       ...output omitted...
      
    4. List all pods in the cli-resources project.

       [student@workstation ~]$ oc get pods
       NAME                     READY   STATUS    RESTARTS   AGE
       myapp-54fcdcd9d7-2h5vx   1/1     Running   0          4m25s
      

      A single pod exists in the cli-resources project. The pod name might differ in your output.

    5. Use the describe command to view the configuration and events for the pod. Specify the pod name from the previous step.

       [student@workstation ~]$ oc describe pod myapp-54fcdcd9d7-2h5vx
       Name:             myapp-54fcdcd9d7-2h5vx
       Namespace:        cli-resources
       ...output omitted...
       Status:           Running
       IP:               10.8.0.127
       IPs:
         IP:           10.8.0.127
       Controlled By:  ReplicaSet/myapp-54fcdcd9d7
       Containers:
         myapp:
           Container ID:   cri-o://e0da...669d
           Image:          registry.ocp4.example.com:8443/ubi8/httpd-24:1-215
           Image ID:       registry.ocp4.example.com:8443/ubi8/httpd-24@sha256:91ad...fd83
       ...output omitted...
           Limits:
             cpu:     500m
             memory:  128Mi
           Requests:
             cpu:        500m
             memory:     128Mi
           Environment:  <none>
       ...output omitted...
       Events:
         Type    Reason          Age    From               Message
         ----    ------          ----   ----               -------
         Normal  Scheduled       10m    default-scheduler  Successfully assigned cli-resources/myapp-54fcdcd9d7-2h5vx to master01
       ....output omitted...
      
    6. Retrieve the details of the pod in a structured format. Use the get command and specify the output as the YAML format. Compare the results of the describe command versus the get command.

       [student@workstation ~]$ oc get pod myapp-54fcdcd9d7-2h5vx -o yaml
       apiVersion: v1
       kind: Pod
       metadata:
         annotations:
       ...output omitted...
         labels:
           app: myapp
           pod-template-hash: 54fcdcd9d7
         name: myapp-54fcdcd9d7-2h5vx
         namespace: cli-resources
       ...output omitted...
       spec:
         containers:
         - image: registry.ocp4.example.com:8443/ubi8/httpd-24:1-215
           imagePullPolicy: Always
           name: myapp
           resources:
             limits:
               cpu: 500m
               memory: 128Mi
             requests:
               cpu: 500m
               memory: 128Mi
       ...output omitted...
      

      Using a structured format with the get command provides more details about a resource than the describe command.

  4. Identify and explain the available cluster resource types that the Kubernetes apps API group provides. Then, describe a resource from the apps API group in the cli-resources project.

    1. List the resource types that the apps API group provides.

       [student@workstation ~]$ oc api-resources --api-group apps
       NAME                  SHORTNAMES   APIVERSION   NAMESPACED   KIND
       controllerrevisions                apps/v1      true         ControllerRevision
       daemonsets            ds           apps/v1      true         DaemonSet
       deployments           deploy       apps/v1      true         Deployment
       replicasets           rs           apps/v1      true         ReplicaSet
       statefulsets          sts          apps/v1      true         StatefulSet
      
    2. Use the explain command to list a description and fields for the deployments resource type.

       [student@workstation ~]$ oc explain deployments
       KIND:     Deployment
       VERSION:  apps/v1
      
       DESCRIPTION:
            Deployment enables declarative updates for Pods and ReplicaSets.
      
       FIELDS:
          apiVersion    <string>
            APIVersion defines the versioned schema of this representation of an
            object. Servers should convert recognized schemas to the latest internal
            value, and may reject unrecognized values.
       ...output omitted...
      
    3. Use the get command to identify any deployment resources in the cli-resources project.

       [student@workstation ~]$ oc get deploy
       NAME    READY   UP-TO-DATE   AVAILABLE   AGE
       myapp   1/1     1            1           25m
      
    4. The myapp deployment exists in the cli-resources project. Use the get command and the -o wide option to identify the container name and the container image in the deployment.

       [student@workstation ~]$ oc get deploy myapp -o wide
       NAME  ... CONTAINERS IMAGES                                              SELECTOR
       myapp ... myapp      registry.ocp4.example.com:8443/ubi8/httpd-24:1-215  app=myapp
      

      The myapp deployment uses the registry.ocp4.example.com:8443/ubi8/httpd-24:1-215 container image for the myapp container.

    5. Describe the myapp deployment to view more details about the resource.

       [student@workstation ~]$ oc describe deployment myapp
       Name:                   myapp
       Namespace:              cli-resources
       CreationTimestamp:      Wed, 01 Mar 2023 18:41:39 -0500
       Labels:                 my-app
       Annotations:            deployment.kubernetes.io/revision: 1
       Selector:               app=myapp
       Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
       StrategyType:           RollingUpdate
       MinReadySeconds:        0
       RollingUpdateStrategy:  25% max unavailable, 25% max surge
       Pod Template:
         Labels:  app=myapp
         Containers:
          myapp:
           Image:      registry.ocp4.example.com:8443/ubi8/httpd-24:1-215
           Port:       8080
           Host Port:  8080
           Limits:
             cpu:        500m
             memory:     128Mi
           Environment:  <none>
           Mounts:       <none>
         Volumes:        <none>
       Conditions:
         Type           Status  Reason
         ----           ------  ------
         Available      True    MinimumReplicasAvailable
         Progressing    True    NewReplicaSetAvailable
       OldReplicaSets:  <none>
       NewReplicaSet:   myapp-54fcdcd9d7 (1/1 replicas created)
       Events:
         Type    Reason             Age   From                   Message
         ----    ------             ----  ----                   -------
         Normal  ScalingReplicaSet  30m   deployment-controller  Scaled up replica set myapp-54fcdcd9d7 to 1
      
  5. Identify and explain the available cluster resource types that the OpenShift configuration API group provides. Then, describe a resource from the OpenShift configuration API group.

    1. List the resource types that the OpenShift configuration API group provides.

       [student@workstation ~]$ oc api-resources --api-group config.openshift.io
       NAME                 SHORTNAMES  APIVERSION              NAMESPACED KIND
       apiservers                       config.openshift.io/v1  false      APIServer
       authentications                  config.openshift.io/v1  false      Authentication
       builds                           config.openshift.io/v1  false      Build
       clusteroperators     co          config.openshift.io/v1  false      ClusterOperator
       clusterversions                  config.openshift.io/v1  false      ClusterVersion
       consoles                         config.openshift.io/v1  false      Console
       dnses                            config.openshift.io/v1  false      DNS
       featuregates                     config.openshift.io/v1  false      FeatureGate
       imagecontentpolicies             config.openshift.io/v1  false      ImageContentPolicy
       images                           config.openshift.io/v1  false      Image
       infrastructures                  config.openshift.io/v1  false      Infrastructure
       ingresses                        config.openshift.io/v1  false      Ingress
       networks                         config.openshift.io/v1  false      Network
       nodes                            config.openshift.io/v1  false      Node
       oauths                           config.openshift.io/v1  false      OAuth
       operatorhubs                     config.openshift.io/v1  false      OperatorHub
       projects                         config.openshift.io/v1  false      Project
       proxies                          config.openshift.io/v1  false      Proxy
       schedulers                       config.openshift.io/v1  false      Scheduler
      

      The config.openshift.io API group provides multiple, non-namespaced resource types.

    2. Use the explain command to list a description and fields for the projects resource type.

       [student@workstation ~]$ oc explain projects
       KIND:     Project
       VERSION:  project.openshift.io/v1
      
       DESCRIPTION:
            Projects are the unit of isolation and collaboration in OpenShift. A
            project has one or more members, a quota on the resources that the project
            may consume, and the security controls on the resources in the project.
            Within a project, members may have different roles - project administrators
            can set membership, editors can create and manage the resources, and
            viewers can see but not access running containers. In a normal cluster
            project administrators are not able to alter their quotas - that is
            restricted to cluster administrators.
      
            Listing or watching projects will return only projects the user has the
            reader role on.
       ...output omitted...
      
    3. Describe the cli-resources project.

       [student@workstation ~]$ oc describe project cli-resources
       Name:            cli-resources
       Created:        10 minutes ago
       Labels:            kubernetes.io/metadata.name=cli-resources
                   pod-security.kubernetes.io/audit=restricted
                   pod-security.kubernetes.io/audit-version=v1.24
                   pod-security.kubernetes.io/warn=restricted
                   pod-security.kubernetes.io/warn-version=v1.24
       Annotations:        openshift.io/description=
                   openshift.io/display-name=
                   openshift.io/requester=system:admin
                   openshift.io/sa.scc.mcs=s0:c27,c4
                   openshift.io/sa.scc.supplemental-groups=1000710000/10000
                   openshift.io/sa.scc.uid-range=1000710000/10000
       Display Name:        <none>
       Description:        <none>
       Status:            Active
       Node Selector:        <none>
       Quota:            <none>
       Resource limits:    <none>
      
    4. Retrieve more details of the cli-resources project. Use the get command, and format the output to use JSON.

       [student@workstation ~]$ oc get project cli-resources -o json
       {
           "apiVersion": "project.openshift.io/v1",
           "kind": "Project",
           "metadata": {
       ...output omitted....
               "labels": {
                   "kubernetes.io/metadata.name": "cli-resources",
                   "pod-security.kubernetes.io/audit": "restricted",
                   "pod-security.kubernetes.io/audit-version": "v1.24",
                   "pod-security.kubernetes.io/warn": "restricted",
                   "pod-security.kubernetes.io/warn-version": "v1.24"
               },
               "name": "cli-resources",
               "resourceVersion": "705313",
               "uid": "53cbbe45-31ea-4b41-93a9-4ba5c2c4c1f3"
           },
       ...output omitted...
           "status": {
               "phase": "Active"
           }
       }
      

      The get command provides additional details, such as the kind and apiVersion attributes, of the project resource.

  6. Verify the cluster status by inspecting cluster services. Format command outputs by using filters.

    1. Retrieve the list of pods for the Etcd operator. The Etcd operator is available in the openshift-etcd namespace. Specify the namespace with the --namespace or -n option.

       [student@workstation ~]$ oc get pods -n openshift-etcd
       Error from server (Forbidden): pods is forbidden: User "developer" cannot list resource "pods" in API group "" in the namespace "openshift-etcd"
      

      The developer user cannot access resources in the openshift-etcd namespace. Regular cluster users, such as the developer user, cannot query resources in the openshift- namespaces.

      Log in as the admin user with the redhatocp password. Then, retrieve the list of pods in the openshift-etcd namespace.

       [student@workstation ~]$ oc login -u admin -p redhatocp
       Login successful
       ...output omitted...
       [student@workstation ~]$ oc get pods -n openshift-etcd
       NAME                   READY   STATUS      RESTARTS   AGE
       etcd-master01          4/4     Running     36         25d
       installer-2-master01   0/1     Completed   0          25d
       installer-3-master01   0/1     Completed   0          25d
      
    2. Retrieve the conditions status of the etcd-master01 pod in the openshift-etcd namespace. Use filters to limit the output to the .status.conditions attribute of the pod. Compare the outputs of the JSONPath and jq filters.

       [student@workstation ~]$ oc get pods etcd-master01 -n openshift-etcd \
         -o=jsonpath='{.status.conditions}{"\n"}'
       [{"lastProbeTime":null,"lastTransitionTime":"2023-03-07T18:05:13Z",
       "status":"True","type":"Initialized"},{"lastProbeTime":null,
       "lastTransitionTime":"2023-03-07T18:05:28Z","status":"True","type":"Ready"},
       {"lastProbeTime":null,"lastTransitionTime":"2023-03-07T18:05:28Z",
       "status":"True","type":"ContainersReady"},
       {"lastProbeTime":null,"lastTransitionTime":"2023-03-07T18:05:06Z",
       "status":"True","type":"PodScheduled"}]
      
       [student@workstation ~]$ oc get pods -n openshift-etcd etcd-master01 \
         -o json | jq .status.conditions
       [
         {
           "lastProbeTime": null,
           "lastTransitionTime": "2023-03-07T18:05:13Z",
           "status": "True",
           "type": "Initialized"
         },
         {
           "lastProbeTime": null,
           "lastTransitionTime": "2023-03-07T18:05:28Z",
           "status": "True",
           "type": "Ready"
         },
         {
           "lastProbeTime": null,
           "lastTransitionTime": "2023-03-07T18:05:28Z",
           "status": "True",
           "type": "ContainersReady"
         },
         {
           "lastProbeTime": null,
           "lastTransitionTime": "2023-03-07T18:05:06Z",
           "status": "True",
           "type": "PodScheduled"
         }
       ]
      

      Using the JSON format and the jq filter provides a structured output for the .status.conditions attribute.

    3. Retrieve the condition status of the prometheus-k8s-0 pod in the openshift-monitoring namespace. Configure the output to use the YAML format, and then filter the output with the yq filter.

       [student@workstation ~]$ oc get pods -n openshift-monitoring prometheus-k8s-0 \
         -o yaml |  yq r - 'status.conditions'
       - lastProbeTime: null
         lastTransitionTime: "2023-03-07T18:07:17Z"
         status: "True"
         type: Initialized
       - lastProbeTime: null
         lastTransitionTime: "2023-03-07T18:07:45Z"
         status: "True"
         type: Ready
       - lastProbeTime: null
         lastTransitionTime: "2023-03-07T18:07:45Z"
         status: "True"
         type: ContainersReady
       - lastProbeTime: null
         lastTransitionTime: "2023-02-09T22:39:52Z"
         status: "True"
         type: PodScheduled
      

      The r - option tells the yq command to read the standard input (STDIN) for the YAML output of the get command.

    4. Use the get command to retrieve detailed information for the pods in the openshift-storage namespace. Use the YAML format and custom columns to filter the output according to the following table:

      | Column title | Object | | --- | --- | | Pod | metadata.name | | API | apiVersion | | Container | spec.containers[].name | | Phase | status.phase | | IP | status.podIP | | Ports | spec.containers[].ports[].containerPort |

       [student@workstation ~]$ oc get pods -n openshift-storage -o yaml \
         -o custom-columns=PodName:".metadata.name",\
       ContainerName:"spec.containers[].name",\
       Phase:"status.phase",\
       IP:"status.podIP",\
       Ports:"spec.containers[].ports[].containerPort"
       PodName                      ContainerName       Phase    IP          Ports
       lvm-operator-controller-...  kube-rbac-proxy     Running  10.8.0.97   8443
       topolvm-controller-....      topolvm-controller  Running  10.8.0.98   9808
       topolvm-node-9spzf           lvmd                Running  10.8.0.100  <none>
       vg-manager-z8g5k             vg-manager          Running  10.8.0.101  <none>
      

Finish

On the workstation machine, use the lab command to complete this exercise. This step is important to ensure that resources from previous exercises do not impact upcoming exercises.

[student@workstation ~]$ lab finish cli-resources