Skip to main content

Signadot Operator

Overview​

The Signadot Operator is the in-cluster component of the Signadot Platform. To see a high-level view of all components, visit the architecture page. The operator is installed into the Kubernetes cluster using Helm into the signadot namespace. Once authorized, it connects and maintains a persistent & TLS secured connection with the Signadot Control Plane.

It provides the following functionality:

  1. Creating and managing in-cluster workloads (Kubernetes Deployments, Rollouts, etc) associated with a Sandbox.
  2. Creating and managing resources associated with Sandboxes.
  3. Supporting workstation-local development by:
    • Managing connections to workloads running on a workstation.
    • Proxying requests from a local workstation to within the cluster.
    • Serving a gRPC API for configuration and management of connections.
  4. Managing routing for requests between services within the cluster.
  5. Mananging Job Runner Groups within your cluster.
  6. Managing in-cluster execution of Smart Tests and Signadot Jobs.

Agent​

The agent is a deployment which maintains a secure persistent connection to the Signadot Control Plane at tunnel.signadot.com:443. This connection is used to:

  1. Provide access to the Kubernetes API to the Signadot Control Plane.
  2. Proxy requests from Endpoint URLs to services within the cluster, including performing in-cluster DNS resolution.

Since the agent provides access to the cluster Kubernetes API server, it tends to have scaled down read-only permissions for resources not owned by Signadot, and read-write permissions for Signadot owned objects. You can learn more about the permissions that are requested by the agent here.

Controller Manager​

The controller manager implements Kubernetes controllers for various Signadot Custom Resources, including

  1. Jobs (sdjb)
  2. JobRunnerGroups (sdjrg)
  3. Sandboxes (sds)
  4. Route Groups (sdrg)
  5. Forked Workloads (sdfw)
  6. External Workloads (sdxw)
  7. Resources (sdmr)
  8. RoutingConfig (sdrc)
  9. IstioRoutes (sdir)
  10. Signadot Object Lifecycle Methods (solm)

The high-level interactions between these objects are depicted below:

The controller-manager needs read-write permissions for any resources which are manipulated on behalf of sandboxes. You can learn more about the permissions that are requested by the controller manager here.

note

In versions of the operator prior < v0.14, there was an additional Signadot Routes (sdr) Custom Resource and Controller in use, which are now not in use anymore and scheduled for removal in a future release.

note

Signadot custom resource names are generally derived from concatenation, truncation where needed, and hashing for uniqueness. The names nonetheless often contain information provided in Sandboxes and RouteGroups at the control plane layer, including references to baseline workloads. We do not recommend using the custom resource Kubernetes object names to identify related objects, as this information can become lost. The presentation of such information in custom resource names is only best-effort and can not be relied upon.

JobRunnerGroup Custom Resource & Controller​

The JobRunnerGroup Custom Resource (CR) manages associated Pods on which Jobs associated with the JobRunnerGroup run. It also manages a Service for addressing those Pods and provides information to the control plane for coordinating Job queues, which live on the control plane.

The Pods are automatically equipped with a GRPC service which handles Job execution on behalf of the Job controller. This service also provides support for accessing logs to the control plane.

Job Custom Resource & Controller​

The Job CR represents a specification of how to run a job, including its command or script, environment and routing context. The controller is responsible for assigning and unassigning jobs to Pods managed by the associated JobRunnerGroup, and for tracking execution state in the Job CR's status.

Sandbox Custom Resource & Controller​

The Sandbox CR is a low-level implementation of the Signadot Sandbox specification. The controller for a Sandbox performs all the important actions associated with creating a Sandbox such as:

  1. Creating and finalizing ForkedWorkload CRs running within the cluster.
  2. Creating and finalizing ExternalWorkload CRs used for local development.
  3. Creating and finalizing Resources CRs.
  4. Creating and finalizing RoutingConfig CRs.

RouteGroup Custom Resource & Controller​

A RouteGroup CR represents a set of sandboxes added into a single routing context. As such, a RouteGroup CR represents a way to route requests to a set of sandboxes. The RouteGroup controller creates RoutingConfig CRs and coordinates RouteGroup status with the constituent Sandboxes.

Resource Custom Resource & Controllers​

A Resource CR represents the state of a resource associated with a sandbox. Controlling a resource entails making it available to sandboxed workloads according to the associated resource plugin spec, and deleting the resource when the associated sandbox is terminated.

The Object Lifecycle Methods CR is a low-level implementation of a workflow manager attached to the lifecycle of a target object. In the context of Resources, its controller is responsible for the execution of the defined workflows for the create and delete phases.

ForkedWorkload Custom Resource & Controller​

A ForkedWorkload CR represents a modification to a baseline workload running as part of a Sandbox. The ForkedWorkload controller is responsible for:

  1. Applying customizations specified to create a sandboxed workload from the baseline.
  2. Creating and finalizing the sandboxed workload.
  3. Watching all services which match the baseline workload and creating corresponding services for routing requests to the sandboxed workload.

Note that each service that matches the baseline workload will have a corresponding service for the sandboxed workload. The newly created services will map the same ports as the baseline workload services, but to the sandboxed workload instead.

ExternalWorkload Custom Resource & Controller​

An ExternalWorkload CR represents the state of a workload run outside of the Kubernetes cluster as part of a Sandbox. It is used for local development, where the state of the associated workload amounts only to the state of a tunnel used to direct traffic destined for the workload to the connected workstation. The controller performs health checks on the tunnel and garbage collection of unhealthy tunnels.

Like the ForkedWorkload controller, the ExternalWorkload controller finds all services which match the baseline workload and creates corresponding services for the external workload. Unlike the ForkedWorkload controller, those services do not have selectors for matching pods. Rather, the ExternalWorkload controller also creates Endpoints associated with each created Service in order to direct traffic to the associated tunnel running on the tunnel-proxy pods.

RoutingConfig Custom Resource & Controller​

A RoutingConfig CR represents the desired routing associated with a given routing key. A routing key, in turn, is either associated with a Sandbox or a RouteGroup. The primary information in a RoutingConfig is a list of references to Forked or External workloads. The RoutingConfig controller checks the status of the ForkedWorkload and ExternalWorkload CRs to see what associated services have been created. With this information, it:

  • Produces any associated mesh-specific CRs (such as IstioRoute) if enabled, one for each service matching some ForkedWorkload or ExternalWorkload in the Spec.
  • If the Signadot DevMesh is in use, it will not create any additional CRs, but instead use its own status to supply routing configuration to the RouteServer running as part of the operator.
  • Generates a status representing the state of the routing for the associated routing key, by observing any associated CRs.

IstioRoute Custom Resource & Controller​

IstioRoute CRs represent the desired and observed state of Istio VirtualServices with respect to a single Service. This service is presented to the RoutingConfig controller by a ForkedWorkload CR or an ExternalWorkload CR and passed on to the RoutingConfig which owns the IstioRoute. The controller finds matching Istio VirtualServices and reconciles them with respect to the Service. One Istio HTTPRoute is maintained for each port in the associated service.

More information about using Istio with Signadot is available here.

Route Server​

The route server is a Kubernetes Deployment that is tasked with keeping track of where to route requests provided the destination address and associated sandbox. To accomplish this, the route server watches RoutingConfigs.

Starting with v0.15.0 of the operator, the route server provides gRPC and rest HTTP APIs at routeserver.signadot.svc:7777 and routeserver.signadot.svc:7778, respectively. The API specs, along with documentation, examples, and supporting Go client code are available here. The REST API documentation is available here.

Signadot DevMesh Sidecars​

The Signadot Operator also contains optional built-in functionality to add a DevMesh sidecar to each baseline pod. The injection occurs into pods using a Kubernetes Mutating Webhook. The sidecar integrates with the Route Server and provides dynamic request routing capabilities in a cluster that does not have a Service Mesh such as Istio installed. The DevMesh sidecar is applied only to baseline workloads.

Sandbox Traffic Manager Sidecars​

In-cluster workloads within Sandboxes have a sidecar dedicated to managing traffic. This sidecar provides HTTP traffic capture when Smart Tests are run that opt into specific functionality. In this case, captured traffic is uploaded to the Traffic Model Server in the control plane. The Sandbox Traffic Manager sidecar is applied only to forked workloads in a sandbox.

Tunnel API​

The Tunnel API deployment hosts a gRPC server which workstations consume to coordinate in-cluster state with workstation connection state. The Tunnel API provides information about Kubernetes Service network addresses and DNS, and Sandbox state.

Tunnel Proxy​

The Tunnel Proxy is a Deployment responsible for providing access to the Tunnel API, proxying requests from workstations to the cluster, proxying traffic from the cluster to workstations, and providing an audit log of all connections. It also hosts a metrics server providing statistics about active and total connections.

IO Context Server​

The io context server (aka ICS) is a Kubernetes Deployment responsible for storing and serving intermediate results produced by the execution of the work units defined within a Signadot Object Lifecycle Methods workflow. In simple words, whenever a Resource is being created or deleted, a workflow of work units is executed. Each of those work units may produce outputs, which will be stored in ICS. Apart from that, work units may also consume inputs from other work units, those inputs will be read from ICS and injected into the work unit main container.

IO Init Container​

The io init container, is an init container that is appended to the Resource workflow pods. It is responsible for reading the required inputs from ICS and injecting them into the main container.

IO Sidecar Container​

The io sidecar container, is a container that run along side with the main container, whose purpose is to collect the generated outputs from the Resource workflow pods and store them in ICS.

Job Init Container​

the job init container is an init container that is appended to Job Pods. It is responsible for installing the job exection GRPC service used by the Job controller.