本片博客将介绍kuberhealthy的使用以及相关原理 ，kuberhealthy项目地址

kuberhealthy简介

  • kuberhealthy是一个k8s operator
  • kuberhealthy 用于k8s 组件功能检测和验证，保证功能可用
  • kuberhealthy 通过下执行kuberhealthy checker pod进行测试逻辑执行和测试结果收集从而完成组件功能验证

核心概念

  1. CRD: KuberhealthyChecker
    该CRD指定了一次check（可以理解为一次测试）的相关控制信息，比如

    • 运行check的间隔
    • 回收check的时间
    • checker pod的具体内容，包括拉取哪些镜像以及相关参数
      下面是一个典型的KuberhealthyCheck
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    apiVersion: comcast.github.io/v1
    kind: KuberhealthyCheck
    metadata:
    name: kh-test-check 
    spec:
    runInterval: 30s # The interval that Kuberhealthy will run your check on 
    timeout: 2m # After this much time, Kuberhealthy will kill your check and consider it "failed"
    podSpec: # The exact pod spec that will run.  All normal pod spec is valid here.
        containers:
        - env: # Environment variables are optional but a recommended way to configure check behavior
            - name: MY_OPTION_ENV_VAR
            value: "option_setting_here"
        image: quay.io/comcast/test-check:latest # The image of the check you just pushed
        imagePullPolicy: Always # During check development, it helps to set this to 'Always' to prevent on-node image caching.
        name: main

  2. Controller: KuberhealthyController

    kuberhealthy controller负责将

    • 读取用户提交的kuberhealthychecker，创建相关checker pod并维护checker pod生命周期
    • 监听checker pod上传到externalstatus接口的数据，并将数据写入该checker 对应的KuberhealthyStatus中
    • 暴露数据接口，将KuberhealthyStatus中的数据转换成prometheus 数据并提供/metrics和/status http api供查询

  3. checker pod

    checker pod负责

    • 执行具体的健康检测逻辑
    • 将结果上报给externalStatus接口

  4. CRD: KuberhealthyState

    • 负责存储最近一次健康检查checker的结果，KuberhealthyController会将externalStatus接受的数据写入KuberhealthyStatus中
    • 作为KuberhealthyController的数据源转变为prometheus数据

工作原理

  1. 用户创建KuberhealthyCheck resource资源至k8s集群
  2. Kuberhealthy controller监听资源创建并根据KuberhealthyCheck中的image创建checker pod，并schedule到合适的k8s node上
  3. checker pod执行测试逻辑
  4. checker pod 将测试结果通过 http post到kuberhealthy controller的/exteranlstatus接口
  5. kuberhealthy controller 读取/externalstatus 接口的数据，维护该checker 对应KuberhealthyState中