.md .pdf

repository open issue

Contents

Tooling

From the architecture schema of Kebechet, you can see that Kebechet requires multiple components and workflows to provide services. The major components and their functions are described below.

Technical Infrastructure

This section describes the categories of components involved in the operational life of Kebechet application:

• service -> services that are used by the application and provided by the platform

• application -> components or libraries that are created for the application

• monitoring -> components or libraries that are created/used for monitoring and pushing metrics

• analytics -> components or libraries that are created/used for providing analysis on the data collected

Service components

• Kafka - an open-source distributed event streaming platform

• Argo - family of open source tools for Kubernetes to run workflows, manage clusters, and do GitOps right

• Ceph - an open-source, distributed storage system

Application components

• PostgreSQL Database - open source object-relational database

• Thoth storages - Thoth storages and database adapter

• Thoth User API - Thoth service interface and Kafka producer

• Thoth Investigator - Kafka consumer of messages sent by different components

• Argo workflow controller - Controller for scheduling and handling Argo workflows

• Argo workflows. Each of them contain several steps:

  • Kebechet Administrator Argo workflow - run Kebechet administrator that decides based on triggers which repo needs to receive a Kebechet update

  • Adviser Argo workflow - run Thoth adviser recommender system

  • Kebechet Argo workflow - run Kebechet component logic

  • Solver Argo workflow - run solver component to analyze python packages for different runtime environments, stores knowledge in the graph database and sends messages for scheduling Security workflows

Monitoring Components

• Prometheus - Monitoring tool for collecting time series metrics. To learn more about Prometheus, refer to this documentation here

• Grafana - Visualization tool for creating dashboards/graphs of the metrics

• Thoth MI Scheduler - Tool to collect GitHub data

• GitHub Alertmanager - Alertmanager that handles alerts and creates GitHub issues in the chosen repository to notify the respective teams

• Thoth Metrics Exporter - Component that evaluates content metrics from the database and sends metrics to Prometheus

• Thoth Graph metrics exporter - Component that evaluates content metrics for long running processes (evaluating bloat data for example) from the database and sends metrics to Prometheus

• PostgreSQL metrics exporter - Components that collect operational metrics from the database and send them to Prometheus

Analytical Components

• Prometheus API client - A Python wrapper for the Prometheus HTTP API

• Thoth SLO-reporter - Aggregate, analyze and combine metrics from different sources (Prometheus or Ceph data), providing a report on SLI by email

• Thoth reporter - Aggregate, analyze and produce metrics from adviser reports that are stored on Ceph for visualization on Superset

• Thoth MI Scheduler - component that schedules mi workflows based on list of maintained GitHub repositories:

  • MI Argo workflow - component that collect GitHub metrics from GitHub repositories

• Superset - Superset is a modern data exploration and visualization platform

previous

Kebechet

next

Monitoring Metrics