Spike on how CF-deployment (latest) to work with Eirini

[andrew-edgar]

Test and investigate the best way for CAPI to inteface with Eirini and to Diego in the long run.

Output from the story is a design on what would needed to be done in Capi (PR) and what needs to be done Eirini to work

[suhlig]

Goal: Change CC to interface with Eirini instead of Diego (configurable)

Options we looked at:

1. Point CC to Eirini implementing the BBS API
   • Protobuf is being used to serialize models
   • Would be a drop-in replacement without any CC code changes
   • URL is already exposed via config
2. Replace CC code with our version wherever CC talks to Diego today
   • Staging an app is implemented as Diego task
   • The convergence loop in CC tells Diego to stop or start apps.
   • AppStart just writes the desired state into CCDB
   • ProcessesSync and TasksSync instantiated from VCAP::CloudController::Jobs::Diego::Sync take care of convergence
   • For tasks, the cancel request is sent from the TasksSync and missing tasks are handled there, too
   • There seems no way to update a running task, therefore we found no DesireTaskHandler
   • Again, CloudController::DependencyLocator.instance.bbs_task_client is used in TaskDelete in order to get the singleton instance of the BBS client. All task actions like TaskCreate etc. are similar, as well as the app (LRP) actions.
   • DependencyLocator is the place to make the BbsClient wrappers configurable. We can check config and supply an OpiClient instead, there.
   • There is also a DesireAppHandler which used in the convergence loop. Since it uses the clients anyway, there is no need to change it.
3. Replace Diego::Client with our e.g. OPI::Client
   • Diego::Client already has a pretty domain-specific API
   • ProcessesSync and TasksSync use the Diego::Client already, so this could be an even simpler drop-in
   • Diego::Client is not used directly, but wrapped by VCAP::CloudController::Diego::BbsAppsClient and VCAP::CloudController::Diego::BbsTaskClient. There is also BbsStagerClient which is similar.

Preferred way: #3 as it seems to have the smallest set of changes and there really is no benefit of #1.

Stories

Just task execution in k8s

• In DependencyLocator#bbs_task_client, return a new OpiClient that just implements

  desire_task(task_guid: task_guid, task_definition: task_definition, domain:
  cancel_task(guid)
  task_by_guid(guid)
  tasks(domain: TASKS_DOMAIN)
  upsert_domain(domain: TASKS_DOMAIN, ttl: TASKS_DOMAIN_TTL)

• In Eirini, implement an HTTP endpoint that takes JSON (?) and implements the aforementioned API calls

AC: Task execution happens in k8s, while staging and LRPs still run in Diego

Staging also happens in k8s

• In DependencyLocator#bbs_stager_client, return the new OpiClient

AC: Staging happens in k8s, while LRPs still run in Diego

LRPs in k8s

• In DependencyLocator#bbs_apps_client, return the new (enhanced) OpiClient
• Implement the remaining methods of the OpiClient
• In Eirini, implement the HTTP endpoints for the additional API calls

AC: Staging, tasks and apps (LRP) happen in k8s

Consequences / Open Questions

• Since CC already translates app starts into staging + desireLRP, our st8ger might not be needed anymore. Is this true, and is this something we want?
• Since the convergence loop stays in CC, the sink would go away from Eirini as well
• What is upsert_domain being used for?
• Is ping required?