Skip to content

Commit 5dca5b1

Browse files
oleksandr-ncoleksandr-nc
committed
initial K8s support
Signed-off-by: Oleksander Piskun <oleksandr2088@icloud.com>
1 parent d78c25d commit 5dca5b1

4 files changed

Lines changed: 1478 additions & 3 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -545,7 +545,7 @@ Use the Docker Engine `/_ping` endpoint via HaRP’s ExApps HTTP frontend to con
545545
curl -fsS \
546546
-H "harp-shared-key: <HP_SHARED_KEY>" \
547547
-H "docker-engine-port: 24000" \
548-
http://127.0.0.1:8780/exapps/app_api/v1.41/_ping
548+
http://127.0.0.1:8780/exapps/app_api/v1.44/_ping
549549
```
550550
551551
* `24000` is the **default** FRP remote port used by the HaRP container for the **built‑in/local** Docker Engine (enabled when `/var/run/docker.sock` is mounted).

‎development/docs/appapi-emulation-k8s.md‎

Lines changed: 339 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,339 @@
1+
# AppAPI Emulation Guide (HaRP + Kubernetes backend)
2+
3+
This guide documents the `curl` commands used to emulate AppAPI when testing HaRP’s Kubernetes backend.
4+
5+
## Prerequisites
6+
7+
* HaRP is reachable at: `http://nextcloud.local/exapps`
8+
* HaRP was started with the same shared key as used below (`HP_SHARED_KEY`)
9+
* HaRP has Kubernetes backend enabled (`HP_K8S_ENABLED=true`) and can access the K8s API
10+
* `kubectl` is configured to point to the same cluster HaRP uses
11+
* Optional: `jq` for parsing JSON responses
12+
13+
## Environment variables
14+
15+
```bash
16+
export EXAPPS_URL="http://nextcloud.local/exapps"
17+
export APPAPI_URL="${EXAPPS_URL}/app_api"
18+
export HP_SHARED_KEY="some_very_secure_password"
19+
20+
# Optional: Nextcloud base (only used by ExApp container env in this guide)
21+
export NEXTCLOUD_URL="http://nextcloud.local"
22+
```
23+
24+
> Notes:
25+
>
26+
> * All AppAPI-emulation calls go to `$APPAPI_URL/...` and require the header `harp-shared-key`.
27+
> * You can also hit the agent directly on `http://127.0.0.1:8200/...` for debugging, but that bypasses the HAProxy/AppAPI path and may skip shared-key enforcement depending on your routing.
28+
29+
---
30+
31+
## 1) Check if ExApp is present (K8s Deployment exists)
32+
33+
```bash
34+
curl -sS \
35+
-H "harp-shared-key: $HP_SHARED_KEY" \
36+
-H "Content-Type: application/json" \
37+
-X POST \
38+
-d '{
39+
"name": "test-deploy",
40+
"instance_id": ""
41+
}' \
42+
"$APPAPI_URL/k8s/exapp/exists"
43+
```
44+
45+
Expected output:
46+
47+
```json
48+
{"exists": true}
49+
```
50+
51+
or
52+
53+
```json
54+
{"exists": false}
55+
```
56+
57+
---
58+
59+
## 2) Create ExApp (PVC + Deployment with replicas=0)
60+
61+
```bash
62+
curl -sS \
63+
-H "harp-shared-key: $HP_SHARED_KEY" \
64+
-H "Content-Type: application/json" \
65+
-X POST \
66+
-d '{
67+
"name": "test-deploy",
68+
"instance_id": "",
69+
"image": "ghcr.io/nextcloud/test-deploy:latest",
70+
"environment_variables": [
71+
"APP_ID=test-deploy",
72+
"APP_DISPLAY_NAME=Test Deploy",
73+
"APP_VERSION=1.2.1",
74+
"APP_HOST=0.0.0.0",
75+
"APP_PORT=23000",
76+
"NEXTCLOUD_URL='"$NEXTCLOUD_URL"'",
77+
"APP_SECRET=some-dev-secret",
78+
"APP_PERSISTENT_STORAGE=/nc_app_test-deploy_data"
79+
],
80+
"resource_limits": { "cpu": "500m", "memory": "512Mi" }
81+
}' \
82+
"$APPAPI_URL/k8s/exapp/create"
83+
```
84+
85+
Expected output (example):
86+
87+
```json
88+
{"name":"nc-app-test-deploy"}
89+
```
90+
91+
---
92+
93+
## 3) Start ExApp (scale replicas to 1)
94+
95+
```bash
96+
curl -sS \
97+
-H "harp-shared-key: $HP_SHARED_KEY" \
98+
-H "Content-Type: application/json" \
99+
-X POST \
100+
-d '{
101+
"name": "test-deploy",
102+
"instance_id": ""
103+
}' \
104+
"$APPAPI_URL/k8s/exapp/start"
105+
```
106+
107+
Expected: HTTP 204.
108+
109+
---
110+
111+
## 4) Wait for ExApp to become Ready
112+
113+
```bash
114+
curl -sS \
115+
-H "harp-shared-key: $HP_SHARED_KEY" \
116+
-H "Content-Type: application/json" \
117+
-X POST \
118+
-d '{
119+
"name": "test-deploy",
120+
"instance_id": ""
121+
}' \
122+
"$APPAPI_URL/k8s/exapp/wait_for_start"
123+
```
124+
125+
Expected output (example):
126+
127+
```json
128+
{
129+
"started": true,
130+
"status": "running",
131+
"health": "ready",
132+
"reason": null,
133+
"message": null
134+
}
135+
```
136+
137+
---
138+
139+
## 5) Expose + register in HaRP
140+
141+
### 5.1 NodePort (default behavior)
142+
143+
**Minimal (uses defaults, may auto-pick a node address):**
144+
145+
```bash
146+
EXPOSE_JSON=$(
147+
curl -sS \
148+
-H "harp-shared-key: $HP_SHARED_KEY" \
149+
-H "Content-Type: application/json" \
150+
-X POST \
151+
-d '{
152+
"name": "test-deploy",
153+
"instance_id": "",
154+
"port": 23000,
155+
"expose_type": "nodeport"
156+
}' \
157+
"$APPAPI_URL/k8s/exapp/expose"
158+
)
159+
160+
echo "$EXPOSE_JSON"
161+
```
162+
163+
**Recommended (provide a stable host reachable by HaRP):**
164+
165+
```bash
166+
# Example: edge node IP / VIP / L4 LB that forwards NodePort range
167+
UPSTREAM_HOST="172.18.0.2"
168+
169+
EXPOSE_JSON=$(
170+
curl -sS \
171+
-H "harp-shared-key: $HP_SHARED_KEY" \
172+
-H "Content-Type: application/json" \
173+
-X POST \
174+
-d '{
175+
"name": "test-deploy",
176+
"instance_id": "",
177+
"port": 23000,
178+
"expose_type": "nodeport",
179+
"upstream_host": "'"$UPSTREAM_HOST"'"
180+
}' \
181+
"$APPAPI_URL/k8s/exapp/expose"
182+
)
183+
184+
echo "$EXPOSE_JSON"
185+
```
186+
187+
### 5.2 ClusterIP (only if HaRP can reach ClusterIP + resolve service DNS)
188+
189+
```bash
190+
EXPOSE_JSON=$(
191+
curl -sS \
192+
-H "harp-shared-key: $HP_SHARED_KEY" \
193+
-H "Content-Type: application/json" \
194+
-X POST \
195+
-d '{
196+
"name": "test-deploy",
197+
"instance_id": "",
198+
"port": 23000,
199+
"expose_type": "clusterip"
200+
}' \
201+
"$APPAPI_URL/k8s/exapp/expose"
202+
)
203+
204+
echo "$EXPOSE_JSON"
205+
```
206+
207+
### 5.3 Manual (HaRP does not create or inspect any Service)
208+
209+
```bash
210+
EXPOSE_JSON=$(
211+
curl -sS \
212+
-H "harp-shared-key: $HP_SHARED_KEY" \
213+
-H "Content-Type: application/json" \
214+
-X POST \
215+
-d '{
216+
"name": "test-deploy",
217+
"instance_id": "",
218+
"port": 23000,
219+
"expose_type": "manual",
220+
"upstream_host": "exapp-test-deploy.internal",
221+
"upstream_port": 23000
222+
}' \
223+
"$APPAPI_URL/k8s/exapp/expose"
224+
)
225+
226+
echo "$EXPOSE_JSON"
227+
```
228+
229+
---
230+
231+
## 6) Extract exposed host/port for follow-up tests (requires `jq`)
232+
233+
```bash
234+
EXAPP_HOST=$(echo "$EXPOSE_JSON" | jq -r '.host')
235+
EXAPP_PORT=$(echo "$EXPOSE_JSON" | jq -r '.port')
236+
237+
echo "ExApp upstream endpoint: ${EXAPP_HOST}:${EXAPP_PORT}"
238+
```
239+
240+
---
241+
242+
## 7) Check `/heartbeat` via HaRP routing (AppAPI-style direct routing headers)
243+
244+
This checks HaRP’s ability to route to the ExApp given an explicit upstream host/port and AppAPI-style authorization header.
245+
246+
### 7.1 Build `authorization-app-api` value
247+
248+
HaRP typically expects this value to be the **base64 of `user_id:APP_SECRET`** (similar to HTTP Basic without the `Basic ` prefix). For an “anonymous” style request, use `:APP_SECRET`.
249+
250+
```bash
251+
# Option A: anonymous-style
252+
AUTH_APP_API=$(printf '%s' ':some-dev-secret' | base64 | tr -d '\n')
253+
254+
# Option B: user-scoped style (example user "admin")
255+
# AUTH_APP_API=$(printf '%s' 'admin:some-dev-secret' | base64 | tr -d '\n')
256+
```
257+
258+
### 7.2 Call heartbeat
259+
260+
```bash
261+
curl -sS \
262+
"http://nextcloud.local/exapps/test-deploy/heartbeat" \
263+
-H "harp-shared-key: $HP_SHARED_KEY" \
264+
-H "ex-app-version: 1.2.1" \
265+
-H "ex-app-id: test-deploy" \
266+
-H "ex-app-host: $EXAPP_HOST" \
267+
-H "ex-app-port: $EXAPP_PORT" \
268+
-H "authorization-app-api: $AUTH_APP_API"
269+
```
270+
271+
If this fails with auth-related errors, verify:
272+
273+
* `APP_SECRET` in the ExApp matches what you used here,
274+
* your HaProxy config expectations for `authorization-app-api` (raw vs base64).
275+
276+
---
277+
278+
## 8) Stop and remove (API-based cleanup)
279+
280+
### Stop ExApp (scale replicas to 0)
281+
282+
```bash
283+
curl -sS \
284+
-H "harp-shared-key: $HP_SHARED_KEY" \
285+
-H "Content-Type: application/json" \
286+
-X POST \
287+
-d '{
288+
"name": "test-deploy",
289+
"instance_id": ""
290+
}' \
291+
"$APPAPI_URL/k8s/exapp/stop"
292+
```
293+
294+
### Remove ExApp (Deployment + optional PVC; Service may be removed depending on HaRP version)
295+
296+
```bash
297+
curl -sS \
298+
-H "harp-shared-key: $HP_SHARED_KEY" \
299+
-H "Content-Type: application/json" \
300+
-X POST \
301+
-d '{
302+
"name": "test-deploy",
303+
"instance_id": "",
304+
"remove_data": true
305+
}' \
306+
"$APPAPI_URL/k8s/exapp/remove"
307+
```
308+
309+
---
310+
311+
## Useful `kubectl` commands (debug / manual cleanup)
312+
313+
### Check resources
314+
315+
```bash
316+
kubectl get deploy,svc,pvc -n nextcloud-exapps -o wide | grep -E 'test-deploy|NAME' || true
317+
kubectl get pods -n nextcloud-exapps -o wide
318+
```
319+
320+
### Delete Service (if it was exposed and needs manual cleanup)
321+
322+
```bash
323+
kubectl delete svc nc-app-test-deploy -n nextcloud-exapps
324+
```
325+
326+
### Delete Deployment
327+
328+
```bash
329+
kubectl delete deployment nc-app-test-deploy -n nextcloud-exapps
330+
```
331+
332+
### Delete PVC (data)
333+
334+
PVC name is derived from `nc_app_test-deploy_data` and sanitized for K8s, typically:
335+
`nc-app-test-deploy-data`
336+
337+
```bash
338+
kubectl delete pvc nc-app-test-deploy-data -n nextcloud-exapps
339+
```

‎development/redeploy_host_k8s.sh‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/bin/sh
2+
# SPDX-FileCopyrightText: 2025 Nextcloud GmbH and Nextcloud contributors
3+
# SPDX-License-Identifier: AGPL-3.0-or-later
4+
5+
# This file can be used for development for the "manual install" deployment type when FRP is disabled.
6+
# For Julius Docker-Dev, you need to additionally edit the `data/nginx/vhost.d/nextcloud.local_location` file,
7+
# changing `appapi-harp` to `172.17.0.1` and restart the "proxy" container.
8+
9+
docker container remove --force appapi-harp
10+
11+
docker build -t nextcloud-appapi-harp:local .
12+
13+
docker run \
14+
-e HP_SHARED_KEY="some_very_secure_password" \
15+
-e NC_INSTANCE_URL="http://nextcloud.local" \
16+
-e HP_LOG_LEVEL="debug" \
17+
-e HP_VERBOSE_START="1" \
18+
-e HP_K8S_ENABLED="true" \
19+
-e HP_K8S_API_SERVER="https://127.0.0.1:37151" \
20+
-e HP_K8S_BEARER_TOKEN="eyJhbGciOiJSUzI1NiIsImtpZCI6InJwSHRRN04wV0RwcHFiVEtJLVdHblpGTllKMGJwc3NZZ2tZYjRrREdhcEkifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNzczNTgwODI1LCJpYXQiOjE3NjQ5NDA4MjUsImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwianRpIjoiOTBmOTE0MjUtMDYxMy00YzM4LTllNmQtN2U2Y2I1Njk4MDhhIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJuZXh0Y2xvdWQtZXhhcHBzIiwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImhhcnAtZXhhcHBzIiwidWlkIjoiMzI2ZTA5NzEtMGIyOC00NzBkLTlmZTUtMDRjMTc0YjE2ODQ2In19LCJuYmYiOjE3NjQ5NDA4MjUsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpuZXh0Y2xvdWQtZXhhcHBzOmhhcnAtZXhhcHBzIn0.TSDUSEe0NuFMhycrK1XHNgBV3-L70qqLfCR2-x0XSSXGSsms1ZzxKbSnsDDCstAGg6-ZtlJroWFZZiFeZ2E2j53z2-Tt4lXM-ZdH7qqhjsxSh5Ya7l3ncMSS0Tw1YPaEsOJmpXCiDH9KE4g-KyLeSJU5Rqonc5fuWJwDd68wpY8SB2qkgbtr250Srk4nYw28MyxhgXwHvOSIrDhqmGR-NPPmSeoa9u9etAD9qjfCPauF0BYDBcVHGKR2kJL5oGw9-tRs6FqBAt5U-y4Jx6y0Q1RPptdbpHGY9KmSGHIqsLrkJl7lgjlZh4mb2wofwypJvBd2hW_dgS1RTrzcTjoYsQ" \
21+
-e HP_K8S_NAMESPACE="nextcloud-exapps" \
22+
-e HP_K8S_VERIFY_SSL="false" \
23+
-v /var/run/docker.sock:/var/run/docker.sock \
24+
-v `pwd`/certs:/certs \
25+
--name appapi-harp -h appapi-harp \
26+
--restart unless-stopped \
27+
--network=host \
28+
-d nextcloud-appapi-harp:local

0 commit comments

Comments
 (0)