//

Offloading and more from Reedelk Data Integration Services through Kong Enterprise

20.9.2020 | 6 minutes of reading time

The last post in our series on Reedelk was about implementing an ETL process that can be triggered by a rest endpoint. Now this service should be made available to the “outside world” in a secured way. But how can services in modernization projects be secured in a simple way? In this case it makes sense to think about using an API Gateway. What an API Gateway is and how it works has already been discussed in several posts on our blog. With regard to the enterprise context in which the project is moving, the project coincides with the already known Kong Enterprise, which is currently available in version 2.1. If only the pure gateway would be discussed here, Kong could also be used in the Community Edition.

The Encounter of Elk and Gruce

The basis of all following considerations and corresponding implementations is again the “API first” approach. In the previous post , the API specification was used to create the integration service in basis. Now the spec is intended to promote the service at the gateway and also to make it known to users, i.e. developers, via a so-called developer portal. First of all the Docker Compose file will be extended by Kong Enterprise and another PostgreSQL.

1version: "3"
2networks:
3  int-net:
4    driver: bridge
5services:
6  bookings:
7    image: postgres:11.7-alpine
8    networks:
9      - int-net
10    container_name: bookings
11    environment:
12      - POSTGRES_HOST_AUTH_METHOD=trust
13    ports:
14      - 5433:5432
15    volumes:
16      - ./db/10_init.sql:/docker-entrypoint-initdb.d/10_init.sql
17      - ./postgres-bookings:/var/lib/postgresql/data
18    healthcheck:
19      test: [ "CMD", "pg_isready", "-U", "postgres" ]
20      interval: 90s
21      timeout: 5s
22      retries: 5
23  booking-integration-service:
24    build: .
25    depends_on:
26      - bookings
27    ports:
28      - 8484:8484
29      - 9988:9988
30    networks:
31      - int-net
32    container_name: booking-integration-service
33  kong-db:
34    image: postgres:9.6
35    restart: always
36    networks:
37      - int-net
38    container_name: kong-db
39    environment:
40      - POSTGRES_USER=kong
41      - POSTGRES_PASSWORD=kong
42      - POSTGRES_DB=kong
43    volumes:
44      - ./postgres-kong:/var/lib/postgresql/data
45    healthcheck:
46      test: ["CMD", "pg_isready", "-U", "postgres"]
47      interval: 10s
48      timeout: 5s
49      retries: 5
50  kong-migration:
51    image: kong-docker-kong-enterprise-edition-docker.bintray.io/kong-enterprise-edition:latest
52    command: kong migrations bootstrap
53    restart: on-failure
54    networks:
55      - int-net
56    container_name: kong-migration
57    environment:
58      - KONG_PG_HOST=kong-db
59      - KONG_PG_PASSWORD=kong
60      - KONG_DATABASE=postgres
61      - 'KONG_LICENSE_DATA=${KONG_LICENSE_DATA}'
62      - KONG_PASSWORD=${KONG_PASSWORD}
63    depends_on:
64      - kong-db
65  kong-enterprise:
66    image: kong-docker-kong-enterprise-edition-docker.bintray.io/kong-enterprise-edition:latest
67    depends_on:
68      - kong-migration
69      - kong-db
70    restart: always
71    networks:
72      - int-net
73    container_name: kong-enterprise
74    ports:
75      - 8000:8000
76      - 8001:8001
77      - 8002:8002
78      - 8003:8003
79      - 8004:8004
80      - 8005:8005
81    environment:
82      - KONG_ENFORCE_RBAC=on
83      - KONG_ADMIN_GUI_AUTH=basic-auth
84      - KONG_ADMIN_GUI_SESSION_CONF=${KONG_ADMIN_GUI_SESSION_CONF}
85      - KONG_AUDIT_LOG=on
86      - KONG_LOG_LEVEL=debug
87      - KONG_PORTAL_GUI_HOST=localhost:8003
88      - KONG_PORTAL_GUI_PROTOCOL=http
89      - KONG_PORTAL=on
90      - KONG_PORTAL_AUTH=basic-auth
91      - KONG_PORTAL_SESSION_CONF=${KONG_PORTAL_SESSION_CONF}
92      - KONG_ADMIN_GUI_URL=http://localhost:8002
93      - KONG_DATABASE=postgres
94      - KONG_PG_PASSWORD=kong
95      - KONG_PG_HOST=kong-db
96      - KONG_PG_DATABASE=kong
97      - KONG_CASSANDRA_CONTACT_POINTS=cassandra
98      - 'KONG_LICENSE_DATA=${KONG_LICENSE_DATA}'
99      - KONG_VITALS=on
100      - KONG_ANONYMOUS_REPORTS=off
101      - KONG_PROXY_ACCESS_LOG=/dev/stdout
102      - KONG_ADMIN_ACCESS_LOG=/dev/stdout
103      - KONG_PROXY_ERROR_LOG=/dev/stderr
104      - KONG_ADMIN_ERROR_LOG=/dev/stderr
105      - KONG_PROXY_LISTEN=0.0.0.0:8000, 0.0.0.0:8443 ssl
106      - KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl
107      - KONG_ADMIN_GUI_LISTEN=0.0.0.0:8002, 0.0.0.0:8445 ssl
108      - KONG_PORTAL_GUI_LISTEN=0.0.0.0:8003, 0.0.0.0:8446 ssl
109      - KONG_PORTAL_API_LISTEN=0.0.0.0:8004, 0.0.0.0:8447 ssl
110      - KONG_VITALS_STRATEGY=database
111

Now the following architecture is available:

From OpenAPI spec to configuration as code

To promote the integration service at the gateway, the first step is to use Insomnia Designer . Through the “Kong Bundle” plugin, Insomnia Designer is able to create a configuration for the Kong gateway based on an OpenAPI specification. By adding an OpenAPI extension (x-), configuration parameters for services can be included in the API specification. To keep the demo YAML file simple, the plugin configuration is located directly on the server level to provide a security configuration for the whole service.

1openapi: 3.0.2
2info:
3  title: Bookings API
4  description: API for Bookings
5  version: 0.2.0
6servers:
7  - url: http://host.docker.internal:8484/
8    x-kong-plugin-key-auth:
9      name: key-auth
10      enabled: true
11      config:
12        key_names: [api_key, apikey]
13        key_in_body: false
14        hide_credentials: true
15paths:
16  /bookings:
17

Now in Insomnia Designer, the declarative configuration can be created with a single click and must then be manually copied to a corresponding file in the repository. Also a new workspace is created in Kong Enterprise.

1http :8001/workspaces name=ccPlayground Kong-Admin-Token:<needstobesetinenvfile>
2deck dump --workspace ccPlayground --skip-workspace-crud --headers kong-admin-token:<needstobesetinenvfile>
3

By using DecK , a configuration tool for the Kong Gateway, the configuration is synchronized and can be used directly.

1deck sync --workspace ccPlayground --skip-workspace-crud --headers kong-admin-token:<needstobesetinenvfile>
2

The service created with Reedelk is registered at the gateway. Calling the service route http :8001/services Kong-Admin-Token: returns

1HTTP/1.1 200 OK
2Access-Control-Allow-Credentials: true
3Access-Control-Allow-Origin: http://localhost:8002
4Connection: keep-alive
5Content-Length: 392
6Content-Type: application/json; charset=utf-8
7Date: Tue, 15 Sep 2020 09:55:45 GMT
8Server: kong/2.1.3.0-enterprise-edition
9X-Kong-Admin-Latency: 48
10X-Kong-Admin-Request-ID: uangDLXehlqnehUqqZBV1B95Jb6W8CCo
11vary: Origin
12
13{
14    "data": [
15        {
16            "ca_certificates": null,
17            "client_certificate": null,
18            "connect_timeout": 60000,
19            "created_at": 1600150691,
20            "host": "localhost",
21            "id": "bb9d1b9c-d1e4-4271-92cf-23e32544c72f",
22            "name": "Bookings_API",
23            "path": "/",
24            "port": 8484,
25            "protocol": "http",
26            "read_timeout": 60000,
27            "retries": 5,
28            "tags": [
29                "OAS3_import"
30            ],
31            "tls_verify": null,
32            "tls_verify_depth": null,
33            "updated_at": 1600150691,
34            "write_timeout": 60000
35        }
36    ],
37    "next": null
38}
39

The service with its end points is now behind the gateway. To use the KeyAuth plugin, a consumer must still be created. For this consumer a key will be created as well.

1http POST :8001/ccPlayground/consumers username=bookings Kong-Admin-Token:<needstobesetinenvfile>
2http POST :8001/ccPlayground/consumers/bookings/key-auth Kong-Admin-Token:<needstobesetinenvfile>
3

The key (visible in the lower code block) can now be used for the call via the gateway.

1HTTP/1.1 201 Created
2Access-Control-Allow-Credentials: true
3Access-Control-Allow-Origin: http://localhost:8002
4Connection: keep-alive
5Content-Length: 190
6Content-Type: application/json; charset=utf-8
7Date: Tue, 15 Sep 2020 14:03:16 GMT
8Server: kong/2.1.3.0-enterprise-edition
9X-Kong-Admin-Latency: 89
10X-Kong-Admin-Request-ID: 0DaEWPpwa2IZ0xTH9X08BLjI3Cl20t2l
11vary: Origin
12
13{
14    "consumer": {
15        "id": "c06250bf-a018-478a-a60b-bd739b1af820"
16    },
17    "created_at": 1600178596,
18    "id": "0b4ea73e-dde8-4c2f-8b6c-0ecd0c227da1",
19    "key": "S0llbYJjebrNoCG4PURHyjNPUK2Yv9Ds",
20    "tags": null,
21    "ttl": null
22}
23

Calling the route http :8000/bookings/00002D apikey:S0llbYJjebrNoCG4PURHyjNPUK2Yv9Ds on the gateway now returns the following response.

1HTTP/1.1 200 OK
2Connection: keep-alive
3Content-Length: 81
4Content-Type: application/json
5Via: kong/2.1.3.0-enterprise-edition
6X-Kong-Proxy-Latency: 1
7X-Kong-Upstream-Latency: 254
8
9[
10    {
11        "book_date": "2017-05-20 15:45:00.0",
12        "book_ref": "00002D",
13        "total_amount": 114700
14    }
15]
16

Adding some developer experience

After the Booking Integration Service has been granted an authorization, the main focus will be on the developer portal. For this purpose it will be activated via the Admin Rest API of Kong Enterprise.

1http PATCH :8001/workspaces/ccPlayground Kong-Admin-Token:<needstobesetinenvfile> config.portal=true -f
2

Subsequently, Insomnia Designer is used to deploy the API spec to the existing portal.

With the portal one wants to deliver an improved user experience (UX), in particular a developer experience (DX) for APIs. The simplest form, more precisely the entry page of such a portal is the catalog of all available APIs, as shown in the picture.

Kong Enterprise is able to create a developer portal for each workspace, either via the Admin Rest API or the Kong Manager. Through workspaces, current APIs can be grouped according to individual categorization. This can also be a first step towards API as a Product . Each portal can be completely customized to individual needs. For the developer of a possible client, supported by the plugin “Application Registration”, the DX can be improved even further.

1http POST :8001/ccPlayground/services/Bookings_API/plugins name=application-registration Kong-Admin-Token:<needstobesetinenvfile> config.auto_approve=false config.description="All about bookings" config.display_name=Bookings config.show_issuer=false -f
2

With the help of the plugin, client applications can now be registered directly via the portal and coupled with the corresponding services.


This form of self-service helps enormously to further promote the distribution and use of the APIs created.

With these few steps, the integration service has now been secured by a gateway, in particular Kong Enterprise, and at the same time access for APIs has been improved through a developer portal. The use of API first, Kong’s Admin Rest API and Configuration as Code also shows which steps can be automated in terms of a CI/CD pipeline.
This is now a way to integrate Reedelk with Kong. In a following blogpost I would like to introduce the Kong Reedelk Transformer plugin, which is another approach to integrate the two components. The sources for the demo project are available at GitHub .

share post

Likes

0

//

More articles in this subject area\n

Discover exciting further topics and let the codecentric world inspire you.

//

Gemeinsam bessere Projekte umsetzen

Wir helfen Deinem Unternehmen

Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.

Hilf uns, noch besser zu werden.

Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.