- Installation
Download or clone the repository source code to your workstation.
- Trough terminal:
git clone https://github.com/industrial-edge/Apache-Kafka-Connector.git
- Trough VSCode:
CTRL+↑ SHIFT+P or F1 to open VSCode's command pallette and typegit clone
:
- Navigate into
src/consumer
andsrc/producer
and find the file namedDockerfile.example
. TheDockerfile.example
is an example Dockerfile that can be used to build the docker image(s) of the service(s) that runs in this application example. If you choose to use these, rename them toDockerfile
before proceeding - Open a console in the root folder (where the
docker-compose
file is) - Use the
docker compose build
(replaces the olderdocker-compose build
) command to build the docker image of the service which is specified in the docker-compose.yml file. - These Docker images can now be used to build your app with the Industrial Edge App Publisher
docker images
can be used to check for the images
You find below a short description how to publish your application in your IEM.
For more detailed information please see the section for uploading apps to the IEM.
- Connect your Industrial Edge App Publisher to your Docker engine
- Connect your Industrial Edge App Publisher to your Industrial Edge Management
-
Create a new Project or select a existing one
-
Create new Application
-
Import the docker-compose file using the Import YAML button
-
The warnings
Build (Detail) (services >> kafka-producer >> build) is not supported.
Build (Detail) (services >> kafka-consumer >> build) is not supported.
can be ignored -
Click on
Review
andValidate & Create
. -
Start Upload to transfer the app to Industrial Edge Managment
-
Further information about using the Industrial Edge App Publisher can be found in the IE Hub
- Configure a user with password in the Databus for the OPC UA Connector and the Apache Kafka Connector Application for publishing and subscribing to topics on the Databus.
User name: edge Password: edge Topic: ie/# Permission: Publish and Subscribe
- Add the PLC as a data source with data source type e.g. OPC-UA.
- Add variables to collect data.
- Enter Databus credentials
The Apache Kafka Connector can be configured with a form. The form is based on JSONForms. If no configuration is provided during app installation, the application uses default values seen in the following json-file.
{
"MQTT": {
"HOST": "ie-databus",
"PORT": "1883",
"USERNAME": "edge",
"PASSWORD": "edge"
},
"KAFKA": {
"HOST": "192.168.253.143",
"PORT": "9092"
},
"CONSUMER_TOPICS": [
{
"MQTT": "ie/d/kafka",
"KAFKA": "EdgeDevice"
}
],
"PRODUCER_TOPICS": [
{
"MQTT": "ie/d/j/simatic/v1/opcuac1/dp/r/#",
"KAFKA": "EdgeDevice",
"KEY": "OPC-UA-Connector"
}
]
}
- HOST: This is the service name of the Databus
- PORT: This is the port of the Databus
- USER, PASSWORD: The user and password are configured in the Databus and used in the OPC UA Connector for accessing (publish, subscribe) to topics on the IE Databus
- HOST: This is the DNS-Name or IP-Adress of the Kafka-Broker
- PORT: This is the port of the Kafka-Broker
This is a Topic Map list, mapping Apache Kafka topics to Databus topics
- MQTT Topic: Topic on Databus
- Kafka Topic: Topic on Apache Kafka
This is a Topic Map list, mapping Databus topics to Apache Kafka topics
- MQTT Topic: Topic on Databus
- Kafka Topic: Topic on Apache Kafka
- Kafka Key: Key of Kafka-Message (message on Kafka consists of a key value pair)
- Select your application in Industrial Edge App Publisher
- Add Configuration to application
Display Name: Configuration Description: JSONForms Configuration Host Path: ./cfg-data/ Add Template - Name: JSONForms - Description: JSONForms Configuration - JSON Schema: set checkbox
JSONForms file is located here
Fill out Input-Form and select checkbox to select configuration(check box green). Insert for example the default values like shown before.
- MQTT Broker IP: optional
- Port: optional
- Usernname: required
- Password: required
- Host: required
- Port: required
- MQTT: required
- Kafka: required
- Key: required
- MQTT: required
- Kafka: required
Install Edge Application to Industrial Edge Device
Setup Apache Kafka Broker to test Apache Kafka Connector. To startup a test environment of Apache Kafka a docker-compose.yml can be found here
- OS: Windows or Linux
- Docker
- Docker Compose
Adjust docker-compose.yml according to your environment. Insert IP address of your environment in "KAFKA_ADVERTISED_LISTENERS".
environment:
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.253.143:9092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
Open terminal where Apache Kafka Broker docker-compose.yml is located and execute:
docker-compose up
Install IE Flow Creator on Industrial Edge Device. Use flow provided here. Insert Security Settings in MQTT-Node before using the Flow.