How to consume a Kafka topic

Owned by Peter Atthem
Last updated: 2023-08-31
3 min read

This guides explains how read a topic in Kafka using Integrations Hub for Jira and create a ticket in Jira or update a ticket.

First step is to create an consumer in the Incoming Webhooks section, this will allow iHub to read messages posted on the Kafka queue. That is called the WHEN section of the incoming integration. On the THEN section you will need to specify the Action that iHub will do in Jira. Actions in the THEN sections are defined in the integrations menu. So the incoming integration will pass the data received down to a action (defined in the integrations menu), in many cases there is also a need to route it to the correct integration based on a condition in the message received, this is done in the IF section.

The integration message defined in this tutorial is one for Create Issue and one for Updating an Issue. This is part 2 of the tutorial.

 

Step 1 - Add Incoming Integration

Goto Incoming Webhooks and click Create Rule and select Kafka.
Incoming Webhooks define any incoming data, hence it is extended with Kafka to read topics.

  1. Enter the Kafka Topic name to read from

  2. Enter the Key and Value deserializer.

  3. Select the Action that shall take the input from each Kafka message and perform something in Jira.

  4. optional specify a Data condition

Step 2. - Add Integration to creata a Jira Issue

  1. Click New (Action)

  2. Select Template Category: Jira Core

  3. Select Template: Create Issue

  4. Click Create

  5. Select a basic Auth Jira user see Basic Auth

  6. Add Body

    { "fields": { "project": { "id": "12200" }, "summary": "{{name}}", "issuetype": { "id": "10001" }, "reporter": { "name": "kg" }, "labels": [ "kafka" ], "description":"{{description}}", "customfield_10201":"{{someother_data}}" } }

This will create the issue based on the incoming data from Kafka. In the message we are sending

{ "name":"the issue name", "description":"some description", "someother_data":"100" }

the data in the message gets converted to an object that we can access using mustache syntax {{name}} etc.

Step 3. - Add Integration to update a Jira Issue

  1. Click New (Action)

  2. Select Template Category: Jira Core

  3. Select Template: Edit Issue

  4. Click Create

  5. Change the URL {{baseUrl}}/rest/api/2/issue/{{issue.key}} to {{baseUrl}}/rest/api/2/issue/{{key}}

  6. Select a basic Auth Jira user see Basic Auth

  7. Add Body

    { "fields": { "description":"{{description}}", "customfield_10201":"{{someother_data}}" } }

This will create the issue based on the incoming data from Kafka. In the message we are sending

the data in the message gets converted to an object that we can access using mustache syntax {{key}} which is used for updating the issue

Â