--- title: "Replicating Data to Apache Kafka" slug: "replicating-data-to-apache-kafka" updated: 2024-04-15T16:12:32Z published: 2024-04-15T16:12:32Z canonical: "skthelp.syniti.com/replicating-data-to-apache-kafka" --- > ## Documentation Index > Fetch the complete documentation index at: https://skthelp.syniti.com/llms.txt > Use this file to discover all available pages before exploring further. # Replicating Data to Apache Kafka or Confluent Cloud With Syniti Replicate, you can replicate relational data to a Kafka stream using JSON, AVRO, CSV or XML serialization of the entire record. Kafka is currently supported as a target both in refresh and mirroring. Every session creates either a .ref or .mir file with the content of the replication. To define a target connection for Kafka: 1. In the Metadata Explorer, choose **Targets**, then **Add New Connection** from the right mouse button. 2. In the Add Target Connection wizard, type a name for the connection and choose **Apache** **Kafka** in the **Data****base** field. 3. In the Set Connection String screen **Output Folder** field, enter the path to a directory to contain the output files and schema. 4. Choose values for the following fields: - **Server** The server name or IP address of the system running Kafka. - **Port** The port number of the server running Kafka. - **Output Folder** The schema name and location to hold config files for the Kafka objects. - **Group ID** Kafka property not used by Syniti Replicate. A string that uniquely identifies the group of consumer processes to which this consumer belongs. By setting the same group id multiple processes indicate that they are all part of the same consumer group. - **Connect Timeout** Amount of time before the connection will throw an exception. - **Default Topic** All Kafka streams need to be targeted to a specific topic. In Syniti Replicate you can indicate a default topic for all replications in that connection. Alternatively, if the Default Topic is left blank, every replication will need to specify a topic name. - **Serialization** Possible types of message serialization. Currently supported: - 0 – JSON - 1 – CSV - 2 – XML - 2 – AVRO - **Schema Registry URL** Used only when AVRO serialization is selected. Because Avro does not include any schema information (column names, types, sizes and format) within the message that is being sent, Kafka producers and consumers use an implicit “contract” to determine how the message is serialized. Schema Registry helps ensure that this contract is met with compatibility checks and it provides centralized schema management and compatibility checks as schemas evolve. - **Auto Offset Reset** Kafka property not used by Syniti Replicate. Determines how to proceed when there is no initial offset in ZooKeeper or if an offset is out of range: - **smallest:** automatically reset the offset to the smallest offset; - **largest:** automatically reset the offset to the largest offset; - **anything else:** throw exception to the consumer. - **Use One Producer Per Group** True by default. If True, the same Producer is used for all streams within the group (this has a better impact on performance). 5. If using Kafka with Kerberos security, there are additional connection values to set. Download the [Kafka setup guide](https://support.syniti.com/hc/en-us/articles/5726159606679) in the [Help Center](https://support.syniti.com/) for complete details You can leave the **ExtendedProperties** field blank. 6. Click **Next** to view the **Select Tables** screen. If this is the first time you have created a connection using the output folder defined above, the table display will be empty. 7. Click **Next** to display the **Actions** screen. 8. Optionally choose to continue with creating replications once the wizard is complete. 9. Click **Next** to display the summary, then click **Finish** to create the connection. The next step is to [](https://sdrhelp.syniti.com/1060/UsersGuide/Adding%20Target%20Output%20to%20the%20Metadata.htm)[add target output representation](https://sdrhelp.syniti.com/1060/UsersGuide/Adding%20Target%20Output%20to%20the%20Metadata.htm) to the Metadata Explorer. This will be represented as relational tables. Now [set up replications](https://sdrhelp.syniti.com/1060/UsersGuide/GUI/Wizards/Replication_Wizard.htm) from whichever source connection you have defined to the Kafka stream.