Kafka Connect Number type fields

Refresh

November 2018

Views

305 time

2

When i would like to use Kafka connect with source RDBMS which is Oracle , Number type fields are seen as bytes like below,

Column "ID" with value "4" as number has been sent ,but at consumer console this value has been seen as "ID":"BA=="

What can i do in order to solve this issue ?

Kafka connect is started with below command

connect-standalone ./etc/kafka/connect-standalone.properties /home/kafka/oracle.properties.test

######## connect-standalone.properties

# These are defaults. This file just demonstrates how to override some settings.
bootstrap.servers=kafkaserver01.localdomain:9092


# The converters specify the format of data in Kafka and how to translate it into Connect data. Every Connect user will
# need to configure these based on the format they want their data in when loaded from or stored into Kafka

key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter

# Converter-specific settings can be passed in by prefixing the Converter's setting with the converter we want to apply
# it to
key.converter.schemas.enable=true
value.converter.schemas.enable=true

# The internal converter used for offsets and config data is configurable and must be specified, but most users will
# always want to use the built-in default. Offset and config data is never visible outside of Kafka Connect in this format.
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false

offset.storage.file.filename=/tmp/connect.offsets
# Flush much faster than normal, which is useful for testing/debugging
offset.flush.interval.ms=10000



######## /home/kafka/oracle.properties.test Configuration File 

name=oracle-connect-test1
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
topic.prefix=
connection.url=jdbc:oracle:thin:test/[email protected]:1521:testdb
table.whitelist=TEST1,TEST2
mode=timestamp
timestamp.column.name=CDC_TIMESTAMP

## Console Consumer

kafka-console-consumer --bootstrap-server kafkaserver01.localdomain:9092  --topic TEST1

Thanks.

1 answers

0

Я нашел решение, пожалуйста, добавьте ниже конфигурации в свойствах источника коннекторов

numeric.precision.mapping = верно

она отключит кодирующая числовое значение на тему

с новой версией Кафки-подключения-JDBC-4.1.1

Вы можете использовать свойство

numeric.mapping=best_fit

для наилучшего результата