Work Terms
• Having 15+ years hands-on experience in IT including 9 years in Mainframe and 5+ years in Big Data (Kafka, Kafka-Connect, Kafka-Stream, KSQL, Confluent Schema-Registry, Cassandra, Elasticsearch, Scala, Python, Spark, Hive, Hadoop)
• Experience in Confluent Kafka/Apache Kafka:
o Gathering requirement of Application Data Pipeline.
o Analysis of requirement of data pipeline.
o Designing of Architecture Confluent Kafka Pipeline for 2 Data Center.
o Building the Confluent Kafka Cluster for 2 Data Center.
o Standardize the naming convention of Topics, consumers group and Client-id.
o Defining the partition of topics as per number of brokers and consumers.
o Defining replication of topics as per number of brokers.
o Build Confluent KSQL cluster and Kafka connect Cluster.
o Configuring Replication between two Data Center.
o Having hand-on experience to build Confluent Replicator and Apache Mirror-Maker.
o Integration between Kafka connect and Mysql/MariaDB, Snowflake, MongodB & Cassandra.
o Writing Kafka Connectors for MongoDB, Cassandra, MySQL, Snowflake
o Installation and configuration of control center for monitoring the Kafka cluster.
o Integration between Kafka-connect, KSQL, Schema-Registry and Control-Center.
o Installation of Kafka-Manager for monitoring of Kafka cluster, rebalancing of Kafka topics.
o Closely monitoring ISR of topics, lag of consumer-group
o Implementation of LDAP with RBAC security to control center, Kafka-Manager.
o Implementation of SSL between zookeeper and Kafka servers.
o Implementation of SSL between Kafka Servers and Kafka Clients like
o Building data pipeline using Kafka, Scala and Elasticsearch and Cassandra.
o Developed Kafka Producer and Consumer in Scala.
o Building Kafka Streaming using KSQL & Kafka-Connect.
• Experience in Datastax Cassandra / Apache Cassandra:
o Developed ETL / Data Preparation / Pre-Processing jobs in Scala to persist data into Cassandra & Elasticsearch.
o Having experience in Cassandra database modeling