Kafka Engineer – InnovaziT (A Happiest Minds company)

We are seeking a highly skilled Confluent Kafka Engineer to join our dynamic team. The ideal candidate will be responsible for the development, User Acceptance Testing (UAT), and production support of our Confluent Kafka-based systems. This role requires deep expertise in Kafka architecture, including Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect. The engineer will collaborate closely with cross-functional teams to ensure the smooth operation of our data streaming services and provide support across various environments.

Key Responsibilities:

Development:

• Design, develop, and implement Kafka-based solutions to meet business requirements, utilizing Confluent Control Center, KSQLDB, Kafka Streams, and Kafka Connect.
• Write and maintain high-quality code for Kafka producers, consumers, and stream processing applications.
• Develop and manage Kafka connectors for seamless integration with external systems, ensuring data consistency and reliability.
• Utilize Kafka Streams for real-time processing of streaming data, transforming and enriching data as it flows through the pipeline.
• Employ KSQLDB for stream processing tasks, including real-time analytics and transformations.
• Collaborate with data engineers, software developers, and DevOps teams to integrate Kafka solutions with existing systems.
• Ensure all Kafka-based solutions are scalable, secure, and optimized for performance.

User Acceptance Testing (UAT):

• Develop and execute UAT plans to validate Kafka solutions, including components such as Kafka Streams, KSQLDB, and Kafka Connect, before deployment to production.
• Work closely with QA teams to identify and resolve defects during the UAT phase.
• Ensure all UAT activities comply with the organization’s standards and best practices.
• Provide detailed reports on UAT outcomes and work with developers to implement necessary fixes.

Production Support:

• Monitor and maintain Kafka clusters, including components like Confluent Control Center, to ensure high availability and reliability of data streaming services.
• Troubleshoot and resolve issues related to Kafka performance, latency, and data integrity, including issues specific to Kafka Streams, KSQLDB, and Kafka Connect.
• Perform routine maintenance tasks such as patching, upgrades, and backups for Kafka clusters and associated components.
• Implement monitoring solutions to proactively identify and mitigate potential production issues, leveraging Confluent Control Center for comprehensive cluster visibility.
• Provide 24/7 support for production systems, including participation in on-call rotation.

Job Details

Preferred Candidate

Suite de l’annonce…