1110 data engineer jobs found in Chennai

Company Description Job Description Responsible to assemble large, complex sets of data that meet non-functional and functional business requirements. Responsible to identify, design and implement internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes. Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using Azure, Databricks and SQL technologies Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues Responsible for the transformation of conceptual algorithms from R&D into efficient, production ready code. The data developer must have a strong mathematical background in order to be able to document and maintain the code Responsible for integrating finished models into larger data processes using UNIX scripting languages such...

About Kinaxis Elevate your career journey by embracing a new challenge with Kinaxis. We are experts in tech, but it's really our people who give us passion to always seek ways to do things better. As such, we're serious about your career growth and professional development, because People matter at Kinaxis. In 1984, we started out as a team of three engineers based in Ottawa, Canada. Today, we have grown to become a global organization with over 2000 employees around the world, and support 40,000+ users in over 100 countries. As a global leader in end-to-end supply chain management, we enable supply chain excellence for all industries. We are expanding our team in Chennai and around the world as we continue to innovate and revolutionize how we support our customers. Our journey in India began in 2020 and we have been growing steadily since then! Building a high-trust and high-performance culture is important to us and we are proud to be Great Place to Work® CertifiedTM. Our...

Job Description: Qualifications: As a Data Engineer, you will play a key role in this exciting journey. Your contributions will go beyond coding, as you'll help bring life to ideas, transforming innovative ideas into tangible solutions that directly impact our business and customers. You'll work in an innovative, fast-paced environment, collaborating with bright minds while enjoying a balance between strategic and hands-on work. We value continuous learning, and you will have the chance to expand your skillset, mastering new tools and technologies that advance our company's goals. We look forward to welcoming a committed team player who thrives on creating value through innovative solutions and is eager to make a significant impact. You will Perform detailed analysis of raw data sources by applying business context and collaborate with cross-functional teams to transform raw data into curated & certified data assets to be used for ML and BI use cases. Create...

Description AWS Data Engineer 1. Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming from various sources. Perform data cleansing, data validation etc 2. Hands on ETL developer who is good at python, SQL. AWS services like glue, glue crawlers, lambda, red shift, athena, s3, EC2, IAM, Monitoring and Logging mechanisms- AWS cloudwatch, setting up alerts. 3. Deployment knowledge on cloud. Integrate CI/CD pipeline to build artifacts and deploy changed to higher Environments. 4. Scheduling frame works Airflow, AWS Step functions 5. Excellent Communication skills, should be able to work collaboratively with other teams

Description AWS Data Engineer Seeking a developer who has good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other AWS serverless resources. Can optimize data models for performance and efficiency. Able to write SQL queries to support data analysis and reporting Design, implement, and maintain the data architecture for all AWS data services. Work with stakeholders to identify business needs and requirements for data-related projects Design and implement ETL processes to load data into the data warehouse Good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other AWS serverless resources.

Description AWS Data Engineer Primary Skills Data architecture for all AWS data services ETL processes to load data into the data warehouse Athena, Python code, Glue Lambda, DMS , RDS, Redshift Cloud Formation Write SQL queries to support data analysis and reporting reports and dashboards to visualize data JD Seeking a developer who has good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other AWS serverless resources. Can optimize data models for performance and efficiency. Able to write SQL queries to support data analysis and reporting Responsibility Design, implement, and maintain the data architecture for all AWS data services Work with stakeholders to identify business needs and requirements for data-related projects Design and implement ETL processes to load data into the data warehouse Good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other

Description AWS Data Engineer Seeking a developer who has good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other AWS serverless resources. Can optimize data models for performance and efficiency. Able to write SQL queries to support data analysis and reporting Design, implement, and maintain the data architecture for all AWS data services. Work with stakeholders to identify business needs and requirements for data-related projects Design and implement ETL processes to load data into the data warehouse Good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other AWS serverless resources.

Description AWS Data Engineer Seeking a developer who has good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other AWS serverless resources. Can optimize data models for performance and efficiency. Able to write SQL queries to support data analysis and reporting Design, implement, and maintain the data architecture for all AWS data services. Work with stakeholders to identify business needs and requirements for data-related projects Design and implement ETL processes to load data into the data warehouse Good Experience in Athena, Python code, Glue, Lambda, DMS , RDS, Redshift Cloud Formation and other AWS serverless resources.

Description AWS Data Engineer 1. Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming from various sources. Perform data cleansing, data validation etc 2. Hands on ETL developer who is good at python, SQL. AWS services like glue, glue crawlers, lambda, red shift, athena, s3, EC2, IAM, Monitoring and Logging mechanisms- AWS cloudwatch, setting up alerts. 3. Deployment knowledge on cloud. Integrate CI/CD pipeline to build artifacts and deploy changed to higher Environments. 4. Scheduling frame works Airflow, AWS Step functions 5. Excellent Communication skills, should be able to work collaboratively with other teams

Description AWS Data Engineer Primary Skills AWS Python AWS Data Engineer Strong Experience in Python and Spark SQL Working experience in AWS Glue Working experience in AWS Boto3 framework Strong working experience in AWS services (DynamoDB, API Gateway, Lambda, EC2); Strong real-life experience in python development especially in PySpark in AWS Cloud environment. Must have created REST API with AWS Lambda Experience with source control systems such as Git, Bitbucket, and Jenkins build and continuous integration tools.

Description AWS Data Engineer Primary Skills AWS Python JD Strong Experience in Python and Spark SQL Working experience in AWS Glue Working experience in AWS Boto3 framework Strong working experience in AWS services (DynamoDB, API Gateway, Lambda, EC2); Strong real-life experience in python development especially in PySpark in AWS Cloud environment. Must have created REST API with AWS Lambda Experience with source control systems such as Git, Bitbucket, and Jenkins build and continuous integration tools.

Your Role and Responsibilities As Data Engineer at IBM you will harness the power of data to unveil captivating stories and intricate patterns. You'll contribute to data gathering, storage, and both batch and real-time processing. Collaborating closely with diverse teams, you'll play an important role in deciding the most suitable data management systems and identifying the crucial data required for insightful analysis. As a Data Engineer, you'll tackle obstacles related to database integration and untangle complex, unstructured data sets. In this role, your responsibilities may include: Implementing and validating predictive models as well as creating and maintain statistical models with a focus on big data, incorporating a variety of statistical and machine learning techniques Designing and implementing various enterprise seach applications such as Elasticsearch and Splunk for client requirements Work in an Agile, collaborative environment, partnering with other...

Experience Required: 16+ years Location: Remote Education: B.E / B.Tech / M.Tech from a recognized university/college Looking for candidates with Backend (Dsitributed system) & Data platform experience Key Skills: 15+ years in building scalable distributed data systems Proficient in Python, Java, or Golang Strong data structures and algorithms knowledge Experience with automated testing and code coverage Real-time stream processing expertise (Flink, Spark, etc.) Familiarity with distributed datastores (Druid, Mongo, Cassandra, BigQuery) Excellent written and verbal communication Open-source contributions (bonus) Responsibilities: Develop next-gen data pipelines for real-time security insights Architect scalable data pipelines for real-time processing challenges Collaborate with security and big data experts, product, and engineering teams to create innovative solutions Evaluate and contribute to open-source technologies Assist teams in architecting...

Job title - Senior Consultant-Data Engineer Career Level - D2 Leverage technology to impact patients and ultimately save lives Do you have expertise in, and passion for, information technology? Would you like to apply your expertise to impact the IT strategy in a company that follows the science and turns ideas into life changing medicines? If so, AstraZeneca might be the one for you! ABOUT ASTRAZENECA AstraZeneca is a global, science-led, patient-focused biopharmaceutical company that focuses on the discovery, development and commercialisation of prescription medicines for some of the world's most serious disease. But we're more than one of the world's leading pharmaceutical companies. At AstraZeneca we're dedicated to being a Great Place to Work. ABOUT OUR IT TEAM It's a dynamic and results-oriented environment to work in - but that's why we like it. There are countless opportunities to learn and grow, whether that's exploring new technologies in hackathons, or...

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. Our organization is in search of a seasoned Senior Data Engineer to enhance our team. In this role, you will focus on projects involving data integration and ETL for cloud environments. Your primary duties will include the design and execution of intricate data solutions, maintaining data accuracy, dependability, and accessibility. #LI-DNI #EasyApply Responsibilities Design and...

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. We are looking for a skilled Lead Data Engineer to enhance our dynamic team. In this role, you will focus on designing, developing, and maintaining data integration solutions for our clients. Your leadership will guide a team of engineers in delivering scalable, high-quality, and efficient data integration solutions. This role is perfect for an experienced data integration expert who is...

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. Our organization is seeking a skilled Senior Data Engineer to become an integral part of our team. In this role, you will focus on projects related to data integration and ETL on cloud-based platforms. You will take charge of creating and executing sophisticated data solutions, ensuring data accuracy, dependability, and accessibility. #LI-DNI #EasyApply Responsibilities Create and...

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. Our organization is currently seeking a skilled Senior Data Engineer to become part of our dynamic team. In this role, as a Senior Data Engineer, you will focus on projects involving data integration and ETL for cloud-based platforms. Your responsibilities will include crafting and executing sophisticated data solutions to ensure the data remains precise, dependable, and readily available....

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. We are seeking a skilled Senior Data Engineer to become an integral part of our team. In this role, you will focus on projects involving data integration and ETL for cloud-based platforms. Your tasks will include creating and executing sophisticated data solutions, ensuring the integrity, reliability, and accessibility of data. #LI-DNI #EasyApply Responsibilities Create and execute...

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. We are seeking a skilled Senior Data Engineer to become a part of our dynamic team. In this role as a Senior Data Engineer, you will focus on projects involving data integration and ETL processes tailored for cloud-based environments. Your main tasks will include crafting and executing sophisticated data structures, while ensuring the integrity, accuracy, and accessibility of data. #LI-DNI...

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. We are searching for a skilled Lead Data Engineer to enhance our energetic team. In the role of Lead Data Engineer, you will take charge of creating, building, and sustaining data integration solutions for our clientele. Your leadership will guide a team of engineers towards achieving high-quality, scalable, and efficient data integration solutions. This role presents a thrilling challenge for...

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential. We are looking for a skilled Lead Data Engineer to become an integral part of our vibrant team. In this role, you will take charge of designing, developing, and maintaining data integration solutions tailored to our clients' needs. You will oversee a team of engineers, ensuring the delivery of high-quality, scalable, and efficient data integration solutions. This role presents a thrilling...

Description AWS Data Engineer 1. Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming from various sources. Perform data cleansing, data validation etc 2. Hands on ETL developer who is good at python, SQL. AWS services like glue, glue crawlers, lambda, red shift, athena, s3, EC2, IAM, Monitoring and Logging mechanisms- AWS cloudwatch, setting up alerts. 3. Deployment knowledge on cloud. Integrate CI/CD pipeline to build artifacts and deploy changed to higher Environments. 4. Scheduling frame works Airflow, AWS Step functions 5. Excellent Communication skills, should be able to work collaboratively with other teams

Description AWS Data Engineer 1. Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming from various sources. Perform data cleansing, data validation etc 2. Hands on ETL developer who is good at python, SQL. AWS services like glue, glue crawlers, lambda, red shift, athena, s3, EC2, IAM, Monitoring and Logging mechanisms- AWS cloudwatch, setting up alerts. 3. Deployment knowledge on cloud. Integrate CI/CD pipeline to build artifacts and deploy changed to higher Environments. 4. Scheduling frame works Airflow, AWS Step functions 5. Excellent Communication skills, should be able to work collaboratively with other teams