Data Engineer

Job Title:

Job Description

Key Responsibilities:

• Data Pipeline Development:
  Design, build, and manage scalable, reliable, and efficient ETL/ELT data pipelines to ingest, process, and store large datasets from various data sources.
• Data Integration:
  Integrate data from different sources such as APIs, relational databases, NoSQL databases, flat files, and streaming data into centralized data lakes or data warehouses.
• Database Management & Optimization:
  Implement and maintain data storage solutions such as relational databases (e.g., PostgreSQL, MySQL), NoSQL databases (e.g., MongoDB, Cassandra), and cloud-based data warehouses (e.g., Amazon Redshift, Google BigQuery).
• Data Quality & Validation:
  Ensure data quality and integrity through the design and implementation of data validation, cleansing, and enrichment processes. Identify and address data discrepancies and inconsistencies.
• Collaboration with Teams:
  Collaborate with data scientists, analysts, and software engineers to understand data requirements, and deliver data solutions that meet analytical and operational needs.
• Performance Tuning:
  Optimize data pipelines and data storage systems for maximum efficiency, scalability, and performance. Proactively monitor system performance and troubleshoot issues.
• Data Governance & Security:
  Work closely with the data governance and security teams to ensure that data solutions comply with organizational standards, privacy laws (e.g., GDPR, CCPA), and security policies.
• Cloud Data Engineering:
  Design and manage data workflows in cloud environments such as AWS, Azure, or Google Cloud, utilizing cloud-native services for data processing and storage.
• Automation & Scheduling:
  Automate data workflows and implement scheduling tools (e.g., Airflow, Cron) to ensure timely data delivery for reports, dashboards, and analytics.
• Documentation:
  Create and maintain technical documentation for data pipelines, data models, and data management processes to ensure knowledge sharing and reproducibility.

Requirements:

• Education:
  Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
• Experience:
  • Minimum of 3-5 years of experience in data engineering, ETL development, or database management.
  • Experience with cloud platforms and cloud-native data processing tools.
• Skills & Expertise:
  • Proficiency with SQL for querying and manipulating data.
  • Experience with data integration tools (e.g., Apache Airflow, Talend, Informatica) and ETL/ELT processes.
  • Strong programming skills in Python, Java, or Scala for data processing.
  • Knowledge of big data technologies (e.g., Hadoop, Spark, Kafka).
  • Experience with data warehousing solutions (e.g., Snowflake, Amazon

#ConcentrixCatalyst

Location:

Language Requirements:

Time Type:

If you are a California resident, by submitting your information, you acknowledge that you have read and have access to the Job Applicant Privacy Notice for California Residents