# Senior Data Engineer – 8+ Years Experience - Hudson Manpower New Jersey, United States · Added today Senior [Data](https://gurify.com/jobs/data) [Apply on the original posting → (opens in a new tab)](https://hudsonmanpower.recruitee.com/o/senior-data-engineer-8-years-experience) ## Job description ### Job Summary We are looking for an experienced Senior Data Engineer with 8+ years of hands-on experience in designing, developing, and maintaining scalable data platforms, data pipelines, and analytics solutions. The ideal candidate will have strong expertise in Python/SQL, ETL/ELT, cloud data platforms, data warehousing, distributed data processing, orchestration, and data architecture. The candidate will work closely with Data Scientists, BI Developers, Software Engineers, Product Managers, and business stakeholders to build reliable, secure, high-performance data solutions that support business-critical analytics and AI/ML initiatives. ### Key Responsibilities - Design, develop, and maintain scalable and reliable batch and real-time data pipelines. - Build robust ETL/ELT workflows to ingest, transform, validate, and distribute data from multiple sources. - Develop highly optimized and complex SQL queries, stored procedures, and data transformations. - Design and implement data warehouses, data lakes, lakehouses, and dimensional data models. - Work with large datasets using distributed processing technologies such as Apache Spark/PySpark. - Develop data pipelines using orchestration tools such as Apache Airflow, Azure Data Factory, AWS Glue, or similar platforms. - Implement data solutions on major cloud platforms such as AWS, Azure, or GCP. - Design and optimize cloud data platforms and services such as Amazon Redshift, Snowflake, Databricks, Azure Synapse, BigQuery, or equivalent technologies. - Implement data quality, data validation, reconciliation, monitoring, and observability frameworks. - Develop solutions for incremental data processing, CDC, slowly changing dimensions, partitioning, and performance optimization. - Build and maintain real-time/streaming data pipelines using technologies such as Kafka, Kinesis, or equivalent tools. - Implement appropriate data security, governance, access control, encryption, and compliance practices. - Collaborate with data architects to translate business requirements into scalable technical solutions. - Perform performance tuning of data pipelines, databases, Spark jobs, and cloud data workloads. - Establish and maintain CI/CD practices for data engineering workflows. - Write unit, integration, and data-quality tests to ensure reliability of production pipelines. - Troubleshoot production data issues and participate in incident resolution and root-cause analysis. - Conduct code reviews and promote engineering best practices across the data engineering team. - Mentor junior and mid-level data engineers and provide technical leadership. - Document data architecture, pipeline designs, data models, operational procedures, and technical decisions. - Stay current with emerging technologies in cloud, big data, data engineering, data platforms, and AI/ML. ### Required Technical Skills ### Programming & Database - Strong proficiency in Python. - Advanced SQL skills. - Experience with relational databases such as PostgreSQL, MySQL, SQL Server, or Oracle. - Experience with NoSQL databases such as MongoDB, DynamoDB, Cassandra, or similar is advantageous. - Strong understanding of database design, indexing, query optimization, and transaction management. ### Big Data & Distributed Processing - Strong experience with Apache Spark / PySpark. - Experience with Hadoop ecosystem technologies is desirable. - Understanding of distributed computing, partitioning, parallel processing, and performance optimization. ### Data Engineering & ETL - Extensive experience building ETL/ELT pipelines. - Experience with tools such as: - Apache Airflow - Azure Data Factory - AWS Glue - dbt - Informatica - Talend - SSIS - Experience handling structured, semi-structured, and unstructured data. ### Cloud Technologies ### Strong experience with at least one major cloud platform: ### AWS - S3 - Glue - EMR - Redshift - Lambda - Kinesis - Athena - IAM ### Azure - Azure Data Factory - Azure Data Lake Storage - Azure Databricks - Azure Synapse Analytics - Azure Functions - Event Hubs - Key Vault ### GCP - BigQuery - Cloud Storage - Dataflow - Dataproc - Pub/Sub - Cloud Composer ### Data Warehousing & Lakehouse - Strong understanding of data warehouse architecture. - Experience with Snowflake, Databricks, Redshift, Synapse, BigQuery, or equivalent. - Expertise in: - Star and Snowflake schemas - Fact and dimension tables - Slowly Changing Dimensions (SCD) - Data marts - Data lakes - Lakehouse architecture - Partitioning and clustering - Data modeling ### Streaming & Real-Time Data - Experience with Apache Kafka or equivalent streaming platforms. - Understanding of producers, consumers, topics, partitions, offsets, consumer groups, and schema management. - Experience developing real-time or near-real-time data processing pipelines. ### DevOps & Engineering Practices - Experience with Git/GitHub/GitLab/Bitbucket. - Experience with CI/CD pipelines. - Knowledge of Docker and Kubernetes is desirable. - Experience with Infrastructure as Code tools such as Terraform is advantageous. - Familiarity with automated testing, deployment, monitoring, and observability. ### Data Governance & Security - Understanding of data governance, metadata management, lineage, data cataloging, and data quality. - Experience implementing role-based access control and secure data access. - Knowledge of privacy and compliance requirements such as GDPR, CCPA, HIPAA, or equivalent regulations, depending on business requirements. **Live in the employer’s hiring system.** Read from the company's own applicant tracking system, not reposted from a job board — so it's a real, open requisition rather than an ad that outlived the role. We remove it as soon as it disappears at source. ## More jobs like this - AR [Data Engineer (.NET / Kafka / ETL)](https://gurify.com/job/data-engineer-net-kafka-etl-at-abra-rnd-30e2d4b42f08) Abra RND · Center, Center District, IL · yesterday - TL [Staff Data Engineer - AI Platform](https://gurify.com/job/staff-data-engineer-ai-platform-at-trm-labs-d14041d83be8) TRM Labs · United States · 5 days ago - TL [Senior Data Engineer - AI Platform](https://gurify.com/job/senior-data-engineer-ai-platform-at-trm-labs-fb76e158eae8) TRM Labs · United States · 5 days ago - MI [AI/Data Engineer – Snowflake, DBT, Python Focus](https://gurify.com/job/ai-data-engineer-snowflake-dbt-python-focus-at-matrix-ifs-d26501ebed08) Matrix Ifs · Hyderabad, IN · 2 weeks ago - TO [Lead Data Engineer](https://gurify.com/job/lead-data-engineer-at-todaytixgroup-3b2926e4836c) Todaytixgroup · New York, NY · 3 weeks ago - 8T [Principal Data Engineer (Remote)](https://gurify.com/job/principal-data-engineer-remote-at-8thlightrebuild-aca2ecae71d6) 8Thlightrebuild · United States · 4 days ago ```json {"@context":"https://schema.org/","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Jobs","item":"https://gurify.com/jobs"},{"@type":"ListItem","position":2,"name":"United States","item":"https://gurify.com/jobs/united-states"},{"@type":"ListItem","position":3,"name":"Senior Data Engineer \u2013 8\u002B Years Experience - Hudson Manpower","item":"https://gurify.com/job/senior-data-engineer-8-years-experience-hudson-manpower-1c0c2536db60"}]} ```