AWS Data Engineer Job at IVID TEK INC, Philadelphia, PA

WTRtQk5LbThCNHVkb21QbG4vM0VtUHdQ
  • IVID TEK INC
  • Philadelphia, PA

Job Description

Senior AWS Data Engineer

Contract (100% Remote)

Job Description:

We are looking for AWS Big Data Platform Engineers who are passionate about resolving the challenges of building robust and scalable data systems aligned to enterprise data strategy.

As Senior Data Platform Engineer you will be part of a team that leads the design and implementation of data platform to support the strategic data needs & support enterprise reporting needs including datamarts, reporting and business intelligence. You will build data capabilities and support a wide range of data flows and use cases to advance the data-driven investment decisioning and deliver transformational programs.

  Required skills:

  • Experience with AWS Data & Analytics, Cloud data platforms, data warehouse, data lakes and analytics products
  • Experience with batch job scheduling, leveraging Apache Airflow, AWS EMR (Multi Tenant ), PySpark, YARN, EKS, S3, HDFS, AWS Athena, AWS Glue, Lakeformation, AWS Step Functions, DMS
  • Experience building data tools using spark , performance tuning , AQE, Spark Listener Events,
  • Experience in data architecture and familiarity with ETL tools, data warehousing & big data architecture.
  • Knowledge of cloud-native approaches to build fast, flexible, and cost-effective data applications on any data lake, open-source column oriented DBMS and/or big data OLAP engines for analytical processing (Apache Pinot, Apache Hudi, Iceberg etc.)

Good to Have:

  • Experience with DataOps and various continuous delivery practices to deploy, support and operate data pipelines ( terraform , terragrunt)

JD- Required skills – Cloud Devops, python, Big Data Admin exp on AWS , AWS lake formation ( lake house vs Data fabric ), Data Product concepts, spark jobs auditing, monitoring and Instrumentation, spark perf tuning, exp in creating service and tools to manage multi tenant env, 

Tech stack ( Must )– terraform, knowledge of spark internals ( spark 3+ features ) – architecture, events, system metrices , AQE, job level resource management, Hudi, Parquet , IAM, AWS Glue, HIVE

Good to have - Iceberg, airflow, terragrunt, dynamo DB

Job Tags

Contract work, Remote job, Flexible hours,

Similar Jobs

Chainalysis Inc.

Senior UX Researcher Job at Chainalysis Inc.

 ...Our UX Research team plays a major role within a diverse, fun, and geographically distributed design team. We have an oversize impact given our team size by driving select research studies and playing a critical role in multiple initiatives related to UXR, customer feedback... 

The Gardens Health Care Center

Quality Assurance Nurse -RN Job at The Gardens Health Care Center

 ...Assists and supports the Director of Nursing (DON) in planning, organizing, developing...  ...DON, to ensure that the highest degree of quality care is always maintained. Qualifications...  ...guidelines, making recommendations for improvements Completes tasks on time, reporting up... 

Get It - Healthcare

Behavioral Health Coach - Remote Job at Get It - Healthcare

 ...support they need to navigate life's challenges, making mental health care more effective and widely available. As part of our collaborative...  ...About the Role:We are seeking a passionate Behavioral Health Coach to support children, teens, and their families through evidence-... 

Safety Management Systems

Medical Sales Executive Job at Safety Management Systems

 ...identifying and pursuing new business opportunities within our Medical Services Division. Our Medical Services Division provides on-site...  ...and training for companies with employees in remote or offshore locations. The primary focus of the Medical Sales Executive is... 

Mission Community Hospital

New Grad RN Job at Mission Community Hospital

Job Summary:Mission Community Hospital is seeking a highly motivated and enthusiastic individual to fill the role of New Grad RN for our MST unit in Panorama City, California. This is a full-time, individual contributor position in the healthcare/medical field. The successful...