Lead Data Engineer Job at WorkHQ, Los Angeles, CA

Wm8rR05xdXlCWU9Rb21UZ25QM0VsZkVLaWc9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Remote work, Shift work,

Similar Jobs

Sanford Health

LPN - PRN - LTC Job at Sanford Health

 ...Department Details Join our team as a PRN LPN! - $28+ per hour, depending on...  ...in establishing and maintaining effective working relationships with resident, health care...  ...time off package to maintain a healthy home-work balance. For more information about Total... 

CRH

Ground/Maintenance Person Job at CRH

 ...aggregate distribution network covers Florida, Southern Mississippi, South Carolina, and Alabama. Position Overview The Plant Maintenance position will be responsible for daily maintenance and repair of the plant as well as other miscellaneous work that needs to be... 

Domino's Franchise

Part Time Delivery Driver Job at Domino's Franchise

 ...professional company, striving to be number one in carryout, delivery, customer service, sales... everything! While the entire...  ...doorsteps. For this role specifically, we are looking for part-time drivers who mostly work during our busiest hours, around dinner time... 

Seva Hospitality

Marriott Hotel General Manager Job at Seva Hospitality

 ...company. Job Description At SpringHill Suites by Marriott , were committed to making travel an elevated experience. We...  ..., Vision, and a rewarding bonus structure. Upon opening the hotel, as a General Manager you will: Oversee all aspects of operations... 

Palm Bay Ford

Accounts Payable / Accounts Receivable Clerk Job at Palm Bay Ford

 ...and is able to multi-task.Posting daily,weekly and monthly invoices and keeping them paid in a timely manner.Collecting payment and posting to accounts receivable accounts. Sending out statements.Full job description with interview.Pay is based on knowledge.