Lead Data Engineer | Infrastructure


London, GB / England, GB
  • Job Type: Full-Time
  • Function: Data Science
  • Post Date: 04/05/2021
  • Website: lyst.com
  • Company Address: Floor 7, The Minster Building, 21 Mincing Lane, London, UK, EC3R 7AG

About Lyst

Lyst is a fashion search platform, used by 104 million shoppers each year to find the items they love and decide what to buy. More than 6 million products from over 12,000 brands and stores including Burberry, Fendi, Gucci, Prada and Saint Laurent can be accessed through our single search bar and app, offering shoppers convenience and unparalleled choice.

Job Description

Lyst is the definitive fashion shopping website and app, used by over 150M shoppers a year to discover and buy fashion. More than 8.5M products from over 12,000 brands and stores can be accessed through our platform, offering shoppers globally convenience and unparalleled choice.

We work in small, self-managing, autonomous teams with end-to-end responsibility for a specific customer-focused project. This structure brings together Lysters from all the disciplines that are needed to deliver the squad's goals. We reward these squads for the impact they make and value the innovative approaches that autonomy and alignment can bring. We hire great people and get out of their way.


The Role

We are looking for a Lead Data Engineer to head up our Data Engineering team in the Infrastructure tribe. You’ll be responsible for managing and growing the analytics pipeline that's used by every team in the company to join together all of the data that flows through Lyst.

You’ll help to set the structure for data events, manage the infrastructure and software that teams depend on to shuffle events into the data lake, liaise with engineers throughout the company on their needs for the pipeline, help maintain the tooling and developer experience of what you build, and manage the Data Engineering team.

You’ll also be responsible for helping shape and enforce our data governance policies that ensure events are built against and adhering to a defined schema, develop data models that heuristically analyze and warn against issues with data quality, and help to coordinate the rectification of any data problems through the development of monitoring and alerting that allows the engineering teams to own and self-serve their event pipelines.


What you will be working on?

  • Our analytics pipeline: building and maintaining the necessary infrastructure to move and process event data from where it’s created or ingested and into our data lake
  • Working in Python on top of the AWS technology stack: we utilise Kinesis, Firehose, Lambda, S3 and Glue to deliver events into Snowflake for reporting on by our BI teams
  • Using infrastructure-as-code in Terraform to create and maintain scalable infrastructure to support our data lake pipelines
  • Driving our data engineering chapter to liaise with other data engineers throughout the company to gather feedback and ideas for development of the analytics pipeline
  • Cultivate a great developer experience that engineers can use both locally and in production to build and maintain events in our pipeline
  • Create processes to help engineers monitor and improve data quality and integrity
  • Perform analysis to troubleshoot and fix data and infrastructure related issues and provide recommendations for improving the pipeline to help avoid problems
  • Investigate the modelling of data events to proactively identify data quality issues and find ways to autonomously introduce this feedback into production workflows


  • Interest in data pipelines, analytics and event handling, and building the infrastructure necessary to support this
  • Experience supporting and mentoring other engineers and working directly with them to achieve career growth
  • Able to work comfortably with the Python code and AWS infrastructure that powers our pipeline
  • Experience of technical problem solving and troubleshooting and incident management
  • Able to triage and manage the project workload for the team in a structured but flexible kanban-style process
  • The ability to work in a supportive but autonomous environment: you’ll be expected to take stakeholder direction to develop well thought-out solutions but with the ability to have your own input on direction


  • You get 29 days' time off throughout the year to take a well earned rest, in addition to the 8 public bank holidays.
  • Private Healthcare by Vitality. Your health is important to us which is why we offer all employees a comprehensive healthcare scheme from the day that you start.
  • Conferences and events. We’re big on learning, so all Lysters are allocated an individual training & development budget
  • Discounted eye tests and glasses.
  • Team meet-ups, social events, sports and exercise events.
  • Cycle-to-work scheme
  • Transport season ticket loans

Diversity and inclusion is an integral part of our culture. We recognise and celebrate the value and impact diversity brings to our company and are committed to ensuring this is a consistent focus, for which we are held to account. We are committed to treating all applicants fairly and equally, and encourage candidates from all backgrounds to apply for this role.

Related Jobs

App Operations Manager

Lyst - London, GB

Partner Growth Coordinator

Lyst - London, GB

Android Engineer - Contract Opportunity

Lyst - London, GBEngland, GB

Backend Engineer - Contract Opportunity

Lyst - London, GBEngland, GB

iOS Engineer - Contract Opportunity

Lyst - London, GBEngland, GB
Disclaimer: Local Candidates Only
This company does NOT accept candidates from outside recruiting firms. Agency contacts are not welcome.