Hadoop Infrastructure Engineer


Copenhagen, DK
  • Job Type: Full-Time
  • Function: IT
  • Post Date: 04/16/2021
  • Website: liveintent.com
  • Company Address: 100 Church St btwn Barclay & Park Place, New York, NY, 10007

About LiveIntent

LiveIntent is a people-based marketing technology platform changing the way over 2,000 of the world's largest brands and publishers think about email.

The LiveIntent platform transforms email from something you send and receive into the simplest, most powerful way to resolve identity and market to people in a mobile-first world.

Job Description

About the job

About the Job:

To our highly-skilled development team in Copenhagen we seek a versatile Hadoop Infrastructure Engineer to join our Data Pipeline team with the passion for algorithms as well as an understanding of software engineering. LiveIntent Copenhagen stands for the development and deployment of the company's next generation technology in close collaboration with the company's other departments in New York, Berlin and Moscow.


Our team in Copenhagen consists of  a few veterans from the start-up environment and 20 new hand-picked colleagues. Both the level of competence and focus is very high. For that reason, many exciting tasks are constantly flowing from the rest of the organization to Copenhagen, so we want to expand the team with more skilled and innovative talents. 


The Position:

As our new Hadoop InfrastructureEngineer, you will join our Data Pipeline team and be part of improving our technologies for distributed batch processing. The Data Pipeline teams dream is to provide an environment where it is extremely easy to deploy a hadoop or spark job that processes terabytes of data, such that we can improve the daily life of our colleagues that use the platform. This involves solving problems like


  • Automatic configuration of spark jobs based on previous job history
  • Automatic detection of skewness in data that is processed
  • Instance selection in AWS so we balance spot loss versus price
  • Automatic restarting of  jobs that failed due to instabilities in underlying infrastructure


Your daily work life will include but will not be limited to the following tasks:


  • Create and maintain hadoop and spark  architecture/infrastructure as IaC
  • Identify, design and implement internal process improvement:
  • Automating manual processes
  • Optimizing job run time and costs
  • Build analytics tools to provide actionable insights:
  • Ease troubleshooting
  • Job optimization
  • Reporting
  • Work with other teams inside the Engineering department to debug potential job issues, such as data skewness, load imbalance, poor performance and other difficult to debug issues. 


We believe in flexibility under responsibility, and our development process is therefore a loosely defined hybrid of agile methods and rapid prototyping.


Your Profile:

  • Experience with data analysis and/or data processing. Experience with big data is a plus
  • Approach problems analytically and solutions pragmatically. Experience with solving complex problems is a plus
  • Solid understanding of fundamental Computer Science concepts, e.g., operating systems, network fundamentals. Knowledge and/or experience with distributed systems is a plus
  • Practical experience in programming. Ability to analyze java bytecode is a plus
  • Eager to understand and find solutions in the problem areas
  • Good interpersonal skills and ability to work well with all levels of colleagues, e.g., listening and communicating clearly, sharing knowledge and learning from others
  • Experience with Hadoop/Spark is highly desired but not required
  • Experience with one or more of the following is a plus: Linux ,Cloud services, e.g. AWS, Scala/Python, Docker/Kubernetes or Ansible


You are full of good ideas, able to see opportunities and weaknesses in other people's ideas, and can come up with your own ideas in a constructive way. You are happy to implement and optimize and you are curious about new tasks.


You are in a dynamic environment where overall priorities and definitions will change over time, and therefore tasks will not always be prioritized. You can hand over tasks, but prefer to follow them to the door itself. You work independently and in the initiative.


We Offer:

You can look forward to being a key player in a challenging and dynamic developmental story where you will be part of a team with some of the industry's hottest talents. There will be rich opportunities to learn from others, and you will have the flexibility to dive into experiments on your own. We are passionate about new solutions, technologies and algorithms, but pragmatic in our choice of solutions.


We believe in freedom and flexibility under responsibility so that office hours can be adapted to your life as long as meetings are covered. If you need to work from home one day, it is completely fine, as long as you live up to what is expected. We are an international company, with frequent contact between the departments in Berlin and New York City. The position is located in our office in Copenhagen.


The Company:

LiveIntent, one of the world’s largest people-based marketing platforms, connects 2,500 publishing and advertising brands with over 250 MM verified people every month across all types of media. With the anonymized email address at the center of its industry-leading identity graph, LiveIntent provides brands with solutions that help them monetize, acquire, and retain real people, even where cookies don’t work. LiveIntent enriches a brands’ data, making it possible for them to deepen their understanding of their audiences, and more effectively market to people wherever they are present and paying attention. LiveIntent is home to over 180 people worldwide with offices in New York, Chicago, Berlin, and Copenhagen.


Instilling diversity, a sense of belonging, and empowerment into LiveIntent’s culture, core values, and vision is core to our success. We gladly support several employee resource groups such as BID with Intent (belonging, inclusion, & diversity), Lead with Intent, Parents with Intent, and Women with Intent, which provide individuals at the organization the chance to voice their opinions and feel heard in a space that is safe and welcoming.


LiveIntent is an Equal Opportunity Employer that does not discriminate on the basis of actual or perceived race, creed, color, religion, alienage or national origin, ancestry, citizenship status, age, disability or handicap, sex, marital status, veteran status, sexual orientation, genetic information, arrest record, or any other characteristic protected by applicable federal, state or local laws. Our management team is dedicated to this policy with respect to recruitment, hiring, placement, promotion, transfer, training, compensation, benefits, employee activities and general treatment during employment.

Related Jobs

Product Manager, Identity Solutions

LiveIntent - New York City, NY, US

Events Marketing Manager

LiveIntent - New York City, NY, US

Sales Planner Intern

LiveIntent - New York City, NY, US

Talent Acquisition Specialist

LiveIntent - New York City, NY, US

Technical Account Manager

LiveIntent - New York City, NY, US
Disclaimer: Local Candidates Only
This company does NOT accept candidates from outside recruiting firms. Agency contacts are not welcome.