Data Crawling Lead – REF 237 – 01

Full Time, Permanent
Remote, Romania

 

Purpose of the Role

Your role will be to build out a data acquisition team as part of a larger product team. You will bring your expertise in data crawling, analytical mind, keen eye for detail and leadership abilities into building a team that will acquire significant amounts of data to feed groundbreaking products redefining enterprise software. This is a startup environment and we expect you to be able to work hands-on as needed and mentor and build a functional team. 

Duties and Responsibilities

  • Work closely with the clients to identify and acquire data, crawl information from multiple online data sources
  • Define the strategy and tools to be used for web crawlers, web scrapers and other automation tools, to help extract the content
  • Lead a team of web crawling engineers and be responsible for their mentorship
  • Drive change by staying up to date with the data science trends and technologies

Required Experience & Knowledge

  • Previous experience leading a team and defining workload
  • Previous hands-on experience working in a web crawling role is a must
  • Senior expertise in Java and experience working with high volume web crawling/scraping
  • Experience with Java related frameworks such as Spring, Hibernate
  • Experience working with open source tools such as ApacheNutch, StormCrawler etc
  • Experience working with relational databases such as MySQL, PostgreSQL, Oracle and noSQL databases
  • Experience with queuing systems like RabbitMQ / AMQP, Kafka, JMS, Amazon Kinesis
  • Experience with JSON, REST API, HTML, XML

Advantage

  • Familiarity with supplying data to support machine learning efforts a plus
  • Familiarity with Elasticsearch, AWS and the process of scaling and approaches 

Skills and Attributes

  • Fluency in English (both written and spoken) is a must
  • Client-oriented approach, high integrity and commitment
  • Excellent communication and teamwork skills
  • Detail-oriented and focused on quality
  • Excellent organizational skills

Required Education & Qualifications

  • BA or MSc Degree, or relevant experience

Apply Online

A valid email address is required.