Big Data Developer


Job ID 18-00032

Industry Computer/IT

Job Type Permanent

Location Red Bank, NJ

Description

Company is looking for strong application developers with Big Data experience who will help design, build and maintain our automated data workflows/pipelines and company's industry-leading Identity Graph. You will be part of the team automating the analysis, processing and testing of their big data. You will help develop solutions to ingest, store and distribute their data onto a wide array of online platforms and tools. You'll be instrumental in the development of quality processes and making sure high quality, reliable information is delivered to the rest of the company team. You'll need keen analytical skills and a willingness to work in a fast-paced collaborative environment.
RESPONSIBILITIES
- Design, build and maintain Big Data workflows/pipelines to process billions of records into and out of our data lake and Identity Graph
- Fine tune application performance
- Troubleshoot and resolve data processing issues
- Engage in application design and data modeling discussions
- Participate in developing and enforcing data security policies
- Participate in capacity monitoring and planning
- Build, maintain and execute unit test cases with high code coverage



Qualifications

QUALIFICATIONS
- BS/BA degree in Computer Science, Information Systems or related field
- 3 years programming in Scala, Java, Python or GO
- 2 years developing on Hadoop/Spark or AWS EMR
- 2 years developing on an RDBMS such as Microsoft SQL Server, PostgreSQL, MySQL or Oracle
- Experience with large data sets – regularly transforming and querying tables or sets of greater than 20 million records
- Exposure to data hygiene routines and models
- Experience in database design, development and data modeling
- Ability to identify problems, and effectively communicate solutions to team
- Ability to work in a dynamic multi-team environment as well as independently

ADDED VALUABLE SKILLS
- Hadoop: HDFS, MapReduce, Hive, Pig
- Exposure to AWS services such as EMR, Glue, Lambda, Step Functions, Aurora, DynamoDB
- Data architecture
- Database security
- ETL using SQL or a scripting/programming language
- Experience with fuzzy-logic matching and tools
- NoSQL: HBase, AWS DynamoDB, Cassandra, MongoDB
- Familiarity with Linux
- DevOps Environment Experience

Company is a data onboarding company focused on individual and deterministic matching. With a data centric approach, our company Identity Graph allows for the most accurate and transparent view of customers. At company we thrive on technical growth and challenges, value entrepreneurship, and the belief in being a team player.