Salary / Rate Based upon Experience Employment Type: Full Time / Contract
Azure Big Data Architect
Location: Remote until Covid
Apply Now

Job Description

The Azure Big Data Architect will be responsible for Enterprise data warehouse design, development, implementation, migration, maintenance and operation activities. The position is a key resource or various Enterprise data warehouse projects and building critical data marts, and data ingestion to Big Data platforms'

  • Participate in Team activities, Design discussions, Stand up meetings and planning Review with team.
  • Perform data analysis, data profiling, data quality and data ingestion in various layers using big data queries, PySpark programs and UNIX shell scripts.
  • Follow organizational coding standard documents, create mappings, sessions and workflows outlined within mapping specification documents.
  • Perform Gap and impact analysis of ETL and IOP jobs for new requirement and enhancements.
  • Create mockup data, perform unit testing and capture the result sets against the jobs developed in lower environment.
  • Updating the production support run book.
  • Create and update design documents, provide detail description about workflows after every production release.
  • Continuously monitor the production data loads, fix the issues, update the tracker document with the issues, Identify the performance issues.
  • Performance tuning long running ETL/ELT jobs by creating partitions, enabling full load and other standard approaches.
  • Perform Quality assurance check, Reconciliation post data loads and communicate to vendor for receiving fixed data.
  • Participate in ETL/ELT code review and design re-usable frameworks.
  • Create re-usable framework for audit control to capture reconciliation, mapping parameters and variables, serves as single point of reference for workflows.
  • Participate in meetings to upgrade the functional and technical expertise.
Technical Skills and Experience
  • Expertise implementing complex ETL/ELT logic.
  • Develop and enforce strong reconciliation process.
  • Accountable for ETL/ELT design documentation.
  • Good knowledge of Big Data, Hadoop, Hive, data security and dimensional model design.
  • Basic knowledge of UNIX/LINUX shell scripting.
  • Utilize ETL/ELT standards and practices towards establishing and following centralized metadata repository.
  • Effective communication, presentation, & organizational skills.
  • 6+ years of experience with Big Data, Hadoop on Data Warehousing or Data Integration projects,and familiarity with PHI and PII data.
  • Analysis, Design, development, support and Enhancements of ETL/ELT in data warehouse environment with Azure Technologies.
  • Strong development experience in creating powershell scripts, PySpark programs, HDFS commands and HDFS file formats
  • Writing Hadoop/Hive scripts for gathering stats on table post data loads.
  • Writing complex SQL queries and performed tuning based on the Hadoop/Hive explain plan results.
  • Proven ability to write high quality code.
  • 6+ years of experience building data sets
  • Familiar with Project Management methodologies (Agile)
  • Ability to establish priorities & follow through on projects, paying close attention to detail with minimal supervision.
Certifications Required
  • Microsoft Azure DP-200: Implementing an Azure Data Solution
  • Microsoft AzureDP-201: Designing an Azure Data Solution
  • Relevant Bachelor of Science or Bachelor of Arts degree or similar experience