Data Engineer Level
Payrate: $59.00 - $65.90/hr. Summary: The team is seeking a Data Engineer experienced in implementing modern data solutions in Azure, with strong hands-on skills in Databricks, Spark, Python, and cloud-based DataOps practices. The Data Engineer will analyze, design, and develop data products, pipelines, and information architecture deliverables, focusing on data as an enterprise asset. This role also supports cloud infrastructure automation and CI/CD using Terraform, GitHub, and GitHub Actions to deliver scalable, reliable, and secure data solutions. Responsibilities: Analyze, design, and develop enterprise data solutions with a focus on Azure, Databricks, Spark, Python, and SQL. Develop, optimize, and maintain Spark/PySpark data pipelines, including managing performance issues such as data skew, partitioning, caching, and shuffle optimization. Build and support Delta Lake tables and data models for analytical and operational use cases. Apply reusable design patterns, data standards, and architecture guidelines across the enterprise, including collaboration with 84.51° when needed. Use Terraform to provision and manage cloud and Databricks resources, supporting Infrastructure as Code (IaC) practices. Implement and maintain CI/CD workflows using GitHub and GitHub Actions for source control, testing, and pipeline deployment. Manage Git-based workflows for Databricks notebooks, jobs, and data engineering artifacts. Troubleshoot failures and improve reliability across Databricks jobs, clusters, and data pipelines. Apply cloud computing skills to deploy fixes, upgrades, and enhancements in Azure environments. Work closely with engineering teams to enhance tools, systems, development processes, and data security. Participate in the development and communication of data strategy, standards, and roadmaps. Draft architectural diagrams, interface specifications, and other design documents. Promote the reuse of data assets and contribute to enterprise data catalog practices. Deliver timely and effective support and communication to stakeholders and end users. Mentor team members on data engineering principles, best practices, and emerging technologies. Qualifications: 5 years of experience as a Data Engineer. Hands-on experience with Azure Databricks, Spark, and Python. Experience with Delta Live Tables (DLT) or Databricks SQL. Strong SQL and database background. Experience with Azure Functions, messaging services, or orchestration tools. Familiarity with data governance, lineage, or cataloging tools (e.g., Purview, Unity Catalog). Experience monitoring and optimizing Databricks clusters or workflows. Experience working with Azure cloud data services and understanding how they integrate with Databricks and enterprise data platforms. Experience with Terraform for cloud infrastructure provisioning. Experience with GitHub and GitHub Actions for version control and CI/CD automation. Strong understanding of distributed computing concepts (partitions, joins, shuffles, cluster behavior). Familiarity with SDLC and modern engineering practices. Ability to balance multiple priorities, work independently, and stay organized. Desired Skills: Experience in mentoring team members on data engineering principles, best practices, and emerging technologies. Pay Transparency: The typical base pay for this role across the U.S. is: $59.00 - $65.90 /hour. Non-exempt positions are eligible for overtime at a rate of 1.5 times the base hourly rate for all hours worked in excess of 40 in a work week, or as required by state or local law. Final offer amounts, within the base pay set forth above, are determined by factors including your relevant skills, education and experience. Full-time employees are eligible to select from different benefits packages. Packages may include medical, dental, and vision benefits, health savings accounts with qualified medical plan enrollment, 10 paid days off, 3 days paid bereavement leave, 401(k) plan participation with employer match, life and disability insurance, commuter benefits, dependent care flexible spending account, accident insurance, critical illness insurance, hospital indemnity insurance, accommodations and reimbursement for work travel, and discretionary performance or recognition bonus. Sick leave and mobile phone reimbursement provided based on state or local law. Consent to Communication and Use of AI Technology: By submitting your application for this position and providing your email address(es) and/or phone number(s), you consent to receive text (SMS), email, and/or voice communication whether automated (including auto telephone dialing systems or automatic text messaging systems), pre-recorded, AI-assisted, or individually initiated from Aditi Consulting, our agents, representatives, or affiliates at the phone number and/or email address you have provided. These communications may include information about potential opportunities and information. Message and data rates may apply. Message frequency may vary. You represent and warrant that the email address(es) and/or telephone number(s) you provided to us belong to you and that you are permitted to receive calls, text (SMS) messages, and/or emails at these contacts. You also acknowledge and agree to Aditi Consulting LLC’s use of AI technology during the sourcing process, including calls from an AI Voice Recruiter. AI is used solely to gather data and does not replace human-based decision-making in employment decisions. Calls may be recorded. Consent is not a condition of purchasing any property, goods, or services. You may revoke your consent at any time by replying “STOP” to messages or by contacting [email protected] . For information about our collection, use, and disclosure of applicant's personal information as well as applicants' rights over their personal information, please see our Privacy Policy . AditiConsulting 26 - 01737
