Senior Applied AI Engineer - Health Tech AI Startup (United States)
AgentGraph is running this confidential search on behalf of a client: a healthcare technology company modernizing how clinical and administrative documents move between providers, payers, and patients. We share the client's identity and the full team context with candidates early in the process. Why this role exists Healthcare still runs on documents. Referrals, prior authorizations, clinical summaries, lab results, intake forms, and claims move as faxes, scans, and PDFs — unstructured, inconsistently formatted, and processed by people doing work that software should have absorbed a decade ago. The cost shows up as staff hours, delayed care, and errors that surface later in the chart or the bill. This role owns the AI systems that change that. You will design, build, and operate the LLM-powered services that read these documents, extract what matters, route them where they need to go, and drive the multi-step workflows around them — with the accuracy and auditability healthcare demands. It is a rare combination: genuinely hard applied AI problems, on a document corpus that punishes naive approaches, where getting it right removes real friction from how care gets delivered and paid for. Role purpose The Senior Applied AI Engineer designs, builds, and operates the AI-driven services that provide core functionality to the platform. You will shape the technical architecture of the intelligent processing and machine learning systems, with a focus on LLM-powered document understanding, structured extraction, retrieval pipelines, and agentic workflows. You will operate with genuine autonomy, treating technical decisions as product decisions, and taking ownership from ambiguous problem statements to measured production outcomes. You will collaborate closely with Product Management, System Architects, DevOps, and Customer Success, serving as a technical lead who mentors less experienced engineers and engages directly with customers on complex solution design or escalations. Team: Engineering · Reports to: VP of Engineering · Direct reports: None / individual contributor What you will do AI & LLM systems engineering Design, develop, and maintain the LLM-powered services and pipelines at the core of the platform — document understanding, structured data extraction, classification, and summarization across highly variable real-world clinical and administrative documents Build agentic and multi-step workflows where they demonstrably outperform simpler approaches — routing, validation, exception handling, and human-in-the-loop review — with guardrails and fallbacks appropriate to a clinical setting Select the right approach for each problem — hosted frontier models, fine-tuned open-weight models, or classical ML — based on quality, latency, and cost, and implement fine-tuning where it demonstrably beats prompting and retrieval on those measures Create and maintain the high-quality datasets that model development, fine-tuning, retrieval, and evaluation depend on, including labeling strategy for documents where ground truth is genuinely ambiguous Partner with Product Management and Quality Assurance so AI solutions meet real customer needs and performance requirements, following the company's software development life cycle and its reliability, scalability, and reproducibility standards Evaluation, observability & code quality Build and maintain evaluation suites for LLM-powered features — curated golden datasets, automated scoring calibrated against human review, and regression detection — so that model, prompt, and pipeline changes ship with evidence rather than intuition Establish observability for production AI services, including request tracing, token and cost accounting, and monitoring for quality drift and failure modes such as malformed or unfounded model output. In healthcare, a confidently wrong extraction is the failure mode that matters most, and the systems you build should surface it rather than pass it downstream Lead code reviews with particular attention to ML- and LLM-specific concerns such as data processing, prompt and model versioning, experiment tracking, evaluation coverage, and production readiness Problem solving & optimization Provide problem-solving support for ML and AI systems, including performance optimization, debugging model and retrieval behavior, and improving pipeline efficiency Optimize inference for production environments — latency, throughput, token and compute cost, batching, caching, model routing, and model right-sizing — treating inference cost as a product constraint, not an afterthought Optimize ML pipelines for scalability, latency, and resource utilization at real production document volumes Customer engagement & escalation support Serve as a technical point of contact for escalated customer issues involving ML and AI services, partnering with Customer Success to own each issue end to end — reproduce, root-cause, fix, and close the loop with affected stakeholders Participate directly in customer-facing engagements when required — solution design sessions, technical discussions, and integration support — representing the team's capabilities credibly to technical and non-technical audiences, and translating customer needs into engineering direction Feed lessons from customer engagements back into product, dataset, and system improvements Technical thought leadership & mentoring Drive complex projects from ambiguous problem statements through design, implementation, and measured rollout, actively identifying and pursuing high-impact opportunities that align with company strategy Influence the architecture of ML/LLM systems and mentor engineers by sharing best practices, guiding implementation efforts, and providing constructive feedback that raises the team's overall technical standard What we look for Substantial production experience building LLM-powered systems that real users depend on — not prototypes, demos, or notebooks that never met a difficult document Depth in document understanding and structured extraction, and a realistic view of what OCR, layout analysis, and scanned or faxed source material actually do to a pipeline An evidence-driven practice: you build the evaluation before you tune, and you can show how a change moved a measured outcome Judgment about approach — knowing when a fine-tune, a retrieval pipeline, or plain classical ML beats reaching for the largest available model Comfort owning production systems, including debugging under pressure and treating cost and latency as things you are accountable for The ability to handle regulated data responsibly. Experience working under HIPAA with PHI is a strong plus; a willingness to work carefully within those constraints is a requirement Enough customer-facing ability to be trusted in a solution design session with a customer's technical and clinical stakeholders Key relationships Internal: Customer Success, DevOps, Product Management, Quality Assurance, System Architects External: Customer technical and operational stakeholders — engaged as needed for escalations, solution design, and technical integration discussions, typically in partnership with Customer Success and implementation teams About the engineering department The Engineering department designs, develops, and maintains the company's software products, covering development, infrastructure, operations, quality assurance, and product management. Its objective is to build and sustain products that satisfy customer needs and align with the long-term business strategy, to research and integrate new technologies that improve internal products and operations, and to ensure everything it ships meets the company's IT, information security, and compliance requirements. Location Remote-first — open to candidates across the United States. Customer-facing scope This role has the ability to be up to 25% customer-facing. The work is primarily engineering, but the successful candidate should be comfortable stepping into customer-facing solution design, technical discussions, and escalation support as needed. Compensation & benefits The company offers a competitive base salary, equity, and annual bonus opportunity, evaluated as a total compensation package based on the candidate's experience, qualifications, and overall fit. Comprehensive U.S. benefits, including medical, dental, and vision coverage An incredible 401(k) match program to help you invest in your future Company HSA contributions to support your healthcare savings A monthly health and wellness reimbursement for the things that help you feel your best Flexible, unlimited PTO and a remote-first environment that supports life outside of work Competitive paid parental leave, mental health resources, and Employee Assistance Program support Annual bonus opportunities based on role and company performance Equity opportunities that allow you to share in the company's growth and success An annual in-person company kick-off where the team connects, collaborates, and celebrates together
