Principal AI Engineer (Technical Lead)

PRINCIPAL AI ENGINEER

Technical Lead

San Francisco, CA • Hybrid • $250K-$350K

🚀 Join the Data Revolution at Prophecy

About Prophecy

Prophecy is revolutionizing enterprise data transformation with AI-native tools that turn data chaos into reliable insights. We're not just building software—we're reshaping how the world's leading companies work with data. Our AI agents collaborate seamlessly with humans through intuitive visual and document interfaces, empowering teams to analyze, transform, and prepare data at enterprise scale.

🌟 Don't miss the rocket ship—join Prophecy and build the next data revolution! 🌟

The Role

As Principal AI Engineer (Technical Lead), you'll be at the forefront of AI innovation. This is a hands-on leadership role where you'll architect breakthrough solutions, mentor top talent, and drive technical excellence across our AI initiatives. You'll work on cutting-edge problems that directly impact how enterprises worldwide leverage data.

What You'll Bring

Technical Mastery

Deep expertise in at least one:

• ML/LLM: Led fine-tuning (SFT, DPO, RLHF), distillation, quantization with PyTorch/HuggingFace

• Production AI agents: Shipped with LangChain, LlamaIndex, CrewAI

• RAG systems: Expert in vector DBs and prompt orchestration

Leadership Excellence

• Code weekly prototypes, debug complex issues, and unblock your team

• Mentor staff engineers while remaining the go-to expert on the hardest problems

• Partner with PMs and executives on strategy, metrics, and trade-offs

Proven Track Record

• Led 2+ open-ended, user-facing AI projects from ambiguity to impact

• Owned vision, roadmap, and MVPs for AI initiatives

• Presented technical strategies to the C-suite and board

• Enforced MLOps/DevOps excellence: CI/CD, observability, A/B testing

Agentic AI Specialization

• Built multi-agent systems (ETL automation, code-gen loops)

• Created evaluation harnesses and benchmarking systems

• Delivered reliability and latency improvements at scale

Cloud Architecture

• Architected systems handling 10,000+ RPS (AWS/GCP, Kubernetes, serverless)

• Fluent in Python, Go, Java, Scala

• Expert in APIs, async systems, and performance optimization

What Makes You Stand Out

• Experience with code generation (Codex, text-to-SQL) or semantic extraction

• Knowledge graphs expertise (Neo4j, Neptune)

• Big data engine experience (Apache Spark)

• Compiler development (SQL, Python, Scala)

• ML model optimization for low-latency, high-throughput production

• Open-source AI/ML contributions (Hugging Face, PyTorch)

• Retrieval systems or vector database expertise (Pinecone, Weaviate)

• Strategic tech evaluation and build vs. buy decision-making

Position Details

Experience Level

7+ years in industry (Mid to Staff Level)

Compensation

$250,000 - $350,000 base salary

Location

San Francisco, California (Hybrid)

Note: Compensation reflects base pay only. The total package includes equity and comprehensive benefits. Pay varies based on achievements, skills, experience, and location.

Benefits & Perks

We invest in our team's success and well-being:

• 99% employee health insurance coverage, 75% for dependents

• $200/month stipend for remote work, wellness, gym, and self-care

• Flexible PTO to recharge when you need it

• Professional development allowance for continuous learning

• Company-sponsored Long Term Disability and Life Insurance

• FSA/HSA, dental, vision, and comprehensive health benefits

• Direct impact on an innovative, industry-leading platform

• End-to-end ownership of transformative projects

*Benefits and perks may vary by country