ML engineer

About Charm AI

Charm AI makes — a voice-sensing pendant that detects the gap between what someone says and how their voice actually sounds. Built on Pythagorean acoustic physics and the Navarasa framework, it is a state detector not only an emotion labeller. We are a small fast-moving team building at the intersection of emotion labeling, ancient wisdom and measurable science.

About the role

We are looking for a ML engineer to own the full AI stack. Your first and most important responsibility is taking the speech emotion recognition model we have inherited and building it into our proprietary harmonic vocal state engine — a physics-based system that detects vocal state through frequency and acoustic analysis rather than conventional emotion labelling. From there you will own the LLM infrastructure and be the person who keeps all AI systems running, improving, and scaling as the product grows.

This is a hands-on engineering role. You will be working directly with the founder and lead developer. There is no committee, no bureaucracy, and no one else to hand problems to. You need to be someone who owns things end to end.

Responsibilities

Your first workstream is the harmonic vocal state engine. You will take the existing 30-class audio speech emotion recognition model, the dataset, checkpoints, pipeline, and documentation handed over from our previous vendor, and extend it into a physics-based harmonic detection system built on Fourier-derived acoustic analysis and the Navarasa rasa framework. The Hume AI benchmark is no longer available so you will need to design and implement your own validation methodology. Options include Gemini's emotion labelling API as a replacement benchmark, cross-validation against open datasets like IEMOCAP and RAVDESS that are already in our training corpus, and building an internal ground truth evaluation framework. You will own this validation approach from design through to implementation.

Your second workstream is LLM infrastructure. You will deploy and manage a dual-instance vLLM setup on GCP running Llama 3.1 8B on GPU 0 for fast-lane prompts and Qwen 2.5 32B on GPU 1 for reasoning-heavy prompts, on a g2-standard-24 instance. You will own all prompt engineering, output validation, Pydantic schema enforcement, retry logic, and quality monitoring across 42 production prompts. You will handle migrations and scaling without external vendor involvement.

Your ongoing responsibility is model quality. You are the person who catches regressions before users do, designs evaluation pipelines, and continuously improves model performance as real-world pendant audio accumulates.

What we need from you

You need to be comfortable working in the frequency domain — FFT, Fourier analysis, spectral features — and experienced with audio processing libraries like librosa or torchaudio. This is the foundation of the harmonic engine work. Signal processing and acoustic feature extraction is a very important technical skill for this role.

Speech emotion recognition experience is great to have. You should have hands-on experience with audio ML models — wav2vec2, HuBERT, WavLM, or similar — and understand how SER models are trained, evaluated, and fine-tuned.

PyTorch is required. You will be working directly with model checkpoints, training loops, and fine-tuning pipelines.

Model evaluation methodology matters here more than in most roles because our primary external benchmark no longer exists. You need to be someone who can design a rigorous evaluation framework from scratch, whether that means using Gemini as an alternative benchmark, cross-validating against existing open datasets, or building a ground truth labelling pipeline.

Python at a production level is required. Not scripts — production code that other engineers can read, extend, and maintain.

GCP experience is needed for managing the existing infrastructure, storage, and deployment configuration.

vLLM deployment experience is required for the LLM infrastructure workstream.

Strong written English is essential. We are an async-first distributed team and clear written communication is how we stay aligned.

Nice to have

Experience designing or building physics-based audio analysis systems. Familiarity with the Navarasa framework or other classical acoustic or musical theory systems. FastAPI experience for the inference API layer. Prior experience replacing or migrating away from a third-party AI benchmark or provider. Genuine interest in consciousness, sound, acoustic physics, or the intersection of emotions and modern ML.

The hiring test

We do not do whiteboard interviews. We will give you two things and 48 hours. First, the inherited SER model validation report and dataset documentation — we want to see your assessment of where the model stands and how you would approach validation without a Hume benchmark. Second, a Python file from our harmonic engine — we want a one-page technical assessment of the architecture and where you would take it next. Anyone who can do both of these clearly and confidently in 48 hours is the person we are looking for.