Data Scientist - Agentic AI
About the role
We are looking for a Data Scientist, Agentic AI to build and productionize machine learning models and LLM-powered agents using real-world business data. You’ll work at the intersection of applied data science and agentic AI, building models for forecasting, customer behavior, identity resolution, and other business use cases while designing AI agents that can use tools, make decisions, and know when to escalate to a human. This is a hands-on role for someone who can take models and AI systems from development to production and build reliable evaluation and monitoring around them.
What you'll do
- 1. Agentic AI
- Design, build, and evaluate LLM agents with tool use and function calling.
- Build agents that can interact with business systems and perform multi-step tasks.
- Design guardrails, confidence thresholds, and escalation logic.
- Build evaluation frameworks, including offline testing, edge cases, A/B testing, and regression testing.
- Develop and maintain RAG systems, including embeddings, vector search, chunking, retrieval tuning, and re-ranking.
- Manage prompt architecture, versioning, and change control.
- Partner on conversational and voice AI experiences, including latency and quality metrics.
- 2. Data Science & Machine Learning
- Build, validate, and deploy production machine learning models.
- Work on forecasting, customer lifetime value, customer defection, next-service prediction, identity resolution, and demand-related models.
- Own the full ML lifecycle—from feature engineering and training to deployment, monitoring, and retraining.
- Work with Python and modern ML tools against lakehouse/data warehouse environments.
- Monitor models for drift and performance degradation.
- 3. Business Impact
- Translate business questions into well-defined data science and modeling problems.
- Communicate model results and business impact clearly to technical and non-technical stakeholders.
- Document methodology, assumptions, and limitations.
Requirements
- 4+ years of professional experience applying data science in production.
- Strong Python and SQL skills.
- Hands-on experience building LLM agents with tool use/function calling.
- Experience building and evaluating RAG systems, including embeddings, vector search, chunking, and retrieval evaluation.
- Strong statistical and data science fundamentals.
- Experience deploying machine learning models to production.
- Experience with AWS cloud ML services such as Bedrock, SageMaker, or Lambda.
- Experience working with a lakehouse or data warehouse environment.
- Strong communication skills and ability to work independently.
- Nice to Have
- Experience with Databricks, Spark, or Delta Lake.
- Experience with agent frameworks and orchestration patterns.
- Experience with structured outputs/JSON-mode reliability.
- Voice AI or conversational AI experience.
- Experience with time-series forecasting or causal inference.
- Automotive retail, DMS, CRM, or related domain experience.
- Familiarity with AI safety and evaluation practices, including handling PII in prompts and logs.
About Full Scale
Full Scale is a fully remote-first company that helps businesses build dedicated teams of skilled software engineers. We make it easier for growing companies to find, onboard, and retain high-performing software talent.
AI Alerts shares third-party job opportunities for informational purposes only. We are not the employer and are not involved in the hiring process. Always verify the company and role through official channels before applying, and never pay to apply, train, onboard, process documents, or secure a job offer. Legitimate employers do not ask applicants for money. Read our Terms to learn more.