Role Description

About SME Careers As AI systems become more sophisticated, they rely on human knowledge to better understand context and the world. At SME Careers, we partner with leading AI labs to improve how the next generation of intelligent systems learn, reason, and communicate. SME Careers is a fast-growing AI data services company and subsidiary of SuperAnnotate that provides AI training data to many of the world's top AI companies and foundation model labs. Your expertise as an AI Engineer will directly help improve the world's premier AI models. Your expertise will play an important role in shaping the future of AI. ## About the Role In this hourly, remote contract role, you will review and compare AI-generated responses and/or generate training content, evaluating reasoning quality and step-by-step problem-solving while providing expert feedback that helps models produce accurate, logical, and clearly explained answers. You will assess solutions for accuracy, clarity, and adherence to the prompt; identify methodological or conceptual errors; fact-check key claims when required; write high-quality explanations and model solutions; and rate and compare multiple AI responses using consistent criteria. This position is with SME Careers, a fast-growing AI Data Services company and subsidiary of SuperAnnotate that provides AI training data for many of the world’s largest AI companies and foundation model labs. Your work will directly help improve the world’s premier AI models by strengthening both training signal quality and evaluation rigor. ## What You'll Do - Develop AI Training Content: Create detailed prompts in various topics and responses to guide AI learning, ensuring the models reflect a comprehensive understanding of diverse subjects. - Optimize AI Performance: Evaluate and rank AI responses to enhance the model's accuracy, fluency, and contextual relevance. - Ensure Model Integrity: Test AI models for potential inaccuracies or biases, validating their reliability across use cases. - Write clear, C1+ English rationales that explain scoring decisions, highlight reasoning gaps, and propose concise corrections without changing the intent of the original prompt. - Build and maintain evaluation rubrics (scorecards) for common ML/DL task types (classification, regression, ranking), including decision rules for ambiguous cases and severity levels for errors. - Perform structured error analysis on model outputs and training datasets (e.g., label noise, class imbalance, spurious correlations), and recommend targeted data additions or guideline updates. - Run quality controls on training/evaluation data (gold sets, calibration tasks, inter-annotator agreement checks such as Cohen’s kappa/Krippendorff’s alpha) and document recurring disagreement patterns. ## What You Bring - 2–5 years of relevant professional experience in applied machine learning, deep learning, or AI evaluation (industry, research, or applied data science settings). - Minimum Bachelor's degree in Computer Science, Machine Learning, Data Science, Statistics, or a closely related field, or equivalent practical experience. - English proficiency: Minimum C1 level. - Previous experience with AI data training, annotation, or evaluating AI-generated content is strongly preferred. - Working knowledge of core ML methods (supervised/unsupervised learning, feature engineering, regularization, optimization basics) and how to select appropriate evaluation metrics for different task types. - Practical understanding of DL fundamentals (loss functions, backpropagation behavior, normalization, overfitting controls) sufficient to diagnose model behavior from training curves and error patterns. - Proficiency with Python for analysis and dataset inspection (e.g., NumPy, pandas) and familiarity with at least one DL framework (PyTorch or TensorFlow) for reading model outputs and basic experimentation. - Ability to reason about data quality risks (label noise, ambiguity, class imbalance, sampling bias, distribution shift) and apply structured error analysis to propose corrective actions. - Detail-oriented and self-directed work style suitable for remote, flexible-hour contract delivery, including consistent documentation of decisions and outcomes. ## Application & Onboarding Process 1. Apply with your CV/resume and a short summary of your ML/DL evaluation experience. 2. Complete a short, time-boxed skills screening focused on model evaluation, error analysis, and data-quality judgment. 3. Participate in a structured interview covering ML/DL fundamentals and evaluation decision-making. 4. Onboarding: contract setup, evaluation guidelines review, and calibration tasks to align scoring standards. 5. Start tasking with flexible hours; ongoing feedback is provided through periodic quality reviews. ## Job Details - Company: SME Careers (Subsidiary of SuperAnnotate) - Role: AI Engineer (Mid) - Employment type: Contract (Hourly) - Location: Remote (Flexible hours) - Target country: AM - Hourly pay range (USD): $11.08–$15.51 - Openings: 1 ## Why Join Us - Flexible, remote contract work with hourly pay. - Work on real evaluation and training workflows used to improve widely deployed AI systems. - Clear scoring guidelines, calibration support, and quality feedback to help you produce consistent, high-signal work. - Exposure to a variety of ML/DL task types (classification, regression, ranking) and practical evaluation challenges.