AI Grading

We built the solution as wrapper microservices around the existing platform. This approach enabled rapid AI deployment while maintaining zero changes to the legacy grading module. The fragile codebase stayed untouched.

The core grading pipeline uses LangGraph to orchestrate separate AI agents. One agent retrieves relevant curriculum content from the vector database. Another evaluates student responses against rubrics. This separation of concerns improved accuracy and made the system debuggable.

The multi-agent approach solved a critical problem: grounding AI responses in approved curriculum. By retrieving context before evaluation, we ensured grading aligned with course materials rather than hallucinating standards.

We implemented Retrieval-Augmented Generation with a vector database containing all approved curriculum content. Before grading any assignment, the system retrieves relevant lesson materials, rubrics, and example answers.

This increased grading accuracy from generic LLM responses to curriculum-specific evaluation. More importantly, it built educator trust. Teachers could see exactly which materials the AI referenced when making grading decisions.

High-volume assignment processing required asynchronous job handling. We used Redis queues to manage 1000+ simultaneous assignments without timing out user requests. Students submit work, receive immediate confirmation, and get results within minutes rather than days.

Achieving 95% accuracy required more than good prompts. We built systematic quality assurance into every layer of the system.

PromptFoo runs continuous evaluation against sample answers with known correct grades. Every prompt change or model update gets tested against this benchmark. This prevented quality drift as the system evolved.

The automated testing caught edge cases early. When accuracy dropped on specific question types, we identified the pattern before it reached students.

LangFuse provides real-time monitoring of AI decision traces. We can see exactly which curriculum content the retrieval agent found, how the evaluation agent scored each rubric criterion, and where confidence was low.

This made the AI transparent rather than a black box. When educators questioned a grade, we could show the complete reasoning chain. Transparency built trust.

The remaining 5% error rate required human expertise. We built a teacher-facing rubric tool hosted on Railway where educators write, test, and manage lesson-specific rubrics.

This solved the scale problem at its source. Instead of fixing every edge case in code, we put control in expert hands. Teachers refined rubrics for their specific content, and the AI applied them consistently across thousands of students.

Student-facing AI responses go through OpenAI Moderation API plus custom filters. We've maintained zero incidents of inappropriate content reaching students. Safety wasn't an afterthought; it was built into the architecture from day one.

Several architectural choices proved critical to success.

• Wrapper Microservices Over Platform Rewrite: Integrating as microservices rather than modifying the legacy platform enabled rapid deployment without risk. This approach works for any organization with fragile core systems that still need modern capabilities.
• RAG Over Fine-Tuning: We chose retrieval-augmented generation instead of fine-tuning models. RAG allowed curriculum updates without retraining, making the system maintainable by educators rather than ML engineers.
• Multi-Agent Over Single-Prompt: Separating retrieval and evaluation into distinct agents improved both accuracy and debuggability. When grading failed, we could identify whether the problem was finding the right curriculum content or applying the rubric.
• Automated Testing Over Manual QA: PromptFoo automated testing caught regressions before they reached production. Manual QA couldn't scale to the volume of assignments flowing through the system.

The grading system proved the architecture. The platform extended it to other bottlenecks.

• Automated Subtitle Generation: The platform processed 100,000+ videos with automated subtitle generation, replacing an outsourced subtitling process. Same AI infrastructure, different application.
• AI-Powered Tutoring: The RAG system that grounds grading also powers student-facing tutoring. Students ask questions and receive answers grounded in approved curriculum, maintaining educational standards while scaling support.