Chuyển tới nội dung chính

AI Service Layer (Developer Guide)

Implementation of AI_FEATURE_GUIDE.md on branch feature/ai_generation (PR #96).

Learner-facing copy: AI Features.


Goals​

  1. One provider-agnostic client — nothing outside backend/app/services/ai/ imports an LLM SDK or vendor HTTP API.
  2. Three product surfaces: explain, async question generate, non-spoiling hint.
  3. Fail closed (HTTP 503, generic detail) on missing keys, timeouts, 4xx/5xx from the vendor, or unparseable structured output. Never 500, never a raw vendor message.
  4. Core product (cards, SM-2, exam create/answer/finish) must work with empty keys and with the AI kill switch off.
  5. Health probes never ping Gemini/OpenAI.

Out of scope: speech/audio, Redis/SQS, embeddings/RAG, auto-attaching generated questions to templates.


Architecture​

get_db() remains the sole HTTP commit point. The generate worker opens its own AsyncSessionLocal and commits — same shape as jobs/cleanup_guests.py.


Provider routing​

backend/app/core/exam_type_catalog.py:

Exam typeProviderSource
hsk, jlptGeminiCJK_EXAM_TYPES
toeic, ielts, custom, cardsOpenAIdefault

Override with AI_DEFAULT_PROVIDER=gemini|openai. Empty override keeps the catalog split.

Factory: get_ai_client(exam_type=None) — never raises at construction. Empty key $\rightarrow$ UnavailableAIClient (available is False). Routes map that to 503 immediately (generate also fails fast on enqueue so the UI does not poll a doomed job).

Adapters talk HTTP via httpx (already a backend dep). Timeouts default to AI_REQUEST_TIMEOUT_SECONDS (20s). Retries: up to 3 attempts on timeout / 429 / 5xx only.

Logs at info: provider, latency_ms, ok. Never keys, never full user content.


Authz and errors​

RuleStatus
Missing/invalid token401 (get_current_user)
Guest403 Registered account required
Kill switch ai=false503 on every /api/ai/*
Card not owned / session not owned / job not owned / question not in session AnswerRecord404 (do not confirm existence)
Owner asks explain on an in-progress session409
Provider down / timeout / malformed / spoiler after retry503 AI features are temporarily unavailable
Rate limit429 + Retry-After (SlidingWindowLimiter)

Rate limits (keyed by user id, not IP):

LimiterBudget
ai_explain_limiter20 / 60s
ai_generate_limiter5 / 3600s
ai_hint_limiter10 / 60s

Do not reuse the auth IP limiters. Tests disable these in conftest.py like the login limiters.

Exam invariants (do not work around):

  • Template-by-id reads go through ExamService.readable_template_or_404 (is_public OR owned, else 404). Session-owned paths follow get_session_details (already scoped).
  • Session composition is the frozen AnswerRecord set, not the live template.
  • Generate inserts via QuestionService.create_question only. Never update_question, never write exam_template_questions.
  • Do not serialize session questions through a question-ownership-gated builder (seeded questions have user_id = NULL). AI responses return { explanation } / { hint } only.

HTTP contract​

All /api/ai/* bodies/responses use camelCase aliases.

Feature flags (kill switch)​

Same “absence = enabled” semantics as Exam-Type Feature Flags. Dedicated feature_flags table — do not store ai as a fake exam type.

MethodPathAuth
GET/api/feature-flagsNone (global config)
PATCH/api/feature-flags/{key}Root admin

GET example:

[{ "key": "ai", "enabled": true }]

PATCH /api/feature-flags/ai body: { "enabled": false }. Unknown key $\rightarrow$ 404.

Explain​

POST /api/ai/explain-card
{ "cardId": "<uuid>", "locale": "en" }

POST /api/attempts/{attemptId}/items/{position}/explain
{ "locale": "en" }

200:

{ "explanation": "...", "cached": false, "provider": "openai" }

Attempt explain is gated in services/assessment/explain.py: completed attempts, or mode=practice (in-progress exam explain stays blocked). It uses the pinned question/stimulus versions on the attempt item. Cache table ai_explanations, unique (kind, subject_key, locale).

Hint​

Removed — there is no /api/ai/hint route on the assessment-v2 stack.

Generate​

POST /api/ai/generate-questions
{ "topic": "...", "examType": "toeic", "count": 5, "difficulty": "medium", "locale": "en" }

202 { "jobId": "<uuid>" }

GET /api/ai/jobs/{jobId}
{ "jobId": "...", "status": "pending|running|succeeded|failed", "result": { "questionIds": [] }, "error": null }

count 1–10. examType in toeic|ielts|hsk|jlpt|custom. Worker: jobs/generate_questions.py via FastAPI BackgroundTasks. Zero valid candidates $\rightarrow$ failed, not succeeded + []. A running job older than 5 minutes is flipped to failed on the next owner GET.

Job lifecycle (AIGenerationJob.status, services/ai_generate_service.py):


Data model (Alembic)​

RevisionTable
0012_feature_flagsfeature_flags (key PK, enabled, updated_at). Seeds ai=true.
0013_ai_explanationsai_explanations + unique (kind, subject_key, locale)
0014_ai_generation_jobsai_generation_jobs (user_id CASCADE, status, payload, result, error, timestamps)

Missing feature_flags row $\implies$ enabled. Staging/prod: alembic upgrade head via existing entrypoint.sh.


Frontend​

Card explain helper: frontend/src/shared/ai.ts (explainCard $\rightarrow$ /api/ai/explain-card). Attempt explain and generation live under features/assessment/ (runtime ask panel, authoring generate form) and call assessment_ai routes. There is no features/ai/ package and no hint control.

Admin toggle: AdminFlagsPanel $\rightarrow$ PATCH /api/feature-flags/ai.


Environment​

VariableDefaultNotes
GEMINI_API_KEY""Empty = Gemini unavailable
OPENAI_API_KEY""Empty = OpenAI unavailable
AI_DEFAULT_PROVIDER""gemini / openai override
AI_REQUEST_TIMEOUT_SECONDS20Per provider call

No frontend env vars. Secrets stay on Railway.


Operator guide​

Instant disable (no deploy):

  1. Admin Console $\rightarrow$ Feature Flags $\rightarrow$ AI features $\rightarrow$ Deactivate, or
  2. PATCH /api/feature-flags/ai { "enabled": false }, or
  3. UPDATE feature_flags SET enabled = false, updated_at = now() WHERE key = 'ai';

/api/health and /api/health/ready stay green during a vendor outage.

Rollback: leave the flag off; revert PR #96 if you need the code gone. Cache and job rows are disposable; inserted questions are real bank data (existing Postgres backups).

In-process worker limits: a process crash can leave a job running until the 5-minute stale GET marks it failed. Redis/RQ is the documented scale trigger — do not add it until that is measured.


Tests​

CI must never call Gemini or OpenAI. Adapters accept a fake transport; services patch get_ai_client.

ModuleCovers
tests/test_ai_client.pyRouting, empty key, retry vs 4xx, structured parse
tests/test_feature_flags.pyPublic GET, admin PATCH, missing = enabled
tests/test_ai_explain.pyHappy, 401, guest 403, non-owner 404, in-progress 409, 503, flag off
tests/test_ai_generate.py202, owner poll, unattached insert, other-user 404, zero-valid $\rightarrow$ failed
tests/test_ai_hint.pyHappy, spoiler regression (must not leak correct_answer), completed 404, missing AnswerRecord 404
tests/test_exam_visibility.pyRe-run after wiring — no authz regressions

File map​

AreaPath
Clientbackend/app/services/ai/
Product servicesbackend/app/services/ai_{explain,generate,hint}_service.py
Routerbackend/app/routers/ai.py, feature_flags.py
Workerbackend/app/jobs/generate_questions.py
Frontendfrontend/src/features/ai/