Questions

Questions are Fraya's non-artifact assessment layer. They cover both in-course question formats and the final post-course quiz.

Artifacts remain a separate production domain: use them when the learner should perform a structured interactive action rather than answer a question.

What belongs here

This domain includes:

  • section-level knowledge checks such as ChoiceQuestions and FillTheBlanks;
  • open-ended question formats such as QuestionWithFeedback;
  • the final course quiz generated as a publishing asset.

This domain does not include:

  • Artifact interactions;
  • outline planning rules for where assessments should appear in the course;
  • general pedagogical rationale for assessment distribution.

In-course question formats

Section-level question formats are produced during section content generation. They are used when the learner should answer, select, or complete something inside a section rather than manipulate an interactive exercise.

Core formats:

  • ChoiceQuestions — factual recall, interpretation, true/false, single-choice, and multiple-choice blocks;
  • FillTheBlanks — terminology, definitions, and contextual recall;
  • QuestionWithFeedback — open-ended answer evaluated against explicit criteria.

For answer-based question formats, feedback should not stop at saying that an option is wrong. Incorrect-answer feedback should explicitly connect the learner to the correct answer or the distinguishing concept that makes it correct.

Question is intentionally disabled. For free-text answers, use QuestionWithFeedback instead.

Format selection rules

Choose the question format by both assessment type and learner action:

  • use ChoiceQuestions and FillTheBlanks for knowledge checks with clearly correct answers;
  • use QuestionWithFeedback for analysis, judgment, critique, or open responses;
  • never use ChoiceQuestions for self-assessment or reflection;
  • use Artifact instead of a question when the intended activity is matching, categorizing, ordering, mapping, simulating, or other structured interaction.

Final quiz

The final quiz is not a section format. It is a course-level publishing asset generated after the outline is stable.

Runtime source of truth:

  • generated in two steps: flow_asset_quiz produces questions with correct answers, flow_asset_quiz_distractors generates distractors in a separate focused pass;
  • rendered in the target course language, not in a default fallback language;
  • uses the localized course title and course localization glossary so wording, terminology, and feedback stay aligned with the localized course shell.

Current contract:

  • 10–16 questions across the full course scope;
  • SingleChoiceQuestion only;
  • one correct answer per question;
  • answer-level feedback on every option;
  • incorrect-answer feedback must explicitly link back to the correct answer or the distinguishing concept, not only explain why the chosen answer is wrong;
  • question text limited to 255 characters with no Markdown;
  • answer options balanced in length and specificity — the correct answer must not be the longest option; at least one distractor must be longer than the correct answer;
  • questions must be framed as universal knowledge, never as course-specific ("According to the course…" phrasings are forbidden).
  • Section formats — ChoiceQuestions, FillTheBlanks, QuestionWithFeedback format specs
  • Section types — Knowledge_Assessment and Self_Assessment types
  • Course schema — where questions and quiz live in the JSON
  • Course design — assessment distribution rules and quotas
  • Regeneration rules — questions are regenerated, not translated
  • prompts/injections/assessments_instructions.yaml
  • prompts/asset/flow_asset_quiz.yaml
  • prompts/asset/flow_asset_quiz_distractors.yaml