Agentic-Evaluation-System

AutoGrade DNN + LLM

This project is an Automated Assignment Grading System that combines:

• **DNN ** → learns from ~10% convenor-scored assignments
• LLM (rubric-based scoring) → provides general evaluation
• Fusion & Calibration → integrates both to produce final consistent scores

---

Project Structure

AI_Module/
├─ main.py                                       # Main entry point
├─ data/
│   ├── raw/                                     # ASSIGNMENT REQUIREMENT & RUBRIC
│   └── test
│   └── marked/                                  # Marked assignments
│       ├── marks                                # marks
│       └── assignments                          # assignments
├─ artifacts/                                                         
│   ├── rubric                                   # All files for rubric
│       ├── rubric_kw.json #rubric dimension 
│       ├── assignment_cleaned_full.json   
│       ├── assignment_cleaned_para.json
│       ├── rubric_cleaned_full.json
│       ├── rubric_cleaned_para.json
│       ├── rubric_generation.json.              #rubric generated by LLM(assign requirements+rubric)
│       ├── rubric_teacher_study_all.json        #summary_all marked assignment(zid, doc_text,images, tables, scores)
│       ├── rubric_teacher_study_selected.json   #marked files used for teacher rubric extraction 
│       └── rubric_teacher.json                  # Teacher rubric extraction
│   └── prediction                               # prediction results
│       └── assignements_score.json              # All prediction results
├─ scripts/                                      # Step-wise runnable scripts
│   ├── config.py
│   ├── rubric_assign_req.py                     #step 0: Generate rubric by assign_requirement+rubric
│   ├── teacher_rubric_learning.py               #step 1: Generate teacher rubric
│   └── predict_scores.py                        #step 2: Mark assignments                                                             
├─ src/                                          # Core source code (preprocessing, models, etc.) 
│   ├── preprocess                               # Data preprocessing                             
│       ├── Loader.py                            # Load raw data                                  
│       └── Clean.py                             # Unified format                                 
│   ├──rubric_retriever                          # Similarities retriever                         
│       ├── teacher_summary_report.py            # Maked files analysis                     
│       └── rubric_teacher.py                    # Generated teacher's rubric by llm       
│   ├── prompt
│       ├── rubric_generation.md
│       ├── teacher_rubric.md
│       └── teacher_guided_scoring.md
│   ├── LLM 
│       └── LLM_Client.py
│   └── scorer 
│       └── scorer.py
├─ requirements.txt                              # Python dependencies
└─ README.md                                     # Project documentation

---

1.Setup

1.1 Create environment

(1) Create a new environment:

conda create -n 9900 python=3.9

(2) Activate the environment:

conda activate 9900

(3) Install dependencies:

pip install -r requirements.txt

(4) Open AI API setup:

export OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxx"

(5) Commands:

python main.py --o 0                               # Specific rubric generation
python main.py --o 1                               # Teacher scoring pattern generation
python main.py --o 2                               # Assignment prediction
python main.py --o all                             # Whole progress

Update GitHub workflow:
git checkout ai_module
git pull origin ai_module
git status
git add .
git commit -m "describe changes"
git push origin ai_module
git rm --cached .DS_Store