Updated

Author: PythonPlumber

docs/internals/process_scheduler.md

@@ -1,99 +1,242 @@
 # Process & Scheduler Internals
 
 ## 1. Overview
-The Kernel employs a preemptive, multi-class scheduler supporting SMP (Symmetric Multiprocessing). It handles task switching, CPU load balancing, and real-time constraints.
+The Kernel employs a preemptive, multi-class scheduler supporting SMP (Symmetric Multiprocessing). It handles task switching, CPU load balancing, and real-time constraints using a modular scheduling class architecture.
 
 ## 2. Process Model (`process_t`)
 The `process_t` structure is the core entity, containing the execution context, memory map, and resource limits.
 
-### Key Fields
-- **Context**: `esp`, `eip`, `eflags`, general registers.
-- **Memory**: `page_directory` (CR3) and `vm_region_t` array for VMM.
-- **Scheduling**: `priority`, `time_slice`, `vruntime` (CFS), `rt_budget` (Real-Time).
-- **Identity**: `pid`, `uid`, `gid`, `caps` (Capabilities).
-- **Resources**: File descriptors (`fds`), Signals (`pending_signals`).
+### Data Structures
+```c
+// Core Process Control Block
+typedef struct process {
+    uint32_t pid;               // Process ID
+    uint32_t state;             // RUNNING, SLEEPING, ZOMBIE
+    uint32_t flags;             // KERNEL_THREAD, REALTIME
+
+    // Execution Context
+    regs_t context;             // Saved registers
+    uintptr_t kernel_stack;     // Kernel stack pointer (ESP0)
+    page_directory_t* cr3;      // Page Directory (Physical Address)
+
+    // Scheduling
+    struct sched_entity se;     // Scheduling entity info
+    struct sched_class* class;  // Pointer to handling class
+    
+    // Links
+    struct process* parent;
+    struct list_head children;
+    struct list_head siblings;
+    
+    // Resources
+    file_descriptor_t* fds[MAX_FDS];
+    signal_state_t signals;
+} process_t;
+
+// Scheduling Entity (Generic)
+struct sched_entity {
+    uint32_t priority;          // Static priority
+    uint64_t vruntime;          // Virtual Runtime (CFS)
+    uint64_t time_slice;        // Remaining slice (RR)
+    uint64_t deadline;          // Absolute deadline (EDF)
+    struct list_head run_list;  // Link to runqueue
+};
+```
 
 ### Lifecycle Diagram
 ```ascii
-+---------+     fork()     +-------+     exec()     +---------+
-| Parent  | -------------> | Child | -------------> | Running |
-+---------+                +-------+                +---------+
-                              |                          |
-                              | wait()                   | exit()
-                              v                          v
-                         +---------+               +---------+
-                         | Running | <------------ | Zombie  |
-                         +---------+               +---------+
+      fork()          scheduler_pick()
+  +---> [ NEW ] ---------------------> [ READY ] <----+
+  |                                      ^  |         |
+Create                                   |  | schedule()
+  |                                      |  v         | preempt()
+  +---------------------------------- [ RUNNING ] ----+
+                                         |
+                                         | block() / wait()
+                                         v
+                                     [ BLOCKED ]
 ```
 
-### Lifecycle Steps
-1.  **Creation**: `process_create` allocates the structure and a new Page Directory.
-2.  **Fork**: `process_fork` performs a copy-on-write clone of the parent.
-3.  **Exec**: `process_exec` replaces the memory image with an ELF binary.
-4.  **Exit**: `process_exit` releases resources but keeps the structure as a zombie until `process_wait`.
+## 3. Scheduling Classes Implementation
+The scheduler uses a virtual function table (VTable) approach to support multiple scheduling policies.
 
-## 3. Scheduling Classes
-The scheduler supports multiple classes, prioritized in order:
+### Scheduler Class Interface
+```c
+struct sched_class {
+    const char* name;
+    
+    // Core API
+    void (*enqueue_task)(struct runqueue* rq, process_t* p);
+    void (*dequeue_task)(struct runqueue* rq, process_t* p);
+    void (*yield_task)(struct runqueue* rq);
+    
+    // Selection
+    process_t* (*pick_next_task)(struct runqueue* rq);
+    
+    // Events
+    void (*task_tick)(struct runqueue* rq, process_t* p);
+    void (*task_fork)(process_t* p);
+};
+```
 
 ### Priority Matrix
-| Class ID | Name | Priority Range | Algorithm | Use Case |
+| Class ID | Name | Priority | Algorithm | Implementation |
 | :--- | :--- | :--- | :--- | :--- |
-| 0 | `SCHED_CLASS_DEADLINE` | N/A (Earliest Deadline) | EDF | Hard Real-Time (Audio, Control) |
-| 1 | `SCHED_CLASS_RT` | 0-99 | FIFO/RR | Soft Real-Time (UI, Input) |
-| 2 | `SCHED_CLASS_CFS` | 100-139 | Red-Black Tree | Normal User Tasks |
-| 3 | `SCHED_CLASS_IDLE` | 140 | Round-Robin | System Idle Loop |
-
-### A. Deadline (`SCHED_CLASS_DEADLINE`)
-- **Algorithm**: Earliest Deadline First (EDF).
-- **Parameters**: `runtime`, `period`, `deadline`.
-
-### B. Real-Time (`SCHED_CLASS_RT`)
-- **Algorithm**: Fixed Priority Preemptive.
-- **Logic**: Higher priority tasks always preempt lower ones. Round-robin for equal priority.
-
-### C. CFS (Completely Fair Scheduler - `SCHED_CLASS_CFS`)
-- **Algorithm**: Red-Black Tree (or sorted list) based on `vruntime`.
-- **Logic**: Tasks with the lowest virtual runtime are picked next. `vruntime` increases as the task runs, weighted by priority.
-
-## 4. SMP (Symmetric Multiprocessing)
-The `smp_rally` subsystem handles multicore initialization and management.
-
-### Architecture
-- **BSP (Bootstrap Processor)**: Boots the system and wakes up APs (Application Processors).
-- **AP Initialization**:
-    1.  BSP sends INIT IPI.
-    2.  BSP sends SIPI (Start-up IPI) with a trampoline vector.
-    3.  AP jumps to protected mode and enables paging.
-    4.  AP calls `scheduler_init` for its local queue.
-
-### Load Balancing
-- **Affinity**: `cpu_mask` restricts which CPUs a process can run on.
-- **Migration**: `scheduler_balance_load` moves tasks from busy queues to idle queues.
-
-## 5. Context Switching
-The low-level mechanism (`switch_task`, `context_switch`) saves and restores state.
-
-### Flow
-1.  **Interrupt**: Timer or System Call triggers the scheduler.
-2.  **Save**: Current registers are pushed to the kernel stack.
-3.  **Pick**: The scheduler selects the next process.
-4.  **Switch**:
-    - `CR3` is updated (TLB flush).
-    - Kernel stack pointer (`ESP0` in TSS) is updated.
-    - Registers are popped from the new task's stack.
-5.  **Return**: `IRET` jumps to the new task's `EIP`.
-
-## 6. Configuration & Tuning
-System parameters can be adjusted in `sched_config.h`:
+| 0 | `SCHED_DEADLINE` | Top | EDF | Red-Black Tree by Deadline |
+| 1 | `SCHED_RT` | High | FIFO/RR | Array of Linked Lists (O(1)) |
+| 2 | `SCHED_CFS` | Normal | CFS | Red-Black Tree by vruntime |
+| 3 | `SCHED_IDLE` | Low | Round-Robin | Single Linked List |
+
+## 4. Core Scheduler Implementation
+
+### The Runqueue (`runqueue_t`)
+Each CPU has its own runqueue to minimize lock contention.
+
+```c
+typedef struct runqueue {
+    spinlock_t lock;
+    uint32_t nr_running;
+    process_t* curr;
+    
+    // Class-specific sub-queues
+    struct rb_root cfs_root;        // CFS Red-Black Tree
+    struct list_head rt_queues[100];// Real-Time Priority Arrays
+    struct list_head dl_root;       // Deadline Tree
+    
+    uint64_t clock;                 // Monotonic clock
+} runqueue_t;
+```
+
+### The `schedule()` Function
+This is the main entry point, called by interrupt handlers or voluntary yield.
+
+```c
+void schedule(void) {
+    int cpu = get_cpu_id();
+    runqueue_t* rq = &runqueues[cpu];
+    
+    spin_lock(&rq->lock);
+    
+    process_t* prev = rq->curr;
+    
+    // 1. Check for stack overflow (Debug)
+    check_stack_canary(prev);
+    
+    // 2. Put previous task back to queue (if still runnable)
+    if (prev->state == TASK_RUNNING) {
+        prev->class->enqueue_task(rq, prev);
+    }
+    
+    // 3. Pick next task
+    process_t* next = pick_next_task(rq);
+    
+    // 4. Context Switch
+    if (prev != next) {
+        rq->curr = next;
+        context_switch(prev, next);
+    }
+    
+    spin_unlock(&rq->lock);
+}
+```
+
+## 5. Context Switch Mechanism
+The low-level context switch is architecture-specific (x86 example). It must save callee-saved registers and switch the stack pointer.
+
+### Assembly Implementation (`switch.S`)
+```nasm
+; void switch_to(process_t* prev, process_t* next);
+global switch_to
+switch_to:
+    ; [ESP+4] = prev
+    ; [ESP+8] = next
+    
+    mov eax, [esp+4]    ; EAX = prev
+    mov edx, [esp+8]    ; EDX = next
+    
+    ; 1. Save Previous Context
+    push ebp
+    push ebx
+    push esi
+    push edi
+    
+    ; Save ESP to prev->thread.esp
+    mov [eax + OFFSET_ESP], esp
+    
+    ; 2. Restore Next Context
+    ; Load ESP from next->thread.esp
+    mov esp, [edx + OFFSET_ESP]
+    
+    ; 3. Load Page Directory (if different)
+    mov ecx, [edx + OFFSET_CR3]
+    mov ebx, cr3
+    cmp ecx, ebx
+    je .same_pd
+    mov cr3, ecx
+.same_pd:
+    
+    ; 4. Restore Registers
+    pop edi
+    pop esi
+    pop ebx
+    pop ebp
+    
+    ret
+```
+
+## 6. Tick Handler Integration
+The timer interrupt calls `scheduler_tick()` to update runtime statistics and trigger preemption.
 
 ```c
-#define SCHED_TICK_HZ 1000       // Timer interrupt frequency
-#define MIN_GRANULARITY_NS 100000 // Min slice for CFS
-#define LOAD_BALANCE_INTERVAL 50 // Balance every 50 ticks
+void scheduler_tick(void) {
+    int cpu = get_cpu_id();
+    runqueue_t* rq = &runqueues[cpu];
+    process_t* curr = rq->curr;
+    
+    spin_lock(&rq->lock);
+    
+    // Update generic stats
+    rq->clock++;
+    
+    // Class-specific tick handling
+    // e.g., CFS updates vruntime, RT decrements time_slice
+    curr->class->task_tick(rq, curr);
+    
+    spin_unlock(&rq->lock);
+}
 ```
 
-## API Reference
-- `scheduler_yield()`: Voluntarily give up CPU.
-- `scheduler_set_affinity(pid, mask)`: Bind process to specific cores.
-- `scheduler_set_deadline(...)`: Promote a task to Deadline class.
-- `process_fork(parent)`: Create a child process.
+## 7. SMP Load Balancing
+The scheduler periodically balances load across CPUs using work stealing.
+
+### Migration Logic
+1.  **Trigger**: Timer interrupt (every 200ms) or when a CPU goes idle.
+2.  **Find Busiest**: Scan other CPUs' runqueues to find the one with the highest load.
+3.  **Steal**: Move tasks from the busiest runqueue to the current one.
+4.  **Affinity Check**: Ensure the task is allowed to run on the current CPU (`p->cpu_mask`).
+
+## 8. Configuration (`sched_config.h`)
+Tunable parameters for system performance.
+
+```c
+#ifndef SCHED_CONFIG_H
+#define SCHED_CONFIG_H
+
+// Time Slices
+#define BASE_TIME_SLICE_MS    100
+#define MIN_TIME_SLICE_MS     10
+
+// Priorities
+#define MAX_PRIO              140
+#define MAX_RT_PRIO           100
+
+// CFS Parameters
+#define CFS_LATENCY_TARGET_NS 20000000ULL // 20ms
+#define CFS_MIN_GRANULARITY   4000000ULL  // 4ms
+
+// Load Balancing
+#define BALANCE_INTERVAL_MS   200
+#define MIGRATION_COST_NS     500000ULL
+
+#endif
+```

include/elf_loader.h

@@ -1,8 +1,70 @@
+#ifndef ELF_LOADER_H
+#define ELF_LOADER_H
+
 #include "types.h"
 
+// ELF Header Types
+typedef uint16_t Elf32_Half;
+typedef uint32_t Elf32_Word;
+typedef uint32_t Elf32_Addr;
+typedef uint32_t Elf32_Off;
+typedef int32_t  Elf32_Sword;
+
+#define EI_NIDENT 16
+
+typedef struct {
+    uint8_t     e_ident[EI_NIDENT];
+    Elf32_Half  e_type;
+    Elf32_Half  e_machine;
+    Elf32_Word  e_version;
+    Elf32_Addr  e_entry;
+    Elf32_Off   e_phoff;
+    Elf32_Off   e_shoff;
+    Elf32_Word  e_flags;
+    Elf32_Half  e_ehsize;
+    Elf32_Half  e_phentsize;
+    Elf32_Half  e_phnum;
+    Elf32_Half  e_shentsize;
+    Elf32_Half  e_shnum;
+    Elf32_Half  e_shstrndx;
+} Elf32_Ehdr;
+
+// Program Header Types
+#define PT_NULL    0
+#define PT_LOAD    1
+#define PT_DYNAMIC 2
+#define PT_INTERP  3
+#define PT_NOTE    4
+#define PT_SHLIB   5
+#define PT_PHDR    6
+
+#define PF_X       1
+#define PF_W       2
+#define PF_R       4
+
+typedef struct {
+    Elf32_Word  p_type;
+    Elf32_Off   p_offset;
+    Elf32_Addr  p_vaddr;
+    Elf32_Addr  p_paddr;
+    Elf32_Word  p_filesz;
+    Elf32_Word  p_memsz;
+    Elf32_Word  p_flags;
+    Elf32_Word  p_align;
+} Elf32_Phdr;
+
 typedef struct {
     uint32_t entry;
     uint32_t image_base;
+    uint32_t image_size;
 } elf_image_t;
 
+// Parse ELF header and validate
+// Returns 1 on success, 0 on failure
 int elf_loader_parse(const uint8_t* data, uint32_t size, elf_image_t* out);
+
+// Load ELF segments into current address space
+// Returns 1 on success, 0 on failure
+int elf_loader_load(const uint8_t* data, uint32_t size, elf_image_t* out);
+
+#endif // ELF_LOADER_H