HACKMAN – Reinforcement Learning Hangman Solver

HACKMAN is an AI-driven Hangman solver that combines Hidden Markov Models (HMM) and Deep Q-Networks (DQN) to learn efficient letter prediction strategies from a word corpus.
This implementation is optimized for use in Google Colab or any standard Python environment with PyTorch.

---

Features

• Hidden Markov Model (HMM) for probabilistic letter prediction
• Frequency-based baseline for comparison
• Deep Q-Network (DQN) agent with experience replay and target network
• Modular and well-documented implementation
• Progress tracking via tqdm
• GPU acceleration support (cuda detection)

---

Requirements

Install the following dependencies:

pip install torch tqdm numpy
(Colab includes these by default.)

Project Structure
graphql
Copy code
hackman/
│
├── corpus.txt        # Training word corpus
├── test.txt          # Test words for evaluation
├── hackman_final.py  # Main implementation
└── dqn_final.pth     # Saved DQN model after training
How It Works
Word Loading
Words are cleaned and standardized to include only A–Z characters.
Data is split into training (corpus.txt) and test (test.txt) sets.

Hidden Markov Model (HMM)
Builds initial and transition probabilities from the training corpus.
Used to estimate letter probabilities for masked words.

Hangman Environment
Simulates the game mechanics with lives, guessed letters, and word state.
Provides rewards for correct guesses and penalties for incorrect ones.

Deep Q-Network (DQN)
Trains a neural agent to optimize guessing decisions through reinforcement learning.
Uses replay buffer and target network for training stability.

Baselines

Frequency-based agent (common letters first)

HMM-greedy agent (probabilistic selection)

DQN agent (learned strategy)

Evaluation
Each agent is tested on unseen words. Results include:

Number of correctly solved words

Wrong and repeated guesses

Composite score balancing success rate and penalties

Usage
Run the script in a Colab notebook or Python terminal:

bash
Copy code
python hackman_final.py
Example output:

vbnet
Copy code
→ Frequency baseline
→ HMM-greedy baseline
→ DQN training (short demo)
Freq: Score=132.50, SuccessRate=54.00%  Wrong=120 Repeats=3
HMM:  Score=178.20, SuccessRate=62.00%  Wrong=97  Repeats=2
DQN:  Score=230.75, SuccessRate=73.50%  Wrong=80  Repeats=1
Key Components
Component	Description
HMM	Trains and applies transition models for letter prediction.
Hangman	Manages the game environment and reward system.
Net	Deep Q-network architecture for letter selection.
Replay	Experience replay buffer for training the DQN.
train_dqn()	Main training loop for the DQN agent.
eval_dqn()	Evaluates trained models on test data.
freq_order() / eval_freq()	Frequency-based baseline implementation.
eval_hmm()	Greedy HMM baseline evaluation.

Example Corpus Files
corpus.txt

nginx
Copy code
APPLE
BANANA
ORANGE
GRAPE
MANGO
test.txt

nginx
Copy code
CHERRY
PEACH
LEMON
Configuration Tips
Adjust the number of episodes for faster or deeper training:

python
Copy code
net, hmm = train_dqn(corpus, episodes=1000)
Increase to 5000–10000 episodes for improved accuracy.

The trained model is saved automatically as dqn_final.pth.

Dependencies
Library	Minimum Version
Python	3.8
PyTorch	2.0
NumPy	1.24
tqdm	4.66


Acknowledgements
Developed for educational use to demonstrate:

Probabilistic modeling with Hidden Markov Models

Reinforcement learning via Deep Q-Networks

Word and letter encoding for symbolic learning tasks