Skip to content

marouane-dev75/social-summarizer

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

20 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Time Reclamation App

Reclaim time wasted on social media by getting curated summaries instead of endless scrolling.

✨ Features

🎯 Smart Content Curation

  • YouTube Channel Monitoring: Automatically track and fetch new videos from your favorite channels
  • Intelligent Transcript Extraction: Extract video transcripts in multiple languages (English, French, etc.)
  • Configurable Video Limits: Control how many videos to process per channel

πŸ€– AI-Powered Summarization

  • Multi-LLM Support: Choose from various AI providers:
    • Local Models: LlamaCpp (GGUF models), Ollama (llama2, mistral, codellama, etc.)
    • Cloud Providers: Anthropic Claude (Haiku, Sonnet), OpenAI GPT (GPT-5, O4-mini)
  • Custom System Prompts: Tailor summaries to your preferences (conversational, technical, brief, etc.)
  • Flexible Provider Selection: Use different LLMs for different channels or content types

πŸ”Š Text-to-Speech Integration

  • Multiple TTS Engines:
    • Kokoro TTS: High-quality neural voices (American/British English, multiple voice options)
    • Piper TTS: Lightweight, fast synthesis with multi-language support
  • Audio Summary Generation: Convert text summaries into audio files for on-the-go listening
  • Voice Customization: Choose from various voices and accents

πŸ“± Smart Notifications

  • Telegram Integration: Receive summaries directly in your Telegram chat
  • Multi-Bot Support: Configure multiple notification channels for different content types
  • Audio Delivery: Get audio summaries sent directly to your messaging app

πŸ’Ύ Efficient Caching & State Management

  • Local Caching: Store transcripts and summaries to avoid redundant processing
  • SQLite Database: Track processed videos and maintain state across runs
  • Incremental Updates: Only process new content since last run

πŸ› οΈ Developer-Friendly

  • CLI Interface: Easy-to-use command-line tools for all operations
  • Modular Architecture: Clean separation of concerns with pluggable providers
  • Extensible Design: Easy to add new platforms, LLMs, or TTS providers
  • YAML Configuration: Simple, human-readable configuration files

πŸ“‹ Requirements

  • Python 3.8+
  • FFmpeg (for audio processing)
  • Optional: CUDA-capable GPU for faster local LLM inference

πŸš€ Quick Start

Installation

# Clone the repository
git clone https://github.com/marouane-dev75/social-summarizer
cd social-summarizer

# Install dependencies
pip install -r requirements.txt

Configuration

Create a config.local.yml file based on the example below:

# Platform Configuration
platforms:
  youtube:
    enabled: true
    channels:
      - name: "TechChannel"
        scrap: true
        url: "https://www.youtube.com/@TechChannel"
        max_videos: 5
        language: "en"
        cache_folder: "cache_data/youtube_transcripts/tech_channel"
        summary:
          enabled: true
          llm_provider: "ollama_local"
          tts_provider: "kokoro_english"
          notification_provider: "personal_bot"
          system_prompt: |
            You are a YouTube video summarizer that creates a single flowing 
            paragraph in plain text, using natural speech-friendly language 
            without formatting, symbols, or markdown, presenting the main 
            topic followed by key points with smooth transitions and ending 
            with a conclusion.

# Notification Configuration
notifications:
  providers:
    - name: "personal_bot"
      type: "telegram"
      enabled: true
      config:
        bot_token: "YOUR_BOT_TOKEN_HERE"  # Get from @BotFather on Telegram
        chat_id: "YOUR_CHAT_ID_HERE"      # Your Telegram chat ID
        timeout_seconds: 30
        retry_attempts: 3

# LLM Configuration
llm:
  providers:
    # Local Ollama instance
    - name: "ollama_local"
      type: "ollama"
      enabled: true
      config:
        base_url: "http://localhost:11434"
        model: "llama2"  # or mistral, codellama, etc.
        timeout_seconds: 120
        generation_config:
          temperature: 0.7
          num_predict: 4000
          top_p: 0.9
          top_k: 40
        default_system_prompt: "You are a helpful AI assistant."
    
    # Cloud provider example (optional)
    - name: "claude_assistant"
      type: "anthropic"
      enabled: false
      config:
        api_key: "YOUR_ANTHROPIC_API_KEY"
        model: "claude-haiku-4-5"
        max_tokens: 4000
        temperature: 0.7

# TTS Configuration
tts:
  providers:
    - name: "kokoro_english"
      type: "kokoro"
      enabled: true
      config:
        voice: "af_alloy"  # Available: af_heart, af_alloy, af_bella, am_adam, etc.
        lang_code: "a"     # a = American English, b = British English
        repo_id: "hexgrad/Kokoro-82M"
        sample_rate: 24000
        output_dir: "cache_data/tts"
    
    - name: "piper_french"
      type: "piper"
      enabled: false
      config:
        model_path: "/path/to/fr_FR-siwis-medium.onnx"
        output_dir: "cache_data/tts"

Usage

# Process YouTube channels and generate summaries
python main.py youtube process

# Test LLM provider
python main.py llm test --provider ollama_local --prompt "Hello, how are you?"

# Test TTS provider
python main.py tts test --provider kokoro_english --text "This is a test."

# Test notification
python main.py notify test --provider personal_bot --message "Test notification"

# View database info
python main.py db info

# Show version
python main.py version

πŸ“ Project Structure

TimeReclamation/
β”œβ”€β”€ main.py                          # Application entry point
β”œβ”€β”€ requirements.txt                 # Python dependencies
β”œβ”€β”€ README.md                        # This file
β”‚
β”œβ”€β”€ src/time_reclamation/
β”‚   β”œβ”€β”€ __init__.py                  # Package initialization
β”‚   β”‚
β”‚   β”œβ”€β”€ config/                      # Configuration management
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ manager.py               # Config loader and validator
β”‚   β”‚   β”œβ”€β”€ config.yml               # Default configuration template
β”‚   β”‚   └── config.local.yml         # User-specific configuration (gitignored)
β”‚   β”‚
β”‚   β”œβ”€β”€ core/                        # Core business logic
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   └── youtube/                 # YouTube platform implementation
β”‚   β”‚       β”œβ”€β”€ __init__.py
β”‚   β”‚       β”œβ”€β”€ service.py           # Main YouTube service
β”‚   β”‚       β”œβ”€β”€ channel_manager.py   # Channel operations
β”‚   β”‚       β”œβ”€β”€ transcript_fetcher.py # Transcript extraction
β”‚   β”‚       β”œβ”€β”€ cache_manager.py     # Caching logic
β”‚   β”‚       β”œβ”€β”€ summary_service.py   # Summary generation
β”‚   β”‚       └── database.py          # YouTube-specific DB operations
β”‚   β”‚
β”‚   β”œβ”€β”€ infrastructure/              # External integrations
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ llm/                     # LLM providers
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   β”œβ”€β”€ interface.py         # LLM provider interface
β”‚   β”‚   β”‚   β”œβ”€β”€ manager.py           # LLM provider manager
β”‚   β”‚   β”‚   └── providers/
β”‚   β”‚   β”‚       β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚       β”œβ”€β”€ llamacpp.py      # LlamaCpp implementation
β”‚   β”‚   β”‚       β”œβ”€β”€ anthropic.py     # Claude implementation
β”‚   β”‚   β”‚       β”œβ”€β”€ openai.py        # OpenAI GPT implementation
β”‚   β”‚   β”‚       └── ollama.py        # Ollama implementation
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ tts/                     # Text-to-Speech providers
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   β”œβ”€β”€ interface.py         # TTS provider interface
β”‚   β”‚   β”‚   β”œβ”€β”€ manager.py           # TTS provider manager
β”‚   β”‚   β”‚   └── providers/
β”‚   β”‚   β”‚       β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚       β”œβ”€β”€ kokoro.py        # Kokoro TTS implementation
β”‚   β”‚   β”‚       └── piper.py         # Piper TTS implementation
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ notifications/           # Notification providers
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   β”œβ”€β”€ interface.py         # Notification provider interface
β”‚   β”‚   β”‚   β”œβ”€β”€ manager.py           # Notification provider manager
β”‚   β”‚   β”‚   └── providers/
β”‚   β”‚   β”‚       β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚       └── telegram.py      # Telegram implementation
β”‚   β”‚   β”‚
β”‚   β”‚   β”œβ”€β”€ database/                # Database management
β”‚   β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”‚   └── manager.py           # SQLite database manager
β”‚   β”‚   β”‚
β”‚   β”‚   └── logging/                 # Logging infrastructure
β”‚   β”‚       β”œβ”€β”€ __init__.py
β”‚   β”‚       └── logger.py            # Centralized logger
β”‚   β”‚
β”‚   └── interfaces/                  # User interfaces
β”‚       β”œβ”€β”€ __init__.py
β”‚       └── cli/                     # Command-line interface
β”‚           β”œβ”€β”€ __init__.py
β”‚           β”œβ”€β”€ manager.py           # CLI manager
β”‚           β”œβ”€β”€ command_pattern.py   # Command pattern implementation
β”‚           └── commands/            # CLI commands
β”‚               β”œβ”€β”€ __init__.py
β”‚               β”œβ”€β”€ base.py          # Base command class
β”‚               β”œβ”€β”€ youtube.py       # YouTube commands
β”‚               β”œβ”€β”€ llm.py           # LLM commands
β”‚               β”œβ”€β”€ tts.py           # TTS commands
β”‚               β”œβ”€β”€ notify_test.py   # Notification test command
β”‚               β”œβ”€β”€ summary.py       # Summary commands
β”‚               β”œβ”€β”€ db_info.py       # Database info command
β”‚               └── version.py       # Version command
β”‚
β”œβ”€β”€ docs/                            # Documentation
β”‚   β”œβ”€β”€ llm_system.md                # LLM system documentation
β”‚   β”œβ”€β”€ tts_system.md                # TTS system documentation
β”‚   β”œβ”€β”€ youtube_system.md            # YouTube system documentation
β”‚   └── summary_system.md            # Summary system documentation
β”‚
└── cache_data/                      # Runtime data (gitignored)
    β”œβ”€β”€ youtube_transcripts/         # Cached transcripts
    β”œβ”€β”€ tts/                         # Generated audio files
    └── state.db                     # SQLite database

πŸ—οΈ Architecture

Design Patterns

  • Provider Pattern: Pluggable LLM, TTS, and notification providers
  • Command Pattern: CLI commands with consistent interface
  • Manager Pattern: Centralized management of providers and resources
  • Repository Pattern: Database abstraction for state management

Key Components

  1. Configuration Layer (config/manager.py)

    • YAML-based configuration with validation
    • Support for local overrides (config.local.yml)
    • Environment-specific settings
  2. Core Business Logic (core/)

    • Platform-specific implementations (YouTube, Reddit, Twitter)
    • Content extraction and processing
    • Summary generation orchestration
  3. Infrastructure Layer (infrastructure/)

    • LLM providers with unified interface
    • TTS engines for audio generation
    • Notification delivery systems
    • Database and logging utilities
  4. Interface Layer (interfaces/cli/)

    • Command-line interface with subcommands
    • User-friendly command structure
    • Error handling and feedback

πŸ”§ Technical Stack

  • Language: Python 3.8+
  • Configuration: PyYAML
  • Database: SQLite3
  • LLM Integration:
    • llama-cpp-python (local GGUF models)
    • anthropic (Claude API)
    • openai (GPT API)
    • ollama (local/remote Ollama)
  • TTS:
    • kokoro (neural TTS)
    • piper-tts (lightweight TTS)
  • Video Processing: yt-dlp
  • Notifications: requests (Telegram Bot API)
  • Audio: soundfile, numpy, torch

🀝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

About

πŸ“± AI-powered tool to reclaim time wasted on social media by providing intelligent content summarization instead of endless scrolling. Features text-to-speech, smart notifications, and YouTube support.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages