Implement Per-User Rate Limiting Middleware in the Core Service

[Vedant1703]

📋 Description

The Core Service has no rate limiting. Every protected endpoint (recommendations, repo search, setup guide) is guarded by JWTMiddleware, which correctly enforces authentication — but any authenticated user can send unlimited requests, which is a real problem because:

1. Each request may fan out to the GitHub API and/or the AI/LLM service
2. A misbehaving client or a developer running a test loop can trigger GitHub's secondary rate limits and get the service's shared token blocked
3. There's no fairness guarantee — one user can starve everyone else

The fix is to add a per-user token bucket middleware. Each user gets a bucket of N tokens that refills at rate R per second. Each request costs 1 token. If the bucket is empty, the request is rejected with HTTP 429 Too Many Requests.

This is a real, production-grade Go task that requires understanding of concurrency, goroutines, sync.Mutex, and HTTP middleware chaining.

---

📍 Files to Create / Modify

• Create: backend/core_service/internal/ratelimit/limiter.go
• Modify: backend/core_service/routes/routes.go — wrap protected routes with the new middleware

---

🔍 How Token Bucket Works

Each user has a "bucket" with capacity N tokens.
Tokens refill at rate R per second.
Each request consumes 1 token.
If the bucket is empty → reject the request → HTTP 429.

Example: capacity=10, refillRate=2 per second means a user can burst 10 requests then is limited to 2/s.

---

✅ What To Build

Step 1 — Create backend/core_service/internal/ratelimit/limiter.go:

package ratelimit

import (
    "net/http"
    "sync"
    "time"
)

type bucket struct {
    tokens    float64
    capacity  float64
    refillRate float64 // tokens per second
    lastRefill time.Time
    mu          sync.Mutex
}

func (b *bucket) allow() bool {
    b.mu.Lock()
    defer b.mu.Unlock()

    now := time.Now()
    elapsed := now.Sub(b.lastRefill).Seconds()
    b.lastRefill = now

    // Refill tokens based on elapsed time
    b.tokens += elapsed * b.refillRate
    if b.tokens > b.capacity {
        b.tokens = b.capacity
    }

    if b.tokens >= 1.0 {
        b.tokens--
        return true
    }
    return false
}

// Limiter holds per-user buckets
type Limiter struct {
    mu       sync.RWMutex
    buckets  map[string]*bucket
    capacity float64
    rate     float64
}

func NewLimiter(capacity float64, ratePerSecond float64) *Limiter {
    return &Limiter{
        buckets:  make(map[string]*bucket),
        capacity: capacity,
        rate:     ratePerSecond,
    }
}

func (l *Limiter) getBucket(userID string) *bucket {
    // Fast path: read lock
    l.mu.RLock()
    b, ok := l.buckets[userID]
    l.mu.RUnlock()
    if ok {
        return b
    }

    // Slow path: write lock to create new bucket
    l.mu.Lock()
    defer l.mu.Unlock()
    // Double-check after acquiring write lock
    if b, ok = l.buckets[userID]; ok {
        return b
    }
    b = &bucket{
        tokens:     l.capacity,
        capacity:   l.capacity,
        refillRate: l.rate,
        lastRefill: time.Now(),
    }
    l.buckets[userID] = b
    return b
}

// Middleware wraps an http.Handler with per-user rate limiting.
// It reads the authenticated user ID from the request context
// (set by JWTMiddleware using the "user_id" key).
func (l *Limiter) Middleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        userID, ok := r.Context().Value(contextKey("user_id")).(string)
        if !ok || userID == "" {
            // Should not happen if JWTMiddleware ran first, but be safe
            http.Error(w, "unauthorized", http.StatusUnauthorized)
            return
        }

        b := l.getBucket(userID)
        if !b.allow() {
            w.Header().Set("Retry-After", "1")
            http.Error(w, `{"error":"rate limit exceeded, please slow down"}`, http.StatusTooManyRequests)
            return
        }
        next.ServeHTTP(w, r)
    })
}

// Must match the key type in auth/middleware.go
type contextKey string

Step 2 — Wire it into routes/routes.go:

// In the route setup function, after creating JWTMiddleware:
limiter := ratelimit.NewLimiter(10, 2)  // 10 burst, 2 req/sec sustained

// Example: wrap the recommendations handler
mux.Handle("/recommendations",
    auth.JWTMiddleware(jwtSecret,
        limiter.Middleware(
            http.HandlerFunc(recHandler.GetRecommendation),
        ),
    ),
)

The middleware chain is: JWTMiddleware → RateLimitMiddleware → Handler. This order matters — rate limiting only makes sense for authenticated users (known by user ID).

Step 3 (Bonus) — Add periodic cleanup to avoid unbounded memory growth:

If the service runs for weeks, the buckets map grows forever (one entry per unique user). Add a cleanup goroutine that removes buckets for users who haven't made a request in a configurable duration (e.g., 1 hour).

---

🏁 Acceptance Criteria

• A Limiter struct exists in ratelimit/limiter.go using the token bucket algorithm
• The bucket.allow() method is correct: refills on elapsed time, caps at capacity, returns false when empty
• Per-user state is stored safely — bucket creation and token consumption are both protected by mutexes
• The middleware returns HTTP 429 with a Retry-After: 1 header when the limit is exceeded
• The middleware reads user_id from the request context (set by JWTMiddleware)
• At least the /recommendations route is wrapped with the rate limiter
• go test -race ./internal/ratelimit/... passes with zero data races

---

💡 Technical Hints

• Look at backend/core_service/internal/auth/middleware.go for how user_id is stored in context using context.WithValue. Your middleware reads that same key. Be careful: you must use the same contextKey type, otherwise r.Context().Value("user_id") will return nil because Go context keys are compared by both value AND type.
• The double-checked locking pattern in getBucket (check RLock → check WLock) is important for concurrent correctness and performance. Without it, acquiring a write lock on every request would be a bottleneck.
• Test your rate limiter by writing a TestRateLimiter in limiter_test.go that fires 15 requests rapidly and asserts that the first 10 succeed and the next 5 are rejected.

---

🚀 Getting Started

1. Fork the repository
2. Create a branch: git checkout -b feat/issue-29-rate-limiting
3. Create backend/core_service/internal/ratelimit/limiter.go
4. Write tests in backend/core_service/internal/ratelimit/limiter_test.go
5. Wire into routes/routes.go
6. Run: cd backend/core_service && go test -race ./...
7. Open a PR with test output showing the rate limiter working!