Add dockerfiles and cloudrun deployment yaml

DEPLOYMENT.md

@@ -0,0 +1,133 @@
+# Cloud Run Deployment Guide
+
+This guide will help you deploy the webpage replicator application to Google Cloud Run.
+
+## Prerequisites
+
+1. **Google Cloud Account**: You need a Google Cloud account with billing enabled
+2. **Google Cloud CLI**: Install the `gcloud` CLI tool
+3. **Docker**: Ensure Docker is installed and running (for local testing)
+4. **Project Setup**: Create a Google Cloud project
+
+## Quick Deployment
+
+### Option 1: Using the deployment script (Recommended)
+
+1. **Update the deployment script**:
+   ```bash
+   # Edit deploy.sh and replace 'your-project-id' with your actual project ID
+   nano deploy.sh
+   ```
+
+2. **Make the script executable and run it**:
+   ```bash
+   chmod +x deploy.sh
+   ./deploy.sh
+   ```
+
+### Option 2: Manual deployment using YAML files
+
+1. **Set your project ID**:
+   ```bash
+   export PROJECT_ID="your-project-id"
+   gcloud config set project $PROJECT_ID
+   ```
+
+2. **Enable required APIs**:
+   ```bash
+   gcloud services enable cloudbuild.googleapis.com
+   gcloud services enable run.googleapis.com
+   ```
+
+3. **Update YAML files**:
+   - Replace `PROJECT_ID` in both `frontend/cloudrun.yaml` and `backend/cloudrun.yaml` with your actual project ID
+
+4. **Build and deploy backend**:
+   ```bash
+   cd backend
+   gcloud builds submit --tag gcr.io/$PROJECT_ID/webpage-replicator-backend
+   gcloud run services replace cloudrun.yaml --region=us-central1
+   cd ..
+   ```
+
+5. **Build and deploy frontend**:
+   ```bash
+   cd frontend
+   gcloud builds submit --tag gcr.io/$PROJECT_ID/webpage-replicator-frontend
+   gcloud run services replace cloudrun.yaml --region=us-central1
+   cd ..
+   ```
+
+## Configuration
+
+### Backend Configuration
+
+The backend service may require environment variables for API keys and other configuration. You can set these using:
+
+```bash
+gcloud run services update webpage-replicator-backend \
+    --region=us-central1 \
+    --set-env-vars="GEMINI_API_KEY=your-api-key-here"
+```
+
+Or use Google Secret Manager for sensitive data:
+
+```bash
+# Create a secret
+gcloud secrets create gemini-api-key --data-file=api-key.txt
+
+# Update the service to use the secret
+gcloud run services update webpage-replicator-backend \
+    --region=us-central1 \
+    --set-secrets="GEMINI_API_KEY=gemini-api-key:latest"
+```
+
+### Frontend Configuration
+
+If your frontend needs to communicate with the backend, update any API endpoint URLs in your frontend code to use the deployed backend URL.
+
+## Monitoring and Logs
+
+- **View logs**: `gcloud run logs read webpage-replicator-backend --region=us-central1`
+- **Monitor metrics**: Visit the Cloud Console > Cloud Run to view metrics and performance
+
+## Costs
+
+Cloud Run pricing is based on:
+- CPU and memory allocation
+- Number of requests
+- Request duration
+
+The current configuration uses:
+- **Frontend**: 1 vCPU, 512Mi memory
+- **Backend**: 2 vCPU, 1Gi memory
+
+Both services scale to zero when not in use, so you only pay for actual usage.
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Build failures**: Check that all dependencies are properly listed in `package.json`
+2. **Port issues**: Ensure your application listens on the port specified in the `PORT` environment variable
+3. **Health check failures**: Make sure your backend has a `/health` endpoint or update the health check path
+
+### Useful Commands
+
+```bash
+# View service details
+gcloud run services describe webpage-replicator-backend --region=us-central1
+
+# View recent deployments
+gcloud run revisions list --service=webpage-replicator-backend --region=us-central1
+
+# Delete a service
+gcloud run services delete webpage-replicator-backend --region=us-central1
+```
+
+## Security Considerations
+
+- Both services are currently configured to allow unauthenticated access
+- For production, consider implementing authentication
+- Use IAM roles to control access to your services
+- Store sensitive configuration in Google Secret Manager

backend/.dockerignore

@@ -0,0 +1,14 @@
+node_modules
+npm-debug.log
+.git
+.gitignore
+README.md
+.env
+.env.local
+.env.development.local
+.env.test.local
+.env.production.local
+.DS_Store
+*.log
+uploads/
+temp/

backend/Dockerfile

@@ -0,0 +1,20 @@
+# Use Node.js official image
+FROM node:18-alpine
+
+# Set working directory
+WORKDIR /app
+
+# Copy package files
+COPY package*.json ./
+
+# Install dependencies
+RUN npm ci --only=production
+
+# Copy application files
+COPY . .
+
+# Expose port (assuming Express server runs on port 3001 or process.env.PORT)
+EXPOSE 3001
+
+# Start the application
+CMD ["npm", "start"]

backend/cloudrun.yaml

@@ -0,0 +1,57 @@
+apiVersion: serving.knative.dev/v1
+kind: Service
+metadata:
+  name: webpage-replicator-backend
+  annotations:
+    run.googleapis.com/ingress: all
+    run.googleapis.com/ingress-status: all
+spec:
+  template:
+    metadata:
+      annotations:
+        autoscaling.knative.dev/maxScale: "100"
+        run.googleapis.com/cpu-throttling: "false"
+        run.googleapis.com/execution-environment: gen2
+    spec:
+      containerConcurrency: 80
+      timeoutSeconds: 300
+      containers:
+      - image: gcr.io/PROJECT_ID/webpage-replicator-backend:latest
+        ports:
+        - name: http1
+          containerPort: 3001
+        env:
+        - name: PORT
+          value: "3001"
+        - name: NODE_ENV
+          value: "production"
+        # Add your environment variables here
+        # - name: GEMINI_API_KEY
+        #   valueFrom:
+        #     secretKeyRef:
+        #       name: gemini-secrets
+        #       key: api-key
+        resources:
+          limits:
+            cpu: 2000m
+            memory: 1Gi
+          requests:
+            cpu: 200m
+            memory: 256Mi
+        livenessProbe:
+          httpGet:
+            path: /health
+            port: 3001
+          initialDelaySeconds: 30
+          periodSeconds: 30
+          failureThreshold: 3
+        readinessProbe:
+          httpGet:
+            path: /health
+            port: 3001
+          initialDelaySeconds: 5
+          periodSeconds: 10
+          failureThreshold: 3
+  traffic:
+  - percent: 100
+    latestRevision: true

backend/gcs-service.js

@@ -0,0 +1,157 @@
+import { Storage } from '@google-cloud/storage';
+
+class GCSService {
+  constructor() {
+    this.storage = new Storage();
+    this.bucketName = process.env.GCS_BUCKET_NAME;
+
+    if (!this.bucketName) {
+      throw new Error('GCS_BUCKET_NAME environment variable is required');
+    }
+
+    this.bucket = this.storage.bucket(this.bucketName);
+  }
+
+  /**
+   * Upload a file to GCS
+   * @param {string} fileName - The name/path of the file in the bucket
+   * @param {Buffer} fileBuffer - The file content as a buffer
+   * @param {string} contentType - The MIME type of the file
+   * @returns {Promise<string>} - The public URL of the uploaded file
+   */
+  async uploadFile(fileName, fileBuffer, contentType = 'application/octet-stream') {
+    try {
+      const file = this.bucket.file(fileName);
+
+      const stream = file.createWriteStream({
+        metadata: {
+          contentType: contentType,
+        },
+        resumable: false,
+      });
+
+      return new Promise((resolve, reject) => {
+        stream.on('error', (error) => {
+          console.error('Error uploading to GCS:', error);
+          reject(error);
+        });
+
+        stream.on('finish', () => {
+          // Make the file publicly readable
+          file.makePublic().then(() => {
+            const publicUrl = `https://storage.googleapis.com/${this.bucketName}/${fileName}`;
+            resolve(publicUrl);
+          }).catch(reject);
+        });
+
+        stream.end(fileBuffer);
+      });
+    } catch (error) {
+      console.error('Error in uploadFile:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Download a file from GCS
+   * @param {string} fileName - The name/path of the file in the bucket
+   * @returns {Promise<Buffer>} - The file content as a buffer
+   */
+  async downloadFile(fileName) {
+    try {
+      const file = this.bucket.file(fileName);
+      const [fileBuffer] = await file.download();
+      return fileBuffer;
+    } catch (error) {
+      console.error('Error downloading from GCS:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Check if a file exists in GCS
+   * @param {string} fileName - The name/path of the file in the bucket
+   * @returns {Promise<boolean>} - Whether the file exists
+   */
+  async fileExists(fileName) {
+    try {
+      const file = this.bucket.file(fileName);
+      const [exists] = await file.exists();
+      return exists;
+    } catch (error) {
+      console.error('Error checking file existence:', error);
+      return false;
+    }
+  }
+
+  /**
+   * List files in a directory (prefix)
+   * @param {string} prefix - The directory prefix to list
+   * @returns {Promise<Array>} - Array of file objects
+   */
+  async listFiles(prefix = '') {
+    try {
+      const [files] = await this.bucket.getFiles({
+        prefix: prefix,
+      });
+
+      return files.map(file => ({
+        name: file.name,
+        size: file.metadata.size,
+        created: file.metadata.timeCreated,
+        contentType: file.metadata.contentType,
+        publicUrl: `https://storage.googleapis.com/${this.bucketName}/${file.name}`
+      }));
+    } catch (error) {
+      console.error('Error listing files:', error);
+      throw error;
+    }
+  }
+
+  /**
+   * Delete a file from GCS
+   * @param {string} fileName - The name/path of the file in the bucket
+   * @returns {Promise<boolean>} - Whether the deletion was successful
+   */
+  async deleteFile(fileName) {
+    try {
+      const file = this.bucket.file(fileName);
+      await file.delete();
+      return true;
+    } catch (error) {
+      console.error('Error deleting file:', error);
+      return false;
+    }
+  }
+
+  /**
+   * Get a public URL for a file
+   * @param {string} fileName - The name/path of the file in the bucket
+   * @returns {string} - The public URL
+   */
+  getPublicUrl(fileName) {
+    return `https://storage.googleapis.com/${this.bucketName}/${fileName}`;
+  }
+
+  /**
+   * Generate a signed URL for temporary access
+   * @param {string} fileName - The name/path of the file in the bucket
+   * @param {number} expiresInMinutes - Expiration time in minutes (default: 60)
+   * @returns {Promise<string>} - The signed URL
+   */
+  async getSignedUrl(fileName, expiresInMinutes = 60) {
+    try {
+      const file = this.bucket.file(fileName);
+      const [signedUrl] = await file.getSignedUrl({
+        action: 'read',
+        expires: Date.now() + (expiresInMinutes * 60 * 1000),
+      });
+      return signedUrl;
+    } catch (error) {
+      console.error('Error generating signed URL:', error);
+      throw error;
+    }
+  }
+}
+
+export default GCSService;

backend/package.json

@@ -9,6 +9,7 @@
     "dev": "node --watch server.js"
   },
   "dependencies": {
+    "@google-cloud/storage": "^7.13.0",
     "@google/genai": "^1.15.0",
     "cors": "^2.8.5",
     "dotenv": "^17.2.1",