"Your Photos. Their Music."
Ethni-CITY is an AI-powered Creative Storyteller designed to bridge the gap between global travel and local support. By transforming your holiday photos into immersive Sonic Story-Zines, Ethni-CITY bypasses trending chart-toppers to spotlight the hyper-local, niche artists who soundtrack the actual neighborhood in your image.
Important
Hackathon Category: Creative Storyteller (Multimodal Storytelling with Interleaved Output)
Travel is a privilege, but it’s often one-sided. Tourists stay in global hubs, support international brands, and use chart-topping hits for their social media stories. The local music economy—the actual pulse of the destination—is often left behind.
Ethni-CITY changes the narrative. It uses Gemini Multimodal Vision to "read" the culture, architecture, and vibe of your travel photos. It then uses the Agentic DJ (Gemini Live) to curate a playlist of local artists from that exact region, generating a interactive Sonic Story-Zine that you can share to give your trip an authentic, local voice.
- Multimodal Image Analysis: Gemini Vision identifies neighborhoods, cultural markers, and historical context from a single photo.
- The Sonic Zoom (CesiumJS): A high-fidelity 3D Cinematic fly-to experience using Google Photorealistic 3D Tiles to transport you to the photo's location.
- Agentic DJ (Vibe Curation): A bidirectional interaction where you tell the AI "the vibe" you're looking for (e.g., "Afro-soul sunset" or "Balogun Market chaos"), and the DJ finds the perfect niche match.
- Sonic Story-Zine: An interleaved output of generated narrative text, high-quality artist previews (via iTunes/Spotify), and local lore, creating a rich, multi-media story.
- Dual-Key Resilience: A proprietary hierarchical AI logic that switches between Paid and Free API tiers (up to Gemini 3.1 Pro Preview) to ensure the demo never hits a quota limit.
graph TD
User((User)) -->|Uploads Photo| FE[Next.js Frontend]
FE -->|API Request| BE[Next.js API Routes]
subgraph "AI Core (Gemini)"
BE -->|Hierarchical Request| GAI[runSmartGemini Logic]
GAI -->|Primary Key| G31[Gemini 3.1 Pro Preview]
GAI -->|Fallback Tier| G20[Gemini 2.0 Flash]
GAI -->|Emergency Tier| G15[Gemini 1.5 Flash-8b]
end
subgraph "Data & Spatial Layer"
BE -->|Reverse Geocoding| GMA[Google Maps API]
BE -->|Artist/Audio Discovery| ITN[iTunes Search API]
FE -->|3D Spatial Rendering| CES[CesiumJS + Google 3D Tiles]
end
BE -->|Zine Schema| Store[Zustand State Engine]
Store -->|Real-time UI| FE
- Framework: Next.js 15 (Turbopack)
- AI Engine: Google Generative AI SDK (Gemini 1.5 Pro & 2.0 Flash)
- Spatial: CesiumJS (Resium) + Google Cloud Photorealistic 3D Tiles
- APIs:
- Google Cloud: Vertex AI / Generative AI SDK, Maps Geocoding API.
- Multimedia: iTunes Search API (for preview discovery).
- State Management: Zustand
- Styling: Neo-Brutalism (Custom CSS + Tailwind)
- Node.js 20+
- A Google Cloud API Key (Generative AI & Maps enabled)
- A Cesium Ion Token (Free tier available at ion.cesium.com)
git clone https://github.com/david-ac1/Ethni-CITY.git
cd Ethni-citynpm installCreate a .env.local file in the root:
# Gemini API Keys
GOOGLE_GENERATIVE_AI_API_KEY_PAID=your_paid_key (optional)
GOOGLE_GENERATIVE_AI_API_KEY_FREE=your_free_key
# Google Maps API (Geocoding)
GOOGLE_MAPS_API_KEY=your_maps_key
# Cesium Token (for 3D Tiles)
NEXT_PUBLIC_CESIUM_ION_TOKEN=your_cesium_tokennpm run devThe app will be available at http://localhost:3000.
Follow these steps to reproduce the full Ethni-CITY experience:
- Upload a Photo: Drag and drop a travel photo (ideally from the Global South: Brazil, Nigeria, Thailand, etc.) onto the central map "porthole" or use the Upload Photo button in the navbar.
- Observe Vision Grounding: Watch the 3D globe (Cesium) perform a Sonic Zoom to the exact location. Verify technical alignment in the "Your Raw Trips" panel where Gemini Vision outputs the identified location and cultural metadata.
- Curate the Vibe: Click "Set the Vibe" to interact with the Agentic DJ. Type a creative prompt (e.g., "Dark tropical bass" or "Acoustic morning in Lagos").
- Generate the Zine: Click Process to Zine. This triggers the interleaved storytelling engine.
- Verify Interleaved Output: Once the zine is generated, verify that it seamlessly combines:
- Narrative Lore: Historical context of the location.
- Visuals: Your original photo and the spatial context.
- Audio: Streaming song previews of niche local artists discovered by the DJ.
- Multimodal Grounding: We found that Gemini's ability to identify specific architecture styles (e.g., Brazilian brutalism vs. Nigerian colonial) allowed us to match music with unprecedented accuracy.
- Hierarchical Fallbacks: Implementing a multi-tier API model (Phase 9) was crucial for hackathon stability, allowing the app to stay live even when standard free-tier quotas were hit.
- Spatial Narrative: Integrating CesiumJS with AI narration creates a "Spatial Storytelling" effect that makes digital travel feel physically grounded.
This project leverages the Google Cloud AI ecosystem as its core engine. Per the hackathon requirements, the following file serves as proof of our integration with Google Cloud services and APIs:
- lib/gemini.ts: This file demonstrates the use of the
@google/generative-aiSDK to communicate with Vertex AI-compatible Gemini endpoints (gemini-3.1-pro-preview,gemini-2.0-flash, andgemini-1.5-flash), including our custom hierarchical resilience logic.
Furthermore, the project utilizes:
- Google Maps Geocoding API: For cultural/neighborhood spatial grounding.
- Google Cloud Photorealistic 3D Tiles: Served via CesiumJS for high-fidelity spatial storytelling.
Ethni-CITY adheres to the Google Cloud Acceptable Use Policy. All content is generated responsibly, and all source artists are credited and linked via official preview APIs.
Ethni-CITY: Promote the unseen. Sound track the world. 🌍✨