diff --git a/.cursor/rules/task.mdc b/.cursor/rules/task.mdc index f0517dcc..f3aefb0d 100644 --- a/.cursor/rules/task.mdc +++ b/.cursor/rules/task.mdc @@ -3,5 +3,5 @@ description: Current Task for the Agent globs: alwaysApply: true --- -# Feature Request: -We are reworking and cleaning up our ui architecture. Please utilize our best practices and style guidelines. \ No newline at end of file +# Task +Insert the task for the agent. \ No newline at end of file diff --git a/doc/alpha/alpha.md b/doc/alpha/alpha.md new file mode 100644 index 00000000..d89727ff --- /dev/null +++ b/doc/alpha/alpha.md @@ -0,0 +1 @@ +Stub entry for alpha testing. \ No newline at end of file diff --git a/doc/concepts/perspectives.rst b/doc/concepts/perspectives.rst index 6d7c5dc7..07ada047 100644 --- a/doc/concepts/perspectives.rst +++ b/doc/concepts/perspectives.rst @@ -12,20 +12,45 @@ At its core, a perspective is about asking specific questions about an image. Fo - What objects and relationships can we see? (Graph Caption) - How does the composition work? (Art Critic) - What feelings does it evoke? (Emotional Sentiment) -- What story does it tell? (Temporarium) +- What story does it tell? (Storytelling) +- What poetic metaphors might arise? (Poetic Metaphor) +- How does this image relate to time? (Temporarium) Each perspective has its own: -- Focus (what it looks for) -- Language (how it describes things) -- Structure (how it organizes information) -- Balance between describing what's visible and what it means + +- **Focus**: what it looks for in the image +- **Language**: how it describes what it sees +- **Structure**: how it organizes information +- **Balance**: between describing what's visible and interpreting meaning +- **Module**: which family of perspectives it belongs to +- **Tags**: categories that help organize and find perspectives + +The Perspective Ecosystem +======================== + +Perspectives in GraphCap are organized into modules that group related perspectives together. This organization makes it easier to: + +- Find perspectives relevant to your interests +- Enable or disable entire families of perspectives +- Understand relationships between similar perspectives + +Examples of modules include: + +- **Core**: Essential perspectives like Graph Caption and Custom Caption +- **Artistic**: Art Critic, Poetic Metaphor, and other artistic interpretations +- **Narrative**: Storytelling and related perspectives +- **Technical**: Specialized analytical perspectives +- **Synthesizer**: Perspectives that combine multiple captions into a focused output. Built-in Perspectives =================== +GraphCap comes with a diverse set of built-in perspectives, each designed for specific use cases: + Graph Caption ------------ -The "just the facts" perspective. It looks at what's actually in the image: +The "just the facts" perspective that captures objective elements: + - Objects and their relationships - Clear, verifiable descriptions - Confidence scores for each observation @@ -36,7 +61,8 @@ Example output: Art Critic --------- -The formal analysis perspective. It examines: +The formal analysis perspective for visual arts: + - Composition and framing - Color relationships - Technical execution @@ -47,7 +73,8 @@ Example output: Emotional Sentiment ----------------- -The feeling-focused perspective. It considers: +The feeling-focused perspective: + - Mood and atmosphere - Emotional impact - Human elements @@ -56,16 +83,40 @@ The feeling-focused perspective. It considers: Example output: "A serene moment capturing the quiet joy of a peaceful afternoon" -Temporarium ----------- -A temporal contextperspective. It explores: -- Historical or cultural context -- Potential narratives -- Broader implications -- Time-based elements -Example output: -"A snapshot of urban life in transition, where modern architecture meets historical preservation" +Working with Perspectives +======================= + +Discovering and Selecting +------------------------ +GraphCap offers an intuitive way to browse and select perspectives: + +- Browse by module to find related perspectives +- Filter by tags to find perspectives for specific needs +- Search by name or description +- View detailed descriptions to understand what each perspective offers + +Combining Perspectives +-------------------- +Perspectives work best when they complement each other. You might use: + +- Graph Caption + Art Critic for detailed artwork analysis +- Emotional Sentiment + Temporarium for storytelling +- Multiple perspectives for training data generation + +Local Development and Customization +================================= + +GraphCap allows you to create and test new perspectives locally before sharing them more broadly: + +Perspective Workspace +------------------- +Your perspective library can include both: + +- Standard perspectives from the GraphCap library +- Local perspectives you're developing or customizing + +This separation lets you experiment with new ideas while keeping the main system stable. Creating Your Own Perspective =========================== @@ -73,160 +124,38 @@ Creating Your Own Perspective Before You Start -------------- Ask yourself: + - What unique angle are you trying to capture? - Who will use this perspective and why? - How literal vs. interpretative should it be? - What kind of output will be most useful? +- Which module does it belong to? +- What tags would help users find it? -How to create a perspective +How to Create a Perspective -------------------------- -Define a config file for the perspective. Following these examples: - - - .. code-block:: json - { - "name": "graph_caption", - "display_name": "Graph Caption", - "version": "1", - "prompt": "Analyze this image and provide a structured analysis with the following components:\n\n1. Tags: Generate a list of categorized tags with confidence scores for key elements in the image. Each tag should include the tag name, category, and a confidence score between 0 and 1.\n\n2. Short Caption: Create a concise single-sentence caption (max 100 characters) that summarizes the main content of the image.\n\n3. Verification: Provide a brief verification of the tag accuracy and visual grounding, noting any potential issues or uncertainties.\n\n4. Dense Caption: Create a detailed narrative description that incorporates the tagged elements and provides a comprehensive understanding of the image content.\n\nYour analysis should be objective, detailed, and based solely on what is visible in the image.", - "schema_fields": [ - { - "name": "tags_list", - "type": "str", - "description": "List of categorized tags with confidence scores", - "is_list": true, - "is_complex": true, - "fields": [ - { - "name": "tag", - "type": "str", - "description": "Description of the tagged element" - }, - { - "name": "category", - "type": "str", - "description": "Category the tag belongs to" - }, - { - "name": "confidence", - "type": "float", - "description": "Confidence score between 0 and 1" - } - ] - }, - { - "name": "short_caption", - "type": "str", - "description": "Concise single sentence caption (max 100 chars)", - "is_list": false - }, - { - "name": "verification", - "type": "str", - "description": "Verification of tag accuracy and visual grounding", - "is_list": false - }, - { - "name": "dense_caption", - "type": "str", - "description": "Detailed narrative description incorporating tagged elements", - "is_list": false - } - ], - "table_columns": [ - { - "name": "Category", - "style": "cyan" - }, - { - "name": "Content", - "style": "green" - } - ], - "context_template": "\n{short_caption}\n\nTags: {tags_list}\n\n" - } - - .. code-block:: json - { - "name": "temporarium", - "display_name": "Temporarium", - "version": "1", - "prompt": "You are a temporal analysis agent. Analyze this image with a focus on time-related aspects and temporal dimensions. Your response should include a chain-of-thought reasoning process with the following components:\n\n1. Visual Analysis: Provide observations based solely on visible image details.\n\n2. Epoch Reasoning: Present logical reasoning about the implied historical or futuristic epoch.\n\n3. Epoch Context: Provide a concise summary of the inferred epoch context.\n\n4. Narrative Reasoning: Explain how key elements fit within the epoch context.\n\n5. Narrative Elements: Provide a factual description of key visible subjects or objects, linked to the epoch.\n\n6. Continuity Reasoning: Reason on how the scene connects to known historical trends or plausible futures.\n\n7. Continuity Elements: Provide a brief summary of historical or futuristic continuity.\n\n8. Speculative Reasoning: Present step-by-step reasoning behind any imaginative extrapolation.\n\n9. Temporal Speculation: Provide imaginative yet plausible speculative details derived from reasoning.\n\n10. Detailed Caption: Create a final cohesive caption integrating all chain-of-thought steps.\n\nYour analysis should be thoughtful and consider both explicit and implicit temporal elements in the image.", - "schema_fields": [ - { - "name": "visual_analysis", - "type": "str", - "description": "Observations based solely on visible image details.", - "is_list": false - }, - { - "name": "epoch_reasoning", - "type": "str", - "description": "Logical reasoning about the implied historical or futuristic epoch.", - "is_list": false - }, - { - "name": "epoch_context", - "type": "str", - "description": "Concise summary of the inferred epoch context.", - "is_list": false - }, - { - "name": "narrative_reasoning", - "type": "str", - "description": "Explanation of how key elements fit within the epoch context.", - "is_list": false - }, - { - "name": "narrative_elements", - "type": "str", - "description": "Factual description of key visible subjects or objects, linked to the epoch.", - "is_list": false - }, - { - "name": "continuity_reasoning", - "type": "str", - "description": "Reasoning on how the scene connects to known historical trends or plausible futures.", - "is_list": false - }, - { - "name": "continuity_elements", - "type": "str", - "description": "Brief summary of historical or futuristic continuity.", - "is_list": false - }, - { - "name": "speculative_reasoning", - "type": "str", - "description": "Step-by-step reasoning behind any imaginative extrapolation.", - "is_list": false - }, - { - "name": "temporal_speculation", - "type": "str", - "description": "Imaginative yet plausible speculative details derived from reasoning.", - "is_list": false - }, - { - "name": "detailed_caption", - "type": "str", - "description": "Final cohesive caption integrating all chain-of-thought steps.", - "is_list": false - } - ], - "table_columns": [ - { - "name": "Component", - "style": "cyan" - }, - { - "name": "Content", - "style": "green" - } - ], - "context_template": "\n{detailed_caption}\n\n" - } +Every perspective is defined by: + +1. **Basic Information**: + - Name and display name + - Version + - Description + - Module assignment + - Tags for categorization + - Priority level + +2. **Prompt**: + Clear instructions for how to analyze the image + +3. **Schema**: + The structured fields that will contain the analysis + +4. **Presentation**: + How the results will be displayed + +5. **Context Template**: + How the perspective's output can be used in broader contexts Tips for Good Perspectives ======================== @@ -246,20 +175,30 @@ Quality Matters - Get feedback from potential users - Have clear ways to measure success -Make It Useful ------------- -- Write clear documentation -- Include examples -- Make it easy to understand when to use this perspective -- Consider how it fits with other perspectives +Make It Discoverable +------------------ +- Place it in the appropriate module +- Use descriptive tags +- Write a clear, concise description +- Consider including example outputs in the description + +Evolution and Deprecation +----------------------- +As your needs evolve, perspectives can too: + +- Update existing perspectives with new versions +- Mark outdated perspectives as deprecated +- Suggest replacement perspectives when deprecating old ones Real-World Usage ============== -Perspectives work best when they complement each other. You might use: +GraphCap perspectives are designed to be useful in real-world applications: -- Graph Caption + Art Critic for detailed artwork analysis -- Emotional Sentiment + Temporarium for storytelling -- Multiple perspectives for training data generation +- **Content Creation**: Generate rich, varied descriptions for creative projects +- **Accessibility**: Provide detailed image descriptions for visually impaired users +- **Data Analysis**: Extract structured information from visual content +- **Education**: Teach different ways of seeing and analyzing visual material +- **Creative Inspiration**: Generate diverse interpretations to spark new ideas Remember: The goal isn't to replace human understanding, but to provide useful, structured ways of describing and analyzing images for different purposes. diff --git a/doc/index.rst b/doc/index.rst index 0926794c..59852360 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -4,6 +4,37 @@ GraphCap Welcome to GraphCap's documentation! +Alpha Testing +============ + +## GraphCap Project Overview for Alpha Testing + +**What is GraphCap?** + +GraphCap is an open-source, distributed captioning application designed under the Open Model Initiative. Its primary purpose is to generate detailed, insightful captions and analyses of images by leveraging multiple analytical perspectives. +The application utilizes directed acyclic graph structures to capture complex relationships within images, facilitating diverse, context-rich interpretations. + +### Core Features +- **Multi-Perspective Captioning:** GraphCap applies specialized analytical perspectives—such as formal artistic critique, emotional sentiment, storytelling, and temporal analysis—to generate comprehensive captions. +- **Distributed Processing:** Designed to operate efficiently in distributed environments, allowing for scalable, community-based computational resources. +- **Model Flexibility:** Supports integration with multiple Vision-Language Models (VLMs), enabling comparative analysis and ensuring adaptability across varied captioning tasks. +- **OMI-Compatible:** GraphCap is designed to be compatible with the Open Model Initiative data repository, allowing for easy integration with our open source image dataset. + +### Alpha Testing Goals +- Evaluate system stability and performance across diverse hardware and software environments. +- Collect feedback on caption accuracy, perspective usefulness, and overall usability. +- Identify critical bugs and areas for improvement in functionality and user experience. + +### Participating in Alpha Testing +Participants in the alpha test will: +- Test the application in their local or preferred computing environment. +- Provide structured feedback via surveys and discussions. +- Engage collaboratively in community discussions to shape future GraphCap development. + +Your insights and experiences during this alpha phase will directly contribute to refining GraphCap's capabilities and guiding its future development within the Open Model Initiative community. + + + Getting Started ============== diff --git a/doc/static/generate_perspective.png b/doc/static/generate_perspective.png new file mode 100644 index 00000000..7d62a9a8 Binary files /dev/null and b/doc/static/generate_perspective.png differ diff --git a/graphcap_studio/src/app/layout/RootLeftActionPanel.tsx b/graphcap_studio/src/app/layout/RootLeftActionPanel.tsx index 9e163080..bc6948c4 100644 --- a/graphcap_studio/src/app/layout/RootLeftActionPanel.tsx +++ b/graphcap_studio/src/app/layout/RootLeftActionPanel.tsx @@ -7,8 +7,8 @@ import { DatasetIcon, - FilterIcon, FlagIcon, + PerspectiveLayersIcon, ProviderIcon, SettingsIcon, } from "@/components/icons"; @@ -16,7 +16,7 @@ import { SettingsPanel } from "@/features/app-settings"; import { FeatureFlagsPanel } from "@/features/app-settings/feature-flags"; import { DatasetPanel } from "@/features/datasets"; import { ProvidersPanel } from "@/features/inference/providers"; -import { PerspectiveFilterPanel } from "@/features/perspectives/components/PerspectiveFilterPanel"; +import { PerspectiveManagementPanel } from "@/features/perspectives/components/PerspectiveManagement/PerspectiveManagementPanel"; import { ActionPanel } from "./ActionPanel"; /** @@ -48,10 +48,10 @@ export function RootLeftActionPanel() { content: , }, { - id: "perspectives", - title: "Perspectives", - icon: , - content: , + id: "perspective-management", + title: "Perspective Management", + icon: , + content: , }, { id: "settings", diff --git a/graphcap_studio/src/components/icons/PerspectiveLayersIcon.tsx b/graphcap_studio/src/components/icons/PerspectiveLayersIcon.tsx new file mode 100644 index 00000000..11557f0f --- /dev/null +++ b/graphcap_studio/src/components/icons/PerspectiveLayersIcon.tsx @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: Apache-2.0 + +/** + * PerspectiveLayersIcon component + * + * An icon representing perspectives with eye and layers + */ +interface IconProps { + className?: string; +} + +export function PerspectiveLayersIcon({ className = "" }: Readonly) { + return ( + + Perspectives + + + + + + + + + + ); +} diff --git a/graphcap_studio/src/components/icons/index.tsx b/graphcap_studio/src/components/icons/index.tsx index e2081567..9a37668f 100644 --- a/graphcap_studio/src/components/icons/index.tsx +++ b/graphcap_studio/src/components/icons/index.tsx @@ -21,9 +21,11 @@ export function FlagIcon({ className = "" }: Readonly) { className={className} width="20" height="20" + aria-labelledby="flagIconTitle" > - - + Flag + + ); } @@ -44,8 +46,10 @@ export function FilterIcon({ className = "" }: Readonly) { className={className} width="20" height="20" + aria-labelledby="filterIconTitle" > - + Filter + ); } @@ -66,11 +70,13 @@ export function ServerIcon({ className = "" }: Readonly) { className={className} width="20" height="20" + aria-labelledby="serverIconTitle" > - - - - + Server + + + + ); } @@ -91,8 +97,10 @@ export function FolderIcon({ className = "" }: Readonly) { className={className} width="20" height="20" + aria-labelledby="folderIconTitle" > - + Folder + ); } @@ -113,9 +121,11 @@ export function SettingsIcon({ className = "" }: Readonly) { className={className} width="20" height="20" + aria-labelledby="settingsIconTitle" > - - + Settings + + ); } @@ -136,7 +146,9 @@ export function ProviderIcon({ className = "" }: Readonly) { className={className} width="20" height="20" + aria-labelledby="providerIconTitle" > + Provider @@ -163,7 +175,9 @@ export function DatasetIcon({ className = "" }: Readonly) { className={className} width="20" height="20" + aria-labelledby="datasetIconTitle" > + Dataset @@ -172,3 +186,6 @@ export function DatasetIcon({ className = "" }: Readonly) { ); } + +// Export for PerspectiveLayersIcon +export { PerspectiveLayersIcon } from "./PerspectiveLayersIcon"; diff --git a/graphcap_studio/src/features/perspectives/Perspectives.tsx b/graphcap_studio/src/features/perspectives/Perspectives.tsx index d70f750e..18ca7c41 100644 --- a/graphcap_studio/src/features/perspectives/Perspectives.tsx +++ b/graphcap_studio/src/features/perspectives/Perspectives.tsx @@ -12,8 +12,8 @@ import { DEFAULT_OPTIONS, GenerationOptionForm, } from "../inference/generation-options/components/GenerationOptionForm"; -import { EmptyPerspectives } from "./components/EmptyPerspectives"; -import { PerspectivesPager } from "./components/PerspectiveNavigation/PerspectivesPager"; +import { EmptyPerspectives } from "./components/PerspectiveCaption/EmptyPerspectives"; +import { PerspectivesPager } from "./components/PerspectiveCaption/PerspectiveNavigation/PerspectivesPager"; import { PerspectivesErrorState } from "./components/PerspectivesErrorState"; import { usePerspectiveUI, usePerspectivesData } from "./context"; import type { CaptionOptions } from "./types"; diff --git a/graphcap_studio/src/features/perspectives/components/EmptyPerspectives.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/EmptyPerspectives.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/EmptyPerspectives.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/EmptyPerspectives.tsx diff --git a/graphcap_studio/src/features/perspectives/components/ErrorMessage.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/ErrorMessage.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/ErrorMessage.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/ErrorMessage.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveActions/GenerateAllButton.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveActions/GenerateAllButton.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveActions/GenerateAllButton.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveActions/GenerateAllButton.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveActions/PerspectivesFooter.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveActions/PerspectivesFooter.tsx similarity index 98% rename from graphcap_studio/src/features/perspectives/components/PerspectiveActions/PerspectivesFooter.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveActions/PerspectivesFooter.tsx index 8a36b6dd..8435416d 100644 --- a/graphcap_studio/src/features/perspectives/components/PerspectiveActions/PerspectivesFooter.tsx +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveActions/PerspectivesFooter.tsx @@ -16,7 +16,7 @@ import { usePerspectiveUI, usePerspectivesData, } from "@/features/perspectives/context"; -import { CaptionOptions } from "@/features/perspectives/types"; +import type { CaptionOptions } from "@/features/perspectives/types"; import { Box, Button, diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveActions/index.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveActions/index.ts similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveActions/index.ts rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveActions/index.ts diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/MetadataDisplay.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/MetadataDisplay.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/MetadataDisplay.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/MetadataDisplay.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/PerspectiveCardTabbed.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/PerspectiveCardTabbed.tsx similarity index 98% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/PerspectiveCardTabbed.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/PerspectiveCardTabbed.tsx index 9d2986b9..c6453bdc 100644 --- a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/PerspectiveCardTabbed.tsx +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/PerspectiveCardTabbed.tsx @@ -8,7 +8,7 @@ import { useColorModeValue } from "@/components/ui/theme/color-mode"; * This component uses Chakra UI tabs for the tabbed interface. */ import { Box, Card, Stack, Tabs, Text } from "@chakra-ui/react"; -import { PerspectiveSchema } from "../../types"; +import type { PerspectiveSchema } from "../../../types"; import { PerspectiveDebug } from "./PerspectiveDebug"; import { SchemaView } from "./SchemaView"; import { CaptionRenderer } from "./schema-fields"; diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/PerspectiveDebug.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/PerspectiveDebug.tsx similarity index 98% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/PerspectiveDebug.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/PerspectiveDebug.tsx index 6a59b433..15f54073 100644 --- a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/PerspectiveDebug.tsx +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/PerspectiveDebug.tsx @@ -8,7 +8,7 @@ import { Box, Stack } from "@chakra-ui/react"; * including its data, options, and metadata. */ import { useEffect } from "react"; -import { PerspectiveData, PerspectiveSchema } from "../../types"; +import type { PerspectiveData, PerspectiveSchema } from "../../../types"; import { DataStatistics, MetadataSection, diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/SchemaView.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/SchemaView.tsx similarity index 91% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/SchemaView.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/SchemaView.tsx index daf4be9a..34b72a5b 100644 --- a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/SchemaView.tsx +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/SchemaView.tsx @@ -6,7 +6,7 @@ */ import React from "react"; -import { PerspectiveSchema } from "../../types"; +import type { PerspectiveSchema } from "../../../types"; import { SchemaFieldFactory } from "./schema-fields"; interface SchemaViewProps { diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/DataStatistics.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/DataStatistics.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/DataStatistics.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/DataStatistics.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/MetadataSection.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/MetadataSection.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/MetadataSection.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/MetadataSection.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/MissingDataAlert.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/MissingDataAlert.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/MissingDataAlert.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/MissingDataAlert.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/OptionsSection.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/OptionsSection.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/OptionsSection.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/OptionsSection.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/RawDataSection.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/RawDataSection.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/RawDataSection.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/RawDataSection.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/Separator.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/Separator.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/Separator.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/Separator.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/index.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/index.ts similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/debug-fields/index.ts rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/debug-fields/index.ts diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/BaseField.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/BaseField.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/BaseField.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/BaseField.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/CaptionContext.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/CaptionContext.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/CaptionContext.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/CaptionContext.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/CaptionRenderer.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/CaptionRenderer.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/CaptionRenderer.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/CaptionRenderer.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/EdgeField.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/EdgeField.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/EdgeField.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/EdgeField.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/NodeField.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/NodeField.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/NodeField.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/NodeField.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/SchemaFieldFactory.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/SchemaFieldFactory.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/SchemaFieldFactory.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/SchemaFieldFactory.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/TagField.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/TagField.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/TagField.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/TagField.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/formatters.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/formatters.ts similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/formatters.ts rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/formatters.ts diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/index.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/index.ts similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/index.ts rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/index.ts diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/types.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/types.ts similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveCard/schema-fields/types.ts rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveCard/schema-fields/types.ts diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveNavigation/PerspectiveHeader.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveNavigation/PerspectiveHeader.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveNavigation/PerspectiveHeader.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveNavigation/PerspectiveHeader.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveNavigation/PerspectivesPager.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveNavigation/PerspectivesPager.tsx similarity index 100% rename from graphcap_studio/src/features/perspectives/components/PerspectiveNavigation/PerspectivesPager.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveCaption/PerspectiveNavigation/PerspectivesPager.tsx diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/ErrorDisplay.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/ErrorDisplay.tsx new file mode 100644 index 00000000..6345fffe --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/ErrorDisplay.tsx @@ -0,0 +1,23 @@ +import { + Box, + Heading, + Text, +} from "@chakra-ui/react"; + +interface ErrorDisplayProps { + message: string; +} + +/** + * ErrorDisplay component displays an error message + */ +export function ErrorDisplay({ message }: ErrorDisplayProps) { + return ( + + + Error + {message} + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/LoadingDisplay.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/LoadingDisplay.tsx new file mode 100644 index 00000000..b31a60c9 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/LoadingDisplay.tsx @@ -0,0 +1,21 @@ +import { + Box, + Spinner, + Text, +} from "@chakra-ui/react"; + +interface LoadingDisplayProps { + message?: string; +} + +/** + * LoadingDisplay component shows a loading spinner with an optional message + */ +export function LoadingDisplay({ message = "Loading..." }: LoadingDisplayProps) { + return ( + + + {message} + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/NotFound.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/NotFound.tsx new file mode 100644 index 00000000..1d94b7b5 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/NotFound.tsx @@ -0,0 +1,54 @@ +import { + Box, + Button, + Heading, + Text, +} from "@chakra-ui/react"; +import { Link } from "@tanstack/react-router"; + +interface NotFoundProps { + type: "module" | "perspective"; + name: string; + moduleName?: string; +} + +/** + * NotFound component displays a message when a module or perspective is not found + */ +export function NotFound({ type, name, moduleName }: NotFoundProps) { + const getMessage = () => { + if (type === "module") { + return `The module "${name}" could not be found.`; + } + return `Perspective "${name}" not found in module "${moduleName}".`; + }; + + const getBackLink = () => { + if (type === "module") { + return { to: "/perspectives", params: {} }; + } + return { + to: "/perspectives/module/$moduleName", + params: { moduleName: moduleName || "" } + }; + }; + + return ( + + + Not Found + {getMessage()} + + + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/PerspectiveEditor.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/PerspectiveEditor.tsx new file mode 100644 index 00000000..3b69805d --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/PerspectiveEditor.tsx @@ -0,0 +1,84 @@ +import type { Perspective } from "@/features/perspectives/types"; +import { Box, Tabs } from "@chakra-ui/react"; +import { + FieldsTab, + ManagementTab, + PerspectiveHeader, + PromptTab, + SchemaInfoTab +} from "./components"; +import { PerspectiveEditorProvider, usePerspectiveEditor } from "./context/PerspectiveEditorContext"; + +interface PerspectiveEditorProps { + readonly perspective: Perspective; + readonly moduleName: string; + readonly perspectiveList?: Perspective[]; +} + +/** + * PerspectiveEditor component displays detailed information about a perspective + */ +export function PerspectiveEditor({ + perspective, + moduleName, + perspectiveList = [], +}: PerspectiveEditorProps) { + return ( + + + + ); +} + +/** + * PerspectiveEditorContent renders the editor UI using the context + */ +function PerspectiveEditorContent() { + const { colors } = usePerspectiveEditor(); + + return ( + + + + {/* Tabs for different sections */} + + + + Schema + Fields + Prompt + Management + + + + + + + + + + + + + + + + + + + + + + + + ); +} diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/FieldsTab.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/FieldsTab.tsx new file mode 100644 index 00000000..90a50272 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/FieldsTab.tsx @@ -0,0 +1,71 @@ +import { Box, Heading, Text } from "@chakra-ui/react"; +import { usePerspectiveEditor } from "../context/PerspectiveEditorContext"; + +/** + * FieldsTab displays the field definitions for a perspective schema + */ +export function FieldsTab() { + const { perspective, colors, getFieldTypeDisplay } = usePerspectiveEditor(); + + if (!perspective.schema?.schema_fields || perspective.schema.schema_fields.length === 0) { + return No fields defined for this perspective.; + } + + return ( + + + Field Definitions + + + + + + + Name + + + Type + + + Description + + + List + + + Complex + + + + + {perspective.schema.schema_fields.map((field) => ( + + + {field.name} + + + {getFieldTypeDisplay(field.type)} + + + {field.description} + + + {field.is_list ? "Yes" : "No"} + + + {field.is_complex ? "Yes" : "No"} + + + ))} + + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/ManagementTab.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/ManagementTab.tsx new file mode 100644 index 00000000..f47167d3 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/ManagementTab.tsx @@ -0,0 +1,49 @@ +import { Box, Button, Flex, Heading, Stack, Text } from "@chakra-ui/react"; +import { usePerspectiveEditor } from "../context/PerspectiveEditorContext"; +import { TableColumns } from "./TableColumns"; + +/** + * ManagementTab displays the perspective management options + */ +export function ManagementTab() { + const { perspective, colors } = usePerspectiveEditor(); + + return ( + + + Perspective Management + + + This section allows you to edit and manage the perspective configuration. + + + + + {perspective.schema?.table_columns && perspective.schema.table_columns.length > 0 && ( + + )} + + + + Actions + + + + + + + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PerspectiveDescription.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PerspectiveDescription.tsx new file mode 100644 index 00000000..3641d2c8 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PerspectiveDescription.tsx @@ -0,0 +1,27 @@ +import { Box, Heading, Text } from "@chakra-ui/react"; +import { usePerspectiveEditor } from "../context/PerspectiveEditorContext"; + +/** + * PerspectiveDescription displays the description of the perspective + */ +export function PerspectiveDescription() { + const { perspective, colors } = usePerspectiveEditor(); + + return ( + + + Description + + + {perspective.description || "No description available for this perspective."} + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PerspectiveHeader.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PerspectiveHeader.tsx new file mode 100644 index 00000000..c38b994b --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PerspectiveHeader.tsx @@ -0,0 +1,62 @@ +import { Badge, Box, Button, Flex, HStack, Heading, Text } from "@chakra-ui/react"; +import { Link } from "@tanstack/react-router"; +import { LuChevronLeft, LuChevronRight } from "react-icons/lu"; +import { usePerspectiveEditor } from "../context/PerspectiveEditorContext"; + +/** + * PerspectiveHeader displays the perspective title, version, description, and navigation + */ +export function PerspectiveHeader() { + const { perspective, moduleName, colors, onNavigateNext, onNavigatePrevious, hasPrevious, hasNext } = usePerspectiveEditor(); + + return ( + + + + + + {perspective.display_name || perspective.name} + + | Version : {perspective.version} | Module : {moduleName} + + + {perspective.description || "No description available for this perspective."} + + + + + + + + + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PromptTab.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PromptTab.tsx new file mode 100644 index 00000000..d24b1bee --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/PromptTab.tsx @@ -0,0 +1,36 @@ +import { Box, Code, Heading } from "@chakra-ui/react"; +import { usePerspectiveEditor } from "../context/PerspectiveEditorContext"; + +/** + * PromptTab displays the prompt template for the perspective + */ +export function PromptTab() { + const { colors, getPromptContent } = usePerspectiveEditor(); + + return ( + + + Prompt Template + + + + {getPromptContent()} + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/SchemaInfoTab.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/SchemaInfoTab.tsx new file mode 100644 index 00000000..b11c1269 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/SchemaInfoTab.tsx @@ -0,0 +1,46 @@ +import { Box, Code, Heading, Stack, Text } from "@chakra-ui/react"; +import { usePerspectiveEditor } from "../context/PerspectiveEditorContext"; + +/** + * SchemaInfoTab displays the basic schema information for a perspective + */ +export function SchemaInfoTab() { + const { perspective, colors } = usePerspectiveEditor(); + + if (!perspective.schema) { + return No schema information available for this perspective.; + } + + return ( + + + Schema Information + + + + Name: + {perspective.schema.name} + + + Version: + {perspective.schema.version} + + + + Context Template: + + {perspective.schema.context_template || "None"} + + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/TableColumns.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/TableColumns.tsx new file mode 100644 index 00000000..478066e3 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/TableColumns.tsx @@ -0,0 +1,58 @@ +import type { TableColumn } from "@/features/perspectives/types"; +import { Box, Text } from "@chakra-ui/react"; +import { usePerspectiveEditor } from "../context/PerspectiveEditorContext"; + +interface TableColumnsProps { + readonly tableColumns: TableColumn[]; +} + +/** + * TableColumns displays a table of column definitions + */ +export function TableColumns({ tableColumns }: TableColumnsProps) { + const { colors } = usePerspectiveEditor(); + + return ( + + + Table Columns + + + + + + Name + + + Style + + + Description + + + + + {tableColumns.map((column) => ( + + + {column.name} + + + {column.style} + + + {column.description ?? "-"} + + + ))} + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/index.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/index.ts new file mode 100644 index 00000000..dc6a7551 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/components/index.ts @@ -0,0 +1,6 @@ +export * from "./PerspectiveHeader"; +export * from "./SchemaInfoTab"; +export * from "./FieldsTab"; +export * from "./PromptTab"; +export * from "./ManagementTab"; +export * from "./TableColumns"; \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/context/PerspectiveEditorContext.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/context/PerspectiveEditorContext.tsx new file mode 100644 index 00000000..c0bd9872 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/context/PerspectiveEditorContext.tsx @@ -0,0 +1,152 @@ +import { useColorModeValue } from "@/components/ui/theme/color-mode"; +import type { Perspective } from "@/features/perspectives/types"; +import { createContext, useCallback, useContext, useEffect, useMemo, useState } from "react"; + +interface PerspectiveEditorContextType { + // Data + perspective: Perspective; + moduleName: string; + + // Navigation + onNavigateNext: () => void; + onNavigatePrevious: () => void; + hasPrevious: boolean; + hasNext: boolean; + + // Utility functions + getPromptContent: () => string; + getFieldTypeDisplay: (type: string | object) => string; + + // Theme values + colors: { + borderColor: string; + bgColor: string; + headerBgColor: string; + descriptionBgColor: string; + tableHeaderBg: string; + tableBorderColor: string; + }; +} + +const PerspectiveEditorContext = createContext(undefined); + +interface PerspectiveEditorProviderProps { + children: React.ReactNode; + perspective: Perspective; + moduleName: string; + perspectiveList?: Perspective[]; +} + +export function PerspectiveEditorProvider({ + children, + perspective, + moduleName, + perspectiveList = [], +}: PerspectiveEditorProviderProps) { + // State for current perspective + const [currentPerspective, setCurrentPerspective] = useState(perspective); + + // Update current perspective when prop changes + useEffect(() => { + setCurrentPerspective(perspective); + }, [perspective]); + + // Navigation logic + const currentIndex = useMemo(() => { + return perspectiveList.findIndex(p => p.name === currentPerspective.name); + }, [perspectiveList, currentPerspective.name]); + + const hasPrevious = currentIndex > 0; + const hasNext = currentIndex >= 0 && currentIndex < perspectiveList.length - 1; + + const onNavigatePrevious = useCallback(() => { + if (hasPrevious) { + setCurrentPerspective(perspectiveList[currentIndex - 1]); + } + }, [hasPrevious, perspectiveList, currentIndex]); + + const onNavigateNext = useCallback(() => { + if (hasNext) { + setCurrentPerspective(perspectiveList[currentIndex + 1]); + } + }, [hasNext, perspectiveList, currentIndex]); + + // Color mode values + const borderColor = useColorModeValue("gray.200", "gray.700"); + const bgColor = useColorModeValue("white", "gray.800"); + const headerBgColor = useColorModeValue("gray.50", "gray.900"); + const descriptionBgColor = useColorModeValue("blue.50", "gray.700"); + const tableHeaderBg = useColorModeValue("gray.50", "gray.700"); + const tableBorderColor = useColorModeValue("gray.200", "gray.600"); + + const colors = useMemo(() => ({ + borderColor, + bgColor, + headerBgColor, + descriptionBgColor, + tableHeaderBg, + tableBorderColor + }), [borderColor, bgColor, headerBgColor, descriptionBgColor, tableHeaderBg, tableBorderColor]); + + // Format the prompt content to ensure it's a string + const getPromptContent = useMemo(() => { + return (): string => { + const prompt = currentPerspective.schema?.prompt; + if (prompt === undefined || prompt === null) { + return "No prompt template available."; + } + + return typeof prompt === 'string' + ? prompt + : JSON.stringify(prompt, null, 2); + }; + }, [currentPerspective.schema?.prompt]); + + // Format field type to ensure it's displayed as a string + const getFieldTypeDisplay = useMemo(() => { + return (type: string | object): string => { + if (typeof type === 'string') { + return type; + } + return JSON.stringify(type); + }; + }, []); + + const value = useMemo(() => ({ + perspective: currentPerspective, + moduleName, + onNavigateNext, + onNavigatePrevious, + hasPrevious, + hasNext, + getPromptContent, + getFieldTypeDisplay, + colors + }), [ + currentPerspective, + moduleName, + onNavigateNext, + onNavigatePrevious, + hasPrevious, + hasNext, + getPromptContent, + getFieldTypeDisplay, + colors + ]); + + return ( + + {children} + + ); +} + +export function usePerspectiveEditor() { + const context = useContext(PerspectiveEditorContext); + + if (context === undefined) { + throw new Error("usePerspectiveEditor must be used within a PerspectiveEditorProvider"); + } + + return context; +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/index.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/index.ts new file mode 100644 index 00000000..0fb3dd3e --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveEditor/index.ts @@ -0,0 +1,2 @@ +export * from "./PerspectiveEditor"; +export * from "./context/PerspectiveEditorContext"; \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveFilterPanel.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveFilterPanel.tsx similarity index 89% rename from graphcap_studio/src/features/perspectives/components/PerspectiveFilterPanel.tsx rename to graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveFilterPanel.tsx index 5f902e98..177d6f2c 100644 --- a/graphcap_studio/src/features/perspectives/components/PerspectiveFilterPanel.tsx +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveFilterPanel.tsx @@ -16,7 +16,7 @@ import { VStack, } from "@chakra-ui/react"; import { useMemo } from "react"; -import { usePerspectivesData } from "../context/PerspectivesDataContext"; +import { usePerspectivesData } from "../../context/PerspectivesDataContext"; /** * Component for filtering which perspectives are visible in the UI @@ -51,7 +51,7 @@ export function PerspectiveFilterPanel() { - + {perspectives.map((perspective) => ( - perspectives.forEach((p) => togglePerspectiveVisibility(p.name)) - } + onClick={() => { + for (const p of perspectives) { + togglePerspectiveVisibility(p.name); + } + }} > Toggle All diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveManagementPanel.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveManagementPanel.tsx new file mode 100644 index 00000000..b461064e --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveManagementPanel.tsx @@ -0,0 +1,172 @@ +// SPDX-License-Identifier: Apache-2.0 +/** + * Perspective Management Panel + * + * This component displays a panel for managing perspectives and modules + * using an accordion-based UI for better organization and toggleable options. + */ + +import { LoadingSpinner } from "@/components/ui/status/LoadingSpinner"; +import { useColorModeValue } from "@/components/ui/theme"; +import { + Box, + Flex, + Text, +} from "@chakra-ui/react"; +import { useEffect, useState } from "react"; +import { usePerspectivesData } from "../../context/PerspectivesDataContext"; +import { usePerspectiveModules } from "../../hooks"; +import { PerspectiveModuleFilter } from "./PerspectiveModuleFilter"; + +// Local storage key for accordion state +const ACCORDION_STATE_KEY = "graphcap-perspective-accordion-state"; + +/** + * Panel component for perspective management + */ +export function PerspectiveManagementPanel() { + // Use the perspective modules hook to get real data + const { modules, isLoading, error, hasModules } = usePerspectiveModules(); + + // Get perspective visibility control from context + const { + isPerspectiveVisible, + togglePerspectiveVisibility, + hiddenPerspectives, + perspectives, + setAllPerspectivesVisible + } = usePerspectivesData(); + + // State to track which accordion panels are open + const [openPanels, setOpenPanels] = useState>({}); + + // Load accordion state on component mount + useEffect(() => { + try { + const savedState = localStorage.getItem(ACCORDION_STATE_KEY); + if (savedState) { + setOpenPanels(JSON.parse(savedState)); + } else if (modules.length > 0) { + // Default: open all panels + const initialState: Record = {}; + for (const module of modules) { + initialState[module.name] = true; + } + setOpenPanels(initialState); + } + } catch (error) { + console.error("Error loading accordion state:", error); + } + }, [modules]); + + // Save accordion state when it changes + useEffect(() => { + if (Object.keys(openPanels).length > 0) { + try { + localStorage.setItem(ACCORDION_STATE_KEY, JSON.stringify(openPanels)); + } catch (error) { + console.error("Error saving accordion state:", error); + } + } + }, [openPanels]); + + // Toggle accordion panel + const togglePanel = (moduleName: string) => { + setOpenPanels(prev => ({ + ...prev, + [moduleName]: !prev[moduleName] + })); + }; + + // Theme colors + const bgColor = useColorModeValue("white", "#1A202C"); + const textColor = useColorModeValue("gray.800", "white"); + const mutedTextColor = useColorModeValue("gray.500", "gray.400"); + const buttonBorderColor = useColorModeValue("blue.500", "blue.400"); + const buttonColor = useColorModeValue("blue.500", "blue.400"); + const buttonHoverBg = useColorModeValue("blue.50", "blue.900"); + + // Handle loading state + if (isLoading) { + return ( + + + Loading perspectives... + + ); + } + + // Handle error state + if (error) { + return ( + + Error loading perspectives: + {error.message} + + ); + } + + // Count visible perspectives + const totalPerspectives = perspectives.length; + const visibleCount = totalPerspectives - hiddenPerspectives.length; + + return ( + + + Perspective Modules + + {visibleCount} of {totalPerspectives} visible + + + + {!hasModules ? ( + No perspectives found. + ) : ( + + + {modules.map((module) => ( + + ))} + + + {/* Show All button at the bottom */} + + + Show All + + + + )} + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModuleFilter.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModuleFilter.tsx new file mode 100644 index 00000000..808298cc --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModuleFilter.tsx @@ -0,0 +1,292 @@ +// SPDX-License-Identifier: Apache-2.0 +/** + * Perspective Module Filter + * + * This component displays a single module as an accordion item with checkboxes + * for toggling the visibility of perspectives within the module. + */ + +import { useColorModeValue } from "@/components/ui/theme/color-mode"; +import type { Perspective } from "@/features/perspectives/types"; +import { Box, Flex } from "@chakra-ui/react"; +import { Link } from "@tanstack/react-router"; +import { useMemo } from "react"; + +type PerspectiveModuleFilterProps = { + readonly moduleName: string; + readonly displayName: string; + readonly perspectives: Perspective[]; + readonly isOpen: boolean; + readonly onTogglePanel: (moduleName: string) => void; + readonly isPerspectiveVisible: (perspectiveName: string) => boolean; + readonly togglePerspectiveVisibility: (perspectiveName: string) => void; +}; + +export function PerspectiveModuleFilter({ + moduleName, + displayName, + perspectives, + isOpen, + onTogglePanel, + isPerspectiveVisible, + togglePerspectiveVisibility, +}: PerspectiveModuleFilterProps) { + // Theme colors + const headerBgColor = useColorModeValue("gray.50", "#252E3F"); + const borderColor = useColorModeValue("gray.200", "#2D3748"); + const hoverBgColor = useColorModeValue("gray.100", "#2D3748"); + const textColor = useColorModeValue("gray.800", "white"); + const bgColor = useColorModeValue("white", "#1A202C"); + const mutedTextColor = useColorModeValue("gray.500", "gray.400"); + const checkboxActiveBg = useColorModeValue("blue.500", "blue.400"); + const checkboxBorderColor = useColorModeValue("gray.300", "gray.600"); + const itemHoverBg = useColorModeValue("gray.50", "#2A3749"); + const buttonColor = useColorModeValue("blue.500", "blue.400"); + + // Compute the module's visibility state + const moduleVisibilityState = useMemo(() => { + if (!perspectives || perspectives.length === 0) return { checked: false, indeterminate: false }; + + const visibleCount = perspectives.filter(p => isPerspectiveVisible(p.name)).length; + + if (visibleCount === 0) return { checked: false, indeterminate: false }; + if (visibleCount === perspectives.length) return { checked: true, indeterminate: false }; + return { checked: false, indeterminate: true }; + }, [perspectives, isPerspectiveVisible]); + + // Toggle all perspectives in the module + const toggleAllPerspectives = () => { + // If all are visible or indeterminate, hide all; otherwise show all + const shouldShow = !moduleVisibilityState.checked && !moduleVisibilityState.indeterminate; + + for (const perspective of perspectives) { + // Only toggle if the current state doesn't match what we want + if (isPerspectiveVisible(perspective.name) !== shouldShow) { + togglePerspectiveVisibility(perspective.name); + } + } + }; + + return ( + + + {/* Module toggle checkbox */} + e.stopPropagation()} + > + + + {moduleVisibilityState.checked && ( + + ✓ + + )} + {moduleVisibilityState.indeterminate && ( + + )} + + + + {/* Accordion toggle button */} + onTogglePanel(moduleName)} + mr={2} + > + {displayName} + + ▼ + + + + {/* View module button */} + + + → + + + + + {perspectives?.map((perspective) => { + // Extract perspective ID from the full name + const perspectiveId = perspective.name.includes("/") + ? perspective.name.split("/").pop() ?? perspective.name + : perspective.name; + + return ( + + {/* Checkbox for toggling visibility */} + + togglePerspectiveVisibility(perspective.name)} + style={{ display: "none" }} + /> + + {isPerspectiveVisible(perspective.name) && ( + + ✓ + + )} + + + + {/* Perspective name (not a link) */} + + {perspective.display_name} + + + {/* View perspective button */} + + + → + + + + ); + })} + + + ); +} diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/ModuleInfo.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/ModuleInfo.tsx new file mode 100644 index 00000000..230314cf --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/ModuleInfo.tsx @@ -0,0 +1,48 @@ +import { useColorModeValue } from "@/components/ui/theme/color-mode"; +import type { PerspectiveModule } from "@/features/perspectives/types"; +import { + Badge, + Box, + Flex, + Heading, + Text, +} from "@chakra-ui/react"; + +interface ModuleInfoProps { + module: PerspectiveModule; +} + +/** + * ModuleInfo component displays information about a perspective module + */ +export function ModuleInfo({ module }: ModuleInfoProps) { + const borderColor = useColorModeValue("gray.200", "gray.700"); + + return ( + <> + + {module.display_name} + + {module.enabled ? "Enabled" : "Disabled"} + + + + + + + + Module Information + + + This module contains {module.perspectives.length}{" "} + perspectives. + + {module.description && ( + + {module.description} + + )} + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/ModuleList.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/ModuleList.tsx new file mode 100644 index 00000000..4246b609 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/ModuleList.tsx @@ -0,0 +1,80 @@ +import type { Perspective, PerspectiveModule } from "@/features/perspectives/types"; +import { + Badge, + Box, + Flex, + Heading, + Text, + VStack, +} from "@chakra-ui/react"; +import { Link } from "@tanstack/react-router"; + +interface ModuleListProps { + readonly module: PerspectiveModule; +} + +/** + * ModuleList component displays a list of perspectives from a specific module + */ +export function ModuleList({ module }: ModuleListProps) { + if (module.perspectives.length === 0) { + return No perspectives found in this module.; + } + + return ( + + {module.perspectives.map((perspective: Perspective) => ( + + + + + {perspective.display_name || perspective.name} + + + {perspective.version} + + + + {perspective.description?.substring(0, 60) ?? "No description available"}... + + + + ))} + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/index.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/PerspectiveModules/index.ts new file mode 100644 index 00000000..e69de29b diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/SchemaValidationError.tsx b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/SchemaValidationError.tsx new file mode 100644 index 00000000..41e7123f --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/SchemaValidationError.tsx @@ -0,0 +1,56 @@ +import { + Box, + Code, + Flex, + Heading, + Text +} from "@chakra-ui/react"; + +interface SchemaValidationErrorProps { + error: Error; +} + +/** + * Component to display detailed schema validation errors + */ +export function SchemaValidationError({ error }: SchemaValidationErrorProps) { + // Check if the error contains validation information + const isSchemaError = error.message.includes("Invalid enum value") || + error.message.includes("schema_fields") || + error.message.includes("Expected 'str' | 'float'"); + + return ( + + + Schema Validation Error + + + {isSchemaError ? ( + <> + + There was an error loading perspectives due to schema validation issues with complex fields. + The system is expecting only simple field types ('str' or 'float') but found complex nested objects. + + Possible Solutions: + + Update the server to support complex field structures (recommended) + Simplify perspective schemas to avoid nested fields + + Technical Details: + + {error.message} + + + ) : ( + {error.message} + )} + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/index.ts b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/index.ts new file mode 100644 index 00000000..2efe2459 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/components/PerspectiveManagement/index.ts @@ -0,0 +1,10 @@ +export { ModuleList } from './PerspectiveModules/ModuleList'; +export { ModuleInfo } from './PerspectiveModules/ModuleInfo'; +export { NotFound } from './NotFound'; +export { ErrorDisplay } from './ErrorDisplay'; +export { LoadingDisplay } from './LoadingDisplay'; +export { PerspectiveEditor } from './PerspectiveEditor/PerspectiveEditor'; +export { SchemaValidationError } from './SchemaValidationError'; +export { PerspectiveModuleFilter } from './PerspectiveModuleFilter'; +export { PerspectiveManagementPanel } from './PerspectiveManagementPanel'; +export { PerspectiveFilterPanel } from './PerspectiveFilterPanel'; diff --git a/graphcap_studio/src/features/perspectives/components/index.ts b/graphcap_studio/src/features/perspectives/components/index.ts index b5b311e3..07d89562 100644 --- a/graphcap_studio/src/features/perspectives/components/index.ts +++ b/graphcap_studio/src/features/perspectives/components/index.ts @@ -5,11 +5,12 @@ * This module exports all component files from the perspectives feature. */ -export * from "./EmptyPerspectives"; +export * from "./PerspectiveCaption/EmptyPerspectives"; export * from "./PerspectivesErrorState"; -export * from "./PerspectiveFilterPanel"; -export * from "./ErrorMessage"; -export { PerspectiveHeader } from "./PerspectiveNavigation/PerspectiveHeader"; -export { PerspectivesPager } from "./PerspectiveNavigation/PerspectivesPager"; -export { MetadataDisplay } from "./PerspectiveCard/MetadataDisplay"; -export * from "./PerspectiveActions"; +export * from "./PerspectiveManagement/PerspectiveFilterPanel"; +export * from "./PerspectiveCaption/ErrorMessage"; +export { PerspectiveHeader } from "./PerspectiveCaption/PerspectiveNavigation/PerspectiveHeader"; +export { PerspectivesPager } from "./PerspectiveCaption/PerspectiveNavigation/PerspectivesPager"; +export { MetadataDisplay } from "./PerspectiveCaption/PerspectiveCard/MetadataDisplay"; +export * from "./PerspectiveCaption/PerspectiveActions"; +export * from "./PerspectiveManagement/PerspectiveManagementPanel"; diff --git a/graphcap_studio/src/features/perspectives/constants/index.ts b/graphcap_studio/src/features/perspectives/constants/index.ts index 52fec2e7..ef436ffe 100644 --- a/graphcap_studio/src/features/perspectives/constants/index.ts +++ b/graphcap_studio/src/features/perspectives/constants/index.ts @@ -39,6 +39,9 @@ export const PERSPECTIVE_CLASSES = { // Query keys for TanStack Query export const perspectivesQueryKeys = { perspectives: ["perspectives"] as const, + modules: ["perspectives", "modules"] as const, + modulePerspectives: (moduleName: string) => + ["perspectives", "modules", moduleName] as const, caption: (imagePath: string, perspective: string) => [ ...perspectivesQueryKeys.perspectives, @@ -55,11 +58,15 @@ export const API_ENDPOINTS = { VIEW_IMAGE: "/images/view", REST_LIST_PERSPECTIVES: "/perspectives/list", REST_GENERATE_CAPTION: "/perspectives/caption-from-path", + // Module API endpoints + LIST_MODULES: "/perspectives/modules", + MODULE_PERSPECTIVES: "/perspectives/modules/{module_name}", }; // Cache stale times (in milliseconds) export const CACHE_TIMES = { PERSPECTIVES_STALE_TIME: 1000 * 60 * 5, // 5 minutes + MODULES_STALE_TIME: 1000 * 60 * 5, // 5 minutes }; // Default values diff --git a/graphcap_studio/src/features/perspectives/context/PerspectivesDataContext.tsx b/graphcap_studio/src/features/perspectives/context/PerspectivesDataContext.tsx index d51287c4..505f9327 100644 --- a/graphcap_studio/src/features/perspectives/context/PerspectivesDataContext.tsx +++ b/graphcap_studio/src/features/perspectives/context/PerspectivesDataContext.tsx @@ -11,11 +11,11 @@ import { useServerConnectionsContext } from "@/context"; import { SERVER_IDS } from "@/features/server-connections/constants"; -import { Image } from "@/services/images"; +import type { Image } from "@/services/images"; import React, { createContext, useContext, - ReactNode, + type ReactNode, useState, useCallback, useEffect, @@ -24,7 +24,7 @@ import React, { import { useProviders } from "../../inference/services/providers"; import { useGeneratePerspectiveCaption } from "../hooks/useGeneratePerspectiveCaption"; import { usePerspectives } from "../hooks/usePerspectives"; -import { +import type { CaptionOptions, Perspective, PerspectiveData, diff --git a/graphcap_studio/src/features/perspectives/hooks/index.ts b/graphcap_studio/src/features/perspectives/hooks/index.ts index 3f4c234e..d67eb8be 100644 --- a/graphcap_studio/src/features/perspectives/hooks/index.ts +++ b/graphcap_studio/src/features/perspectives/hooks/index.ts @@ -2,7 +2,7 @@ /** * Perspectives Hooks * - * This module exports hooks for the perspectives components. + * This module exports all hooks used by the perspectives feature. */ // UI Hooks @@ -10,8 +10,10 @@ export { usePerspectiveUI } from "./usePerspectiveUI"; // API Hooks export { usePerspectives } from "./usePerspectives"; +export { usePerspectiveModules } from "./usePerspectiveModules"; export { useGeneratePerspectiveCaption } from "./useGeneratePerspectiveCaption"; export { useImagePerspectives } from "./useImagePerspectives"; + // Utilities export * from "./utils"; diff --git a/graphcap_studio/src/features/perspectives/hooks/useGeneratePerspectiveCaption.ts b/graphcap_studio/src/features/perspectives/hooks/useGeneratePerspectiveCaption.ts index e52fc88a..a0de62be 100644 --- a/graphcap_studio/src/features/perspectives/hooks/useGeneratePerspectiveCaption.ts +++ b/graphcap_studio/src/features/perspectives/hooks/useGeneratePerspectiveCaption.ts @@ -7,7 +7,7 @@ import { useServerConnectionsContext } from "@/context"; import { SERVER_IDS } from "@/features/server-connections/constants"; -import { ServerConnection } from "@/features/server-connections/types"; +import type { ServerConnection } from "@/features/server-connections/types"; import { useMutation, useQueryClient } from "@tanstack/react-query"; import { API_ENDPOINTS, perspectivesQueryKeys } from "../services/constants"; import { @@ -15,7 +15,7 @@ import { getGraphCapServerUrl, handleApiError, } from "../services/utils"; -import { CaptionOptions, CaptionResponse } from "../types"; +import type { CaptionOptions, CaptionResponse } from "../types"; /** * Hook to generate a perspective caption for an image diff --git a/graphcap_studio/src/features/perspectives/hooks/useImagePerspectives.ts b/graphcap_studio/src/features/perspectives/hooks/useImagePerspectives.ts index 1eeee5c6..c9bcf2ed 100644 --- a/graphcap_studio/src/features/perspectives/hooks/useImagePerspectives.ts +++ b/graphcap_studio/src/features/perspectives/hooks/useImagePerspectives.ts @@ -8,10 +8,10 @@ import { useServerConnectionsContext } from "@/context"; import { useProviders } from "@/features/inference/services/providers"; import { SERVER_IDS } from "@/features/server-connections/constants"; -import { Image } from "@/services/images"; +import type { Image } from "@/services/images"; import { useCallback, useEffect, useState } from "react"; -import { +import type { CaptionOptions, ImageCaptions, ImagePerspectivesResult, diff --git a/graphcap_studio/src/features/perspectives/hooks/useModulePerspectives.ts b/graphcap_studio/src/features/perspectives/hooks/useModulePerspectives.ts new file mode 100644 index 00000000..89b03138 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/hooks/useModulePerspectives.ts @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: Apache-2.0 +/** + * useModulePerspectives Hook (Compatibility Wrapper) + * + * This hook is kept for backward compatibility. It now uses the enhanced + * usePerspectiveModules hook internally. + * + * @deprecated Use usePerspectiveModules().getModulePerspectives() instead. + */ + +import { useMemo } from "react"; +import { usePerspectiveModules } from "./usePerspectiveModules"; + +/** + * Hook to fetch perspectives for a specific module from the server (compatibility wrapper) + * + * @param moduleName - Name of the module to fetch perspectives for + * @returns A query result with the module and its perspectives + */ +export function useModulePerspectives(moduleName: string) { + // Use the enhanced hooks to get the module data + const { getModulePerspectives } = usePerspectiveModules(); + const moduleData = getModulePerspectives(moduleName); + + // Map to the expected return shape for backward compatibility + return useMemo(() => { + return { + data: moduleData.module ? { + module: moduleData.module, + perspectives: moduleData.perspectives + } : undefined, + isLoading: moduleData.isLoading, + isError: moduleData.isError, + error: moduleData.error, + refetch: moduleData.refetch + }; + }, [moduleData]); +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/hooks/usePerspectiveModules.ts b/graphcap_studio/src/features/perspectives/hooks/usePerspectiveModules.ts new file mode 100644 index 00000000..7f43b7d0 --- /dev/null +++ b/graphcap_studio/src/features/perspectives/hooks/usePerspectiveModules.ts @@ -0,0 +1,340 @@ +// SPDX-License-Identifier: Apache-2.0 +/** + * usePerspectiveModules Hook + * + * This hook fetches available perspective modules and their perspectives from the server. + * It combines functionality from both useModules and useModulePerspectives. + */ + +import { useServerConnectionsContext } from "@/context"; +import { SERVER_IDS } from "@/features/server-connections/constants"; +import { useQuery } from "@tanstack/react-query"; +import { useEffect, useMemo } from "react"; +import { + API_ENDPOINTS, + CACHE_TIMES, + perspectivesQueryKeys, +} from "../services/constants"; +import { getGraphCapServerUrl, handleApiError } from "../services/utils"; +import type { ModuleInfo, ModuleListResponse, Perspective, PerspectiveModule } from "../types"; +import { PerspectiveError } from "./usePerspectives"; + +type ModuleQueryResult = { + isLoading: boolean; + isError: boolean; + error: Error | null; + data: PerspectiveModule[]; + modules: PerspectiveModule[]; + hasModules: boolean; + refetch: () => void; + + // Methods for accessing specific module data + getModule: (moduleName: string) => PerspectiveModule | undefined; + getModulePerspectives: (moduleName: string) => { + module: PerspectiveModule | undefined; + perspectives: Perspective[]; + isLoading: boolean; + isError: boolean; + error: Error | null; + refetch: () => void; + }; +}; + +/** + * Utility function to convert PerspectiveModule to ModuleInfo + * This maintains compatibility with the older data structure + */ +function perspectiveModuleToModuleInfo(module: PerspectiveModule): ModuleInfo { + return { + name: module.name, + display_name: module.display_name, + description: module.description, + enabled: module.enabled, + perspective_count: module.perspectives.length + }; +} + +/** + * Hook to get just the module info without perspectives + * This is useful for components that only need the basic module data + * + * @returns Query result with basic module information + */ +export function useModuleInfo() { + const { modules, isLoading, isError, error, refetch } = usePerspectiveModules(); + + const moduleInfos = useMemo(() => { + return modules.map(perspectiveModuleToModuleInfo); + }, [modules]); + + return { + data: moduleInfos, + isLoading, + isError, + error, + refetch + }; +} + +/** + * This hook provides listing all modules and + * accessing a specific module's data. + * + * @returns Query result with a list of modules and their perspectives, plus methods to access specific module data + */ +export function usePerspectiveModules(): ModuleQueryResult { + const { connections } = useServerConnectionsContext(); + const graphcapServerConnection = connections.find( + (conn) => conn.id === SERVER_IDS.GRAPHCAP_SERVER + ); + const isConnected = graphcapServerConnection?.status === "connected"; + + // Fetch all available modules (previously provided by useModules) + const modulesQuery = useQuery({ + queryKey: perspectivesQueryKeys.modules, + queryFn: async () => { + try { + // Handle case when server connection is not established + if (!isConnected) { + console.warn("Server connection not established"); + throw new PerspectiveError("Server connection not established", { + code: "SERVER_CONNECTION_ERROR", + context: { connections }, + }); + } + + const baseUrl = getGraphCapServerUrl(connections); + if (!baseUrl) { + console.warn("No GraphCap server URL available"); + throw new PerspectiveError("No GraphCap server URL available", { + code: "MISSING_SERVER_URL", + context: { connections }, + }); + } + + const endpoint = API_ENDPOINTS.LIST_MODULES; + const url = `${baseUrl}${endpoint}`; + + console.debug(`Fetching modules from server: ${url}`); + + const response = await fetch(url); + + if (!response.ok) { + return handleApiError(response, "Failed to fetch modules"); + } + + const data = (await response.json()) as ModuleListResponse; + + console.debug( + `Successfully fetched ${data.modules.length} modules`, + ); + + return data.modules; + } catch (error) { + // Improve error handling - log the error and rethrow + console.error("Error fetching modules:", error); + + // If it's already a PerspectiveError, just rethrow it + if (error instanceof PerspectiveError) { + throw error; + } + + // Otherwise, wrap it in a PerspectiveError + throw new PerspectiveError( + error instanceof Error + ? error.message + : "Unknown error fetching modules", + { + code: "MODULE_FETCH_ERROR", + context: { error }, + }, + ); + } + }, + // Only enable the query when the server is connected + enabled: isConnected, + staleTime: CACHE_TIMES.MODULES, + retry: (failureCount, error) => { + // Don't retry for connection errors or missing URLs + if (error instanceof PerspectiveError) { + if ( + ["SERVER_CONNECTION_ERROR", "MISSING_SERVER_URL"].includes(error.code) + ) { + return false; + } + } + + // Retry other errors up to 3 times + return failureCount < 3; + }, + retryDelay: (attempt) => Math.min(1000 * 2 ** attempt, 30000), + }); + + // Watch for changes in connection status and refetch when connected + useEffect(() => { + if (isConnected && modulesQuery.isError) { + modulesQuery.refetch(); + } + }, [isConnected, modulesQuery]); + + // Then fetch all module details in a single query + const modulesWithPerspectivesQuery = useQuery({ + // Use a different query key to avoid conflicts with the modules query + queryKey: [...perspectivesQueryKeys.modules, 'with-perspectives'], + queryFn: async () => { + // If we don't have modules data yet, return empty array + if (!modulesQuery.data || modulesQuery.data.length === 0) { + return []; + } + + try { + // We need to fetch perspectives for each module + const modulePromises = modulesQuery.data.map(async (moduleInfo) => { + // Import the API here to avoid circular dependencies + const { perspectivesApi } = await import("../services/api"); + + try { + // Fetch perspectives for this module + const moduleData = await perspectivesApi.getModulePerspectives(moduleInfo.name); + + // Convert to our internal PerspectiveModule format + return { + name: moduleData.module.name, + display_name: moduleData.module.display_name, + description: moduleData.module.description, + enabled: moduleData.module.enabled, + perspectives: moduleData.perspectives + }; + } catch (error) { + console.error(`Error fetching perspectives for module ${moduleInfo.name}:`, error); + // Return a module with empty perspectives array on error + return { + name: moduleInfo.name, + display_name: moduleInfo.display_name, + description: moduleInfo.description || "", + enabled: moduleInfo.enabled, + perspectives: [] + }; + } + }); + + // Wait for all module promises to resolve + const modules = await Promise.all(modulePromises); + return modules; + } catch (error) { + console.error("Error fetching module perspectives:", error); + + // Return basic module data without perspectives instead of throwing + if (modulesQuery.data) { + return modulesQuery.data.map((moduleInfo) => ({ + name: moduleInfo.name, + display_name: moduleInfo.display_name, + description: moduleInfo.description || "", + enabled: moduleInfo.enabled, + perspectives: [] + })); + } + + throw error; + } + }, + // Only run this query when we have modules data and we're connected + enabled: isConnected && !modulesQuery.isLoading && !!modulesQuery.data, + // Use the previous data as placeholder while fetching new data + placeholderData: (prev) => prev || [] + }); + + /** + * Function to get a specific module by name + * + * @param moduleName - Name of the module to retrieve + * @returns The module if found, undefined otherwise + */ + const getModule = useMemo(() => { + return (moduleName: string): PerspectiveModule | undefined => { + if (!modulesWithPerspectivesQuery.data) return undefined; + return modulesWithPerspectivesQuery.data.find(m => m.name === moduleName); + }; + }, [modulesWithPerspectivesQuery.data]); + + /** + * Function to get perspectives for a specific module + * + * This functionality replaces the previous useModulePerspectives hook + * + * @param moduleName - Name of the module to get perspectives for + * @returns Object with module data, perspectives, and query state + */ + const getModulePerspectives = useMemo(() => { + return (moduleName: string) => { + // Try to get the module from our already-loaded data first + const cachedModule = getModule(moduleName); + + // If modules are still loading, return loading state with the data we have + if (modulesWithPerspectivesQuery.isLoading) { + return { + module: cachedModule, + perspectives: cachedModule?.perspectives || [], + isLoading: true, + isError: false, + error: null, + refetch: () => modulesWithPerspectivesQuery.refetch() + }; + } + + // If we have an error fetching modules, return that error + if (modulesWithPerspectivesQuery.isError) { + return { + module: cachedModule, + perspectives: cachedModule?.perspectives || [], + isLoading: false, + isError: true, + error: modulesWithPerspectivesQuery.error, + refetch: () => modulesWithPerspectivesQuery.refetch() + }; + } + + // If we have the module data, return it + if (cachedModule) { + return { + module: cachedModule, + perspectives: cachedModule.perspectives, + isLoading: false, + isError: false, + error: null, + refetch: () => modulesWithPerspectivesQuery.refetch() + }; + } + + // If we don't have the module data, it might not exist + return { + module: undefined, + perspectives: [], + isLoading: false, + isError: false, + error: null, + refetch: () => modulesWithPerspectivesQuery.refetch() + }; + }; + }, [getModule, modulesWithPerspectivesQuery]); + + // Always return a valid array of modules, even during loading + const modules = modulesWithPerspectivesQuery.data || []; + + return { + isLoading: modulesQuery.isLoading || modulesWithPerspectivesQuery.isLoading, + isError: modulesQuery.isError || modulesWithPerspectivesQuery.isError, + error: modulesQuery.error || modulesWithPerspectivesQuery.error || null, + data: modules, + modules, + // Add a hasModules flag to easily check if we have modules + hasModules: modules.length > 0, + refetch: () => { + modulesQuery.refetch(); + modulesWithPerspectivesQuery.refetch(); + }, + // Add methods for accessing specific module data + getModule, + getModulePerspectives + }; +} \ No newline at end of file diff --git a/graphcap_studio/src/features/perspectives/hooks/usePerspectives.ts b/graphcap_studio/src/features/perspectives/hooks/usePerspectives.ts index 5422bf2a..147ee79f 100644 --- a/graphcap_studio/src/features/perspectives/hooks/usePerspectives.ts +++ b/graphcap_studio/src/features/perspectives/hooks/usePerspectives.ts @@ -8,7 +8,7 @@ import { useServerConnectionsContext } from "@/context"; import { SERVER_IDS } from "@/features/server-connections/constants"; -import { ServerConnection } from "@/features/server-connections/types"; +import type { ServerConnection } from "@/features/server-connections/types"; import { useQuery } from "@tanstack/react-query"; import { useEffect } from "react"; import { @@ -17,7 +17,7 @@ import { perspectivesQueryKeys, } from "../services/constants"; import { getGraphCapServerUrl, handleApiError } from "../services/utils"; -import { Perspective, PerspectiveListResponse } from "../types"; +import type { Perspective, PerspectiveListResponse } from "../types"; /** * Custom error class for perspective fetching errors diff --git a/graphcap_studio/src/features/perspectives/index.ts b/graphcap_studio/src/features/perspectives/index.ts index 29204acd..a198b9b3 100644 --- a/graphcap_studio/src/features/perspectives/index.ts +++ b/graphcap_studio/src/features/perspectives/index.ts @@ -22,3 +22,6 @@ export * from "./types"; // Export hooks export * from "./hooks"; + +// Export components +export * from "./components"; diff --git a/graphcap_studio/src/features/perspectives/services/api.ts b/graphcap_studio/src/features/perspectives/services/api.ts index cdd51088..4f86ae88 100644 --- a/graphcap_studio/src/features/perspectives/services/api.ts +++ b/graphcap_studio/src/features/perspectives/services/api.ts @@ -7,14 +7,39 @@ import { API_ENDPOINTS } from "../constants/index"; import { - CaptionRequest, CaptionRequestSchema, - CaptionResponse, CaptionResponseSchema, - Perspective, + ModuleListResponseSchema, + ModulePerspectivesResponseSchema, PerspectiveListResponseSchema, } from "../types"; -import { ensureWorkspacePath, handleApiError } from "./utils"; +import type { + CaptionRequest, + CaptionResponse, + ModuleListResponse, + ModulePerspectivesResponse, + Perspective, +} from "../types"; +import { ensureWorkspacePath, getGraphCapServerUrl, handleApiError } from "./utils"; + +/** + * Get server connections from local storage + */ +const getConnections = () => { + // Get the current connections from local storage + const connectionsStr = localStorage.getItem("graphcap-server-connections"); + let connections = []; + + if (connectionsStr) { + try { + connections = JSON.parse(connectionsStr); + } catch (e) { + console.warn("Could not parse server connections from localStorage"); + } + } + + return connections; +}; /** * Service for interacting with the perspectives API @@ -30,7 +55,16 @@ export const perspectivesApi = { */ async listPerspectives(): Promise { try { - const response = await fetch(API_ENDPOINTS.REST_LIST_PERSPECTIVES); + // Get the base URL using the utility function + const connections = getConnections(); + const baseUrl = getGraphCapServerUrl(connections); + + // Create the full URL by combining base URL and endpoint path + const url = `${baseUrl}${API_ENDPOINTS.LIST_PERSPECTIVES}`; + + console.debug(`Fetching perspectives from: ${url}`); + + const response = await fetch(url); if (!response.ok) { await handleApiError(response, "Failed to fetch perspectives"); @@ -42,7 +76,107 @@ export const perspectivesApi = { return validatedData.perspectives; } catch (error) { console.error("Error in perspectivesApi.listPerspectives:", error); - return []; + throw error; + } + }, + + /** + * Get a list of available perspective modules + * + * @returns Promise with the list of modules + */ + async listModules(): Promise { + try { + // Get the base URL using the utility function + const connections = getConnections(); + const baseUrl = getGraphCapServerUrl(connections); + + // Create the full URL by combining base URL and endpoint path + const url = `${baseUrl}${API_ENDPOINTS.LIST_MODULES}`; + + console.debug(`Fetching modules from: ${url}`); + + const response = await fetch(url); + + if (!response.ok) { + await handleApiError(response, "Failed to fetch perspective modules"); + } + + const data = await response.json(); + // Validate the response with Zod + const validatedData = ModuleListResponseSchema.parse(data); + return validatedData; + } catch (error) { + console.error("Error in perspectivesApi.listModules:", error); + throw error; + } + }, + + /** + * Get all perspectives for a specific module + * + * @param moduleName - Name of the module to get perspectives for + * @returns Promise with the module and its perspectives + */ + async getModulePerspectives(moduleName: string): Promise { + try { + // Get the base URL using the utility function + const connections = getConnections(); + const baseUrl = getGraphCapServerUrl(connections); + + // Create the endpoint path + const endpointPath = API_ENDPOINTS.MODULE_PERSPECTIVES.replace( + "{module_name}", + encodeURIComponent(moduleName) + ); + + // Create the full URL by combining base URL and endpoint path + const url = `${baseUrl}${endpointPath}`; + + console.debug(`Fetching perspectives for module '${moduleName}' from: ${url}`); + + const response = await fetch(url); + + if (!response.ok) { + // Check if we got HTML instead of JSON + const contentType = response.headers.get("content-type"); + if (contentType?.includes("text/html")) { + throw new Error( + `Received HTML response instead of JSON. Status: ${response.status}. This typically indicates a routing issue or server error.`, + ); + } + + await handleApiError(response, `Failed to fetch perspectives for module '${moduleName}'`); + } + + // Check content type before parsing + const contentType = response.headers.get("content-type"); + if (!contentType?.includes("application/json")) { + console.error(`Unexpected content type: ${contentType}`); + throw new Error( + `Expected JSON response but got ${contentType ?? "unknown content type"}`, + ); + } + + const data = await response.json(); + + // Use looser validation to handle the complex fields + // We'll validate but ignore validation errors to allow the data through + try { + // Try normal validation first + const validatedData = ModulePerspectivesResponseSchema.parse(data); + return validatedData; + } catch (validationError: unknown) { + // Log the validation error but still return the data + console.warn("Response validation warning:", validationError); + console.debug("Returning unvalidated response data for module:", moduleName); + + // Return the unvalidated data + return data as ModulePerspectivesResponse; + } + } catch (error) { + console.error(`Error in perspectivesApi.getModulePerspectives(${moduleName}):`, error); + throw error; } }, @@ -56,10 +190,15 @@ export const perspectivesApi = { requestParams: CaptionRequest, ): Promise { try { + // Get the base URL using the utility function + const connections = getConnections(); + const baseUrl = getGraphCapServerUrl(connections); + // Ensure the image path has the correct workspace prefix const normalizedImagePath = ensureWorkspacePath(requestParams.image_path); console.log("Generating caption for image path:", normalizedImagePath); console.log("Request params:", requestParams); + // Create the request body and validate with Zod const request: CaptionRequest = { ...requestParams, @@ -68,9 +207,12 @@ export const perspectivesApi = { // Validate the request with Zod const validatedRequest = CaptionRequestSchema.parse(request); + + // Create the full URL by combining base URL and endpoint path + const url = `${baseUrl}${API_ENDPOINTS.REST_GENERATE_CAPTION}`; // Make the API request - const response = await fetch(API_ENDPOINTS.REST_GENERATE_CAPTION, { + const response = await fetch(url, { method: "POST", headers: { "Content-Type": "application/json", diff --git a/graphcap_studio/src/features/perspectives/services/constants.ts b/graphcap_studio/src/features/perspectives/services/constants.ts index a66261bf..ff1a850a 100644 --- a/graphcap_studio/src/features/perspectives/services/constants.ts +++ b/graphcap_studio/src/features/perspectives/services/constants.ts @@ -18,6 +18,17 @@ export const API_ENDPOINTS = { * Endpoint to generate a caption for an image using a perspective */ REST_GENERATE_CAPTION: "/perspectives/caption-from-path", + + /** + * Endpoint to list all available perspective modules + */ + LIST_MODULES: "/perspectives/modules", + + /** + * Endpoint to get perspectives for a specific module + * Note: This is a template that needs to be interpolated with the module name + */ + MODULE_PERSPECTIVES: "/perspectives/modules/{module_name}", }; /** @@ -28,6 +39,11 @@ export const CACHE_TIMES = { * Stale time for perspectives data (5 minutes) */ PERSPECTIVES: 5 * 60 * 1000, // 5 minutes + + /** + * Stale time for modules data (5 minutes) + */ + MODULES: 5 * 60 * 1000, // 5 minutes }; /** @@ -54,6 +70,19 @@ export const perspectivesQueryKeys = { */ perspectives: ["perspectives"], + /** + * Query key for fetching all modules + */ + modules: ["perspectives", "modules"], + + /** + * Query key for fetching perspectives in a specific module + * + * @param moduleName - Name of the module + * @returns Query key array + */ + modulePerspectives: (moduleName: string) => ["perspectives", "modules", moduleName], + /** * Query key for fetching a caption for an image using a perspective * diff --git a/graphcap_studio/src/features/perspectives/services/utils.ts b/graphcap_studio/src/features/perspectives/services/utils.ts index 9a08f2ea..ca78209b 100644 --- a/graphcap_studio/src/features/perspectives/services/utils.ts +++ b/graphcap_studio/src/features/perspectives/services/utils.ts @@ -6,7 +6,7 @@ */ import { DEFAULTS } from "@/features/perspectives/constants/index"; -import { ServerConnection } from "@/features/perspectives/types"; +import type { ServerConnection } from "@/features/perspectives/types"; import { SERVER_IDS } from "@/features/server-connections/constants"; /** @@ -17,11 +17,17 @@ export function getGraphCapServerUrl(connections: ServerConnection[]): string { (conn) => conn.id === SERVER_IDS.GRAPHCAP_SERVER, ); - return ( + // Get URL from connection, environment variable, or default + const serverUrl = graphcapServerConnection?.url ?? import.meta.env.VITE_GRAPHCAP_SERVER_URL ?? - DEFAULTS.SERVER_URL - ); + import.meta.env.VITE_API_URL ?? + DEFAULTS.SERVER_URL; + + // Log the server URL being used for debugging + console.debug(`Using GraphCap server URL: ${serverUrl}`); + + return serverUrl; } /** @@ -52,6 +58,39 @@ export function isUrl(path: string): boolean { return path.startsWith("http://") || path.startsWith("https://"); } +/** + * Parse JSON error data into a readable error message + */ +function parseJsonErrorData(errorData: Record): { + message: string; + details: Record; +} { + let message = ""; + + // FastAPI error format + if (errorData.detail) { + if (typeof errorData.detail === "string") { + message = errorData.detail; + } else if (Array.isArray(errorData.detail)) { + // Handle validation errors + message = errorData.detail + .map( + (err: { loc: string[]; msg: string }) => + `${err.loc.join(".")}: ${err.msg}`, + ) + .join(", "); + } else { + message = JSON.stringify(errorData.detail); + } + } else if (errorData.message && typeof errorData.message === "string") { + message = errorData.message; + } else { + message = JSON.stringify(errorData); + } + + return { message, details: errorData }; +} + /** * Handle API error responses */ @@ -60,43 +99,55 @@ export async function handleApiError( defaultMessage: string, ): Promise { let errorMessage = `${defaultMessage}: ${response.status}`; + let errorDetails: Record | null = null; try { - const contentType = response.headers.get("content-type"); + const contentType = response.headers.get("content-type") ?? ""; - if (contentType && contentType.includes("application/json")) { + // Handle response based on content type + if (contentType.includes("application/json")) { const errorData = await response.json(); - - // FastAPI error format - if (errorData.detail) { - if (typeof errorData.detail === "string") { - errorMessage = errorData.detail; - } else if (Array.isArray(errorData.detail)) { - // Handle validation errors - errorMessage = errorData.detail - .map((err: any) => `${err.loc.join(".")}: ${err.msg}`) - .join(", "); - } else { - errorMessage = JSON.stringify(errorData.detail); - } - } else if (errorData.message) { - errorMessage = errorData.message; - } else { - errorMessage = JSON.stringify(errorData); - } + const parsedError = parseJsonErrorData(errorData); + errorMessage = parsedError.message; + errorDetails = parsedError.details; + } else if (contentType.includes("text/html")) { + const textError = await response.text(); + const excerpt = + textError.substring(0, 200) + (textError.length > 200 ? "..." : ""); + errorMessage = `${defaultMessage}: Received HTML response (Status: ${response.status})`; + errorDetails = { htmlExcerpt: excerpt }; + console.warn("Received HTML response instead of expected JSON:", excerpt); } else { - // Try to get text response const textError = await response.text(); if (textError) { - errorMessage = textError; + errorMessage = textError.substring(0, 500); // Limit size + errorDetails = { text: textError }; } } } catch (e) { - // If we can't parse the error as JSON, use the status text console.error("Error parsing API error response", e); - errorMessage = `${defaultMessage}: ${response.statusText}`; + errorMessage = `${defaultMessage}: ${response.statusText} (Status: ${response.status})`; + errorDetails = { + parseError: e instanceof Error ? e.message : "Unknown error", + }; } - console.error(`API Error: ${errorMessage}`); - throw new Error(errorMessage); + // Log error details + console.error(`API Error: ${errorMessage}`, { + status: response.status, + url: response.url, + details: errorDetails, + }); + + // Create and throw error with additional properties + const error = new Error(errorMessage) as Error & { + status?: number; + details?: Record | null; + url?: string; + }; + error.status = response.status; + error.details = errorDetails; + error.url = response.url; + + throw error; } diff --git a/graphcap_studio/src/features/perspectives/types/index.ts b/graphcap_studio/src/features/perspectives/types/index.ts index 83395f28..6b6024f2 100644 --- a/graphcap_studio/src/features/perspectives/types/index.ts +++ b/graphcap_studio/src/features/perspectives/types/index.ts @@ -3,7 +3,8 @@ * Perspectives Types * * This module exports all types used by the perspectives feature. - * All type definitions are consolidated in the perspectivesTypes.ts file. + * Type definitions are consolidated in their respective files. */ export * from "./perspectivesTypes"; +export * from "./perspectiveModuleTypes"; diff --git a/graphcap_studio/src/features/perspectives/types/perspectiveModuleTypes.ts b/graphcap_studio/src/features/perspectives/types/perspectiveModuleTypes.ts new file mode 100644 index 00000000..3084b7ab --- /dev/null +++ b/graphcap_studio/src/features/perspectives/types/perspectiveModuleTypes.ts @@ -0,0 +1,69 @@ +// SPDX-License-Identifier: Apache-2.0 +/** + * Perspective Module Types + * + * This module defines types related to perspective modules and management. + */ + +import { z } from "zod"; +import { PerspectiveSchema } from "./perspectivesTypes"; +import type { Perspective } from "./perspectivesTypes"; + +/** + * Schema for module information + */ +export const ModuleInfoSchema = z.object({ + name: z.string(), + display_name: z.string(), + description: z.string(), + enabled: z.boolean(), + perspective_count: z.number() +}); + +/** + * Schema for module list response + */ +export const ModuleListResponseSchema = z.object({ + modules: z.array(ModuleInfoSchema) +}); + +/** + * Schema for module perspectives response + */ +export const ModulePerspectivesResponseSchema = z.object({ + module: ModuleInfoSchema, + perspectives: z.array(PerspectiveSchema) +}); + +/** + * Type representing module information + */ +export type ModuleInfo = z.infer; + +/** + * Type representing a module list response + */ +export type ModuleListResponse = z.infer; + +/** + * Type representing a module perspectives response + */ +export type ModulePerspectivesResponse = z.infer; + +/** + * Represents a module containing multiple perspectives + */ +export interface PerspectiveModule { + name: string; + display_name: string; + description: string; + enabled: boolean; + perspectives: Perspective[]; +} + +/** + * Represents a list of perspective modules + */ +export interface PerspectiveModuleList { + modules: PerspectiveModule[]; +} diff --git a/graphcap_studio/src/features/perspectives/types/perspectivesTypes.ts b/graphcap_studio/src/features/perspectives/types/perspectivesTypes.ts index 622745f5..2a1804a2 100644 --- a/graphcap_studio/src/features/perspectives/types/perspectivesTypes.ts +++ b/graphcap_studio/src/features/perspectives/types/perspectivesTypes.ts @@ -14,30 +14,39 @@ import { z } from "zod"; // SECTION A - ZOD SCHEMAS // ============================================================================ +/** + * Defines the field type for a schema field. + * This allows for either primitive types or complex nested types. + */ +export const FieldTypeSchema = z.union([ + z.enum(["str", "float"]), + z.object({}).passthrough(), // Allow any object structure +]); + /** * Defines the structure of a schema field with properties like name, type, * description, and optional flags for lists or complex types. */ export const SchemaFieldSchema: z.ZodType<{ name: string; - type: "str" | "float"; + type: string | object; // Changed from enum to accept any type description: string; is_list?: boolean; is_complex?: boolean; fields?: Array<{ name: string; - type: "str" | "float"; + type: string | object; // Changed from enum to accept any type description: string; is_list?: boolean; is_complex?: boolean; - }>; + }> | null; }> = z.object({ name: z.string(), - type: z.enum(["str", "float"]), + type: z.union([z.string(), z.object({}).passthrough()]), // Accept string or object description: z.string(), is_list: z.boolean().optional(), is_complex: z.boolean().optional(), - fields: z.array(z.lazy(() => SchemaFieldSchema)).optional(), + fields: z.nullable(z.array(z.lazy(() => SchemaFieldSchema))).optional(), }); /** diff --git a/graphcap_studio/src/pages/perspectives/index.ts b/graphcap_studio/src/pages/perspectives/index.ts new file mode 100644 index 00000000..b01092f0 --- /dev/null +++ b/graphcap_studio/src/pages/perspectives/index.ts @@ -0,0 +1,2 @@ +export { ModulePage } from './module/ModulePage'; +export { PerspectiveEditorPage } from './module/PerspectiveEditorPage'; \ No newline at end of file diff --git a/graphcap_studio/src/pages/perspectives/module/ModulePage.tsx b/graphcap_studio/src/pages/perspectives/module/ModulePage.tsx new file mode 100644 index 00000000..2e03f0e8 --- /dev/null +++ b/graphcap_studio/src/pages/perspectives/module/ModulePage.tsx @@ -0,0 +1,228 @@ +import { SelectContent, SelectItem, SelectRoot, SelectTrigger, SelectValueText } from "@/components/ui/select"; +import { LoadingDisplay, ModuleList, NotFound, SchemaValidationError } from "@/features/perspectives/components/PerspectiveManagement"; +import { usePerspectiveModules } from "@/features/perspectives/hooks"; +import type { PerspectiveModule } from "@/features/perspectives/types"; +import { + Badge, + Box, + Button, + Flex, + Heading, + Text, + createListCollection +} from "@chakra-ui/react"; +import { Outlet, useMatches, useNavigate } from "@tanstack/react-router"; +import { useEffect, useMemo, useState } from "react"; + +interface ModulePageProps { + readonly moduleName: string; +} + +/** + * Page component that displays a module and its perspectives + */ +export function ModulePage({ moduleName }: ModulePageProps) { + const perspectiveModules = usePerspectiveModules(); + const { modules, isLoading, error, refetch, getModule } = perspectiveModules; + const [selectedModule, setSelectedModule] = + useState(null); + const [errorDetails, setErrorDetails] = useState(null); + const navigate = useNavigate(); + const matches = useMatches(); + + // Check if we're currently at the module root (no perspective selected) + const isModuleRoot = useMemo(() => { + return matches.some(match => + match.routeId === '/perspectives/module/$moduleName' && + match.pathname === `/perspectives/module/${moduleName}` + ); + }, [matches, moduleName]); + + // Create collection for the select component + const modulesCollection = useMemo(() => { + if (!modules) return createListCollection({ items: [] }); + + return createListCollection({ + items: modules.map(module => ({ + label: module.display_name || module.name, + value: module.name, + module: module, + })), + }); + }, [modules]); + + // Find the selected module when data is loaded + useEffect(() => { + if (modules && modules.length > 0) { + // Use getModule to find the module directly + const module = getModule(moduleName); + setSelectedModule(module || null); + + // If module exists but has no perspectives, check if it might be an API error + if (module && module.perspectives.length === 0) { + // Look for HTML response errors in the console logs + console.debug(`Module ${moduleName} has 0 perspectives. This might indicate an API error.`); + setErrorDetails( + "No perspectives found for this module. This could be due to a server configuration issue." + ); + } else { + setErrorDetails(null); + + // If we're at the module root and the module has perspectives, + // automatically navigate to the first perspective + if (isModuleRoot && module && module.perspectives.length > 0) { + const firstPerspective = module.perspectives[0]; + navigate({ + to: "/perspectives/module/$moduleName/perspective/$perspectiveName", + params: { + moduleName: moduleName, + perspectiveName: firstPerspective.name + } + }); + } + } + } + }, [modules, moduleName, navigate, isModuleRoot, getModule]); + + // Handle module change + const handleModuleChange = (details: { value: string[] }) => { + const newModuleName = details.value[0]; + if (newModuleName && newModuleName !== moduleName) { + navigate({ to: "/perspectives/module/$moduleName", params: { moduleName: newModuleName } }); + } + }; + + // Handle loading state + if (isLoading) { + return ; + } + + // Handle error state + if (error) { + return ( + + + + + + + ); + } + + // Handle case when module is not found + if (!selectedModule) { + return ; + } + + return ( + + {/* Left sidebar with module info and perspectives list */} + + {/* Module selector */} + + + + + + + {modulesCollection.items.map((item) => ( + + {item.label} + + ))} + + + + + {/* Module header section */} + + + {selectedModule.display_name || selectedModule.name} + + Enabled + + + Module: {selectedModule.name} + + + {/* Module information */} + + + Module Information + + + This module contains {selectedModule.perspectives.length} {selectedModule.perspectives.length === 1 ? 'perspective' : 'perspectives'}. + + + Contains {selectedModule.perspectives.length} {selectedModule.perspectives.length === 1 ? 'perspective' : 'perspectives'} + + + + {/* Show warning if we have a module with no perspectives */} + {errorDetails && ( + + {errorDetails} + + + )} + + {/* Perspectives list */} + + + + Perspectives + + + + + + + + {/* Right content area with outlet */} + + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/pages/perspectives/module/PerspectiveEditorPage.tsx b/graphcap_studio/src/pages/perspectives/module/PerspectiveEditorPage.tsx new file mode 100644 index 00000000..7a7287e1 --- /dev/null +++ b/graphcap_studio/src/pages/perspectives/module/PerspectiveEditorPage.tsx @@ -0,0 +1,73 @@ +import { ErrorDisplay, LoadingDisplay, NotFound, PerspectiveEditor } from "@/features/perspectives/components/PerspectiveManagement"; +import { usePerspectiveModules } from "@/features/perspectives/hooks"; +import { Box } from "@chakra-ui/react"; +import { useMemo } from "react"; + +interface PerspectiveEditorPageProps { + readonly moduleName: string; + readonly perspectiveName: string; +} + +/** + * Page component that displays the details of a specific perspective + */ +export function PerspectiveEditorPage({ + moduleName, + perspectiveName +}: PerspectiveEditorPageProps) { + const { getModulePerspectives } = usePerspectiveModules(); + const { module, perspectives, isLoading, error } = getModulePerspectives(moduleName); + + // Find the specific perspective + const perspective = useMemo(() => { + if (!perspectives || perspectives.length === 0) return null; + + // First try direct match on the perspective name + let foundPerspective = perspectives.find(p => p.name === perspectiveName); + + // If not found, try to match with module prefix + if (!foundPerspective) { + const fullName = `${moduleName}/${perspectiveName}`; + foundPerspective = perspectives.find(p => p.name === fullName); + } + + // Also try matching just the last part of the name (for module-prefixed perspectives) + if (!foundPerspective) { + foundPerspective = perspectives.find(p => { + const parts = p.name.split('/'); + return parts[parts.length - 1] === perspectiveName; + }); + } + + return foundPerspective; + }, [perspectives, perspectiveName, moduleName]); + + // Handle loading state + if (isLoading) { + return ; + } + + // Handle error state + if (error) { + return ; + } + + // Handle case when module data is missing + if (!module) { + return ; + } + + // Handle case when perspective is not found + if (!perspective) { + return ; + } + + return ( + + + + ); +} \ No newline at end of file diff --git a/graphcap_studio/src/pages/perspectives/module/index.ts b/graphcap_studio/src/pages/perspectives/module/index.ts new file mode 100644 index 00000000..1a007be2 --- /dev/null +++ b/graphcap_studio/src/pages/perspectives/module/index.ts @@ -0,0 +1,2 @@ +export { ModulePage } from './ModulePage'; +export { PerspectiveEditorPage } from './PerspectiveEditorPage'; \ No newline at end of file diff --git a/graphcap_studio/src/routeTree.gen.ts b/graphcap_studio/src/routeTree.gen.ts index 48fd82b8..fc2a3bcf 100644 --- a/graphcap_studio/src/routeTree.gen.ts +++ b/graphcap_studio/src/routeTree.gen.ts @@ -10,123 +10,218 @@ // Import Routes -import { Route as rootRoute } from "./routes/__root"; -import { Route as AboutImport } from "./routes/about"; -import { Route as GalleryDatasetIdImport } from "./routes/gallery/$datasetId"; -import { Route as GalleryIndexImport } from "./routes/gallery/index"; -import { Route as IndexImport } from "./routes/index"; +import { Route as rootRoute } from './routes/__root' +import { Route as AboutImport } from './routes/about' +import { Route as IndexImport } from './routes/index' +import { Route as PerspectivesIndexImport } from './routes/perspectives/index' +import { Route as GalleryIndexImport } from './routes/gallery/index' +import { Route as GalleryDatasetIdImport } from './routes/gallery/$datasetId' +import { Route as PerspectivesModuleModuleNameImport } from './routes/perspectives/module/$moduleName' +import { Route as PerspectivesModuleModuleNamePerspectivePerspectiveNameImport } from './routes/perspectives/module/$moduleName/perspective/$perspectiveName' // Create/Update Routes const AboutRoute = AboutImport.update({ - id: "/about", - path: "/about", - getParentRoute: () => rootRoute, -} as any); + id: '/about', + path: '/about', + getParentRoute: () => rootRoute, +} as any) const IndexRoute = IndexImport.update({ - id: "/", - path: "/", - getParentRoute: () => rootRoute, -} as any); + id: '/', + path: '/', + getParentRoute: () => rootRoute, +} as any) + +const PerspectivesIndexRoute = PerspectivesIndexImport.update({ + id: '/perspectives/', + path: '/perspectives/', + getParentRoute: () => rootRoute, +} as any) const GalleryIndexRoute = GalleryIndexImport.update({ - id: "/gallery/", - path: "/gallery/", - getParentRoute: () => rootRoute, -} as any); + id: '/gallery/', + path: '/gallery/', + getParentRoute: () => rootRoute, +} as any) const GalleryDatasetIdRoute = GalleryDatasetIdImport.update({ - id: "/gallery/$datasetId", - path: "/gallery/$datasetId", - getParentRoute: () => rootRoute, -} as any); + id: '/gallery/$datasetId', + path: '/gallery/$datasetId', + getParentRoute: () => rootRoute, +} as any) + +const PerspectivesModuleModuleNameRoute = + PerspectivesModuleModuleNameImport.update({ + id: '/perspectives/module/$moduleName', + path: '/perspectives/module/$moduleName', + getParentRoute: () => rootRoute, + } as any) + +const PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute = + PerspectivesModuleModuleNamePerspectivePerspectiveNameImport.update({ + id: '/perspective/$perspectiveName', + path: '/perspective/$perspectiveName', + getParentRoute: () => PerspectivesModuleModuleNameRoute, + } as any) // Populate the FileRoutesByPath interface -declare module "@tanstack/react-router" { - interface FileRoutesByPath { - "/": { - id: "/"; - path: "/"; - fullPath: "/"; - preLoaderRoute: typeof IndexImport; - parentRoute: typeof rootRoute; - }; - "/about": { - id: "/about"; - path: "/about"; - fullPath: "/about"; - preLoaderRoute: typeof AboutImport; - parentRoute: typeof rootRoute; - }; - "/gallery/$datasetId": { - id: "/gallery/$datasetId"; - path: "/gallery/$datasetId"; - fullPath: "/gallery/$datasetId"; - preLoaderRoute: typeof GalleryDatasetIdImport; - parentRoute: typeof rootRoute; - }; - "/gallery/": { - id: "/gallery/"; - path: "/gallery"; - fullPath: "/gallery"; - preLoaderRoute: typeof GalleryIndexImport; - parentRoute: typeof rootRoute; - }; - } +declare module '@tanstack/react-router' { + interface FileRoutesByPath { + '/': { + id: '/' + path: '/' + fullPath: '/' + preLoaderRoute: typeof IndexImport + parentRoute: typeof rootRoute + } + '/about': { + id: '/about' + path: '/about' + fullPath: '/about' + preLoaderRoute: typeof AboutImport + parentRoute: typeof rootRoute + } + '/gallery/$datasetId': { + id: '/gallery/$datasetId' + path: '/gallery/$datasetId' + fullPath: '/gallery/$datasetId' + preLoaderRoute: typeof GalleryDatasetIdImport + parentRoute: typeof rootRoute + } + '/gallery/': { + id: '/gallery/' + path: '/gallery' + fullPath: '/gallery' + preLoaderRoute: typeof GalleryIndexImport + parentRoute: typeof rootRoute + } + '/perspectives/': { + id: '/perspectives/' + path: '/perspectives' + fullPath: '/perspectives' + preLoaderRoute: typeof PerspectivesIndexImport + parentRoute: typeof rootRoute + } + '/perspectives/module/$moduleName': { + id: '/perspectives/module/$moduleName' + path: '/perspectives/module/$moduleName' + fullPath: '/perspectives/module/$moduleName' + preLoaderRoute: typeof PerspectivesModuleModuleNameImport + parentRoute: typeof rootRoute + } + '/perspectives/module/$moduleName/perspective/$perspectiveName': { + id: '/perspectives/module/$moduleName/perspective/$perspectiveName' + path: '/perspective/$perspectiveName' + fullPath: '/perspectives/module/$moduleName/perspective/$perspectiveName' + preLoaderRoute: typeof PerspectivesModuleModuleNamePerspectivePerspectiveNameImport + parentRoute: typeof PerspectivesModuleModuleNameImport + } + } } // Create and export the route tree +interface PerspectivesModuleModuleNameRouteChildren { + PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute: typeof PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute +} + +const PerspectivesModuleModuleNameRouteChildren: PerspectivesModuleModuleNameRouteChildren = + { + PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute: + PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute, + } + +const PerspectivesModuleModuleNameRouteWithChildren = + PerspectivesModuleModuleNameRoute._addFileChildren( + PerspectivesModuleModuleNameRouteChildren, + ) + export interface FileRoutesByFullPath { - "/": typeof IndexRoute; - "/about": typeof AboutRoute; - "/gallery/$datasetId": typeof GalleryDatasetIdRoute; - "/gallery": typeof GalleryIndexRoute; + '/': typeof IndexRoute + '/about': typeof AboutRoute + '/gallery/$datasetId': typeof GalleryDatasetIdRoute + '/gallery': typeof GalleryIndexRoute + '/perspectives': typeof PerspectivesIndexRoute + '/perspectives/module/$moduleName': typeof PerspectivesModuleModuleNameRouteWithChildren + '/perspectives/module/$moduleName/perspective/$perspectiveName': typeof PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute } export interface FileRoutesByTo { - "/": typeof IndexRoute; - "/about": typeof AboutRoute; - "/gallery/$datasetId": typeof GalleryDatasetIdRoute; - "/gallery": typeof GalleryIndexRoute; + '/': typeof IndexRoute + '/about': typeof AboutRoute + '/gallery/$datasetId': typeof GalleryDatasetIdRoute + '/gallery': typeof GalleryIndexRoute + '/perspectives': typeof PerspectivesIndexRoute + '/perspectives/module/$moduleName': typeof PerspectivesModuleModuleNameRouteWithChildren + '/perspectives/module/$moduleName/perspective/$perspectiveName': typeof PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute } export interface FileRoutesById { - __root__: typeof rootRoute; - "/": typeof IndexRoute; - "/about": typeof AboutRoute; - "/gallery/$datasetId": typeof GalleryDatasetIdRoute; - "/gallery/": typeof GalleryIndexRoute; + __root__: typeof rootRoute + '/': typeof IndexRoute + '/about': typeof AboutRoute + '/gallery/$datasetId': typeof GalleryDatasetIdRoute + '/gallery/': typeof GalleryIndexRoute + '/perspectives/': typeof PerspectivesIndexRoute + '/perspectives/module/$moduleName': typeof PerspectivesModuleModuleNameRouteWithChildren + '/perspectives/module/$moduleName/perspective/$perspectiveName': typeof PerspectivesModuleModuleNamePerspectivePerspectiveNameRoute } export interface FileRouteTypes { - fileRoutesByFullPath: FileRoutesByFullPath; - fullPaths: "/" | "/about" | "/gallery/$datasetId" | "/gallery"; - fileRoutesByTo: FileRoutesByTo; - to: "/" | "/about" | "/gallery/$datasetId" | "/gallery"; - id: "__root__" | "/" | "/about" | "/gallery/$datasetId" | "/gallery/"; - fileRoutesById: FileRoutesById; + fileRoutesByFullPath: FileRoutesByFullPath + fullPaths: + | '/' + | '/about' + | '/gallery/$datasetId' + | '/gallery' + | '/perspectives' + | '/perspectives/module/$moduleName' + | '/perspectives/module/$moduleName/perspective/$perspectiveName' + fileRoutesByTo: FileRoutesByTo + to: + | '/' + | '/about' + | '/gallery/$datasetId' + | '/gallery' + | '/perspectives' + | '/perspectives/module/$moduleName' + | '/perspectives/module/$moduleName/perspective/$perspectiveName' + id: + | '__root__' + | '/' + | '/about' + | '/gallery/$datasetId' + | '/gallery/' + | '/perspectives/' + | '/perspectives/module/$moduleName' + | '/perspectives/module/$moduleName/perspective/$perspectiveName' + fileRoutesById: FileRoutesById } export interface RootRouteChildren { - IndexRoute: typeof IndexRoute; - AboutRoute: typeof AboutRoute; - GalleryDatasetIdRoute: typeof GalleryDatasetIdRoute; - GalleryIndexRoute: typeof GalleryIndexRoute; + IndexRoute: typeof IndexRoute + AboutRoute: typeof AboutRoute + GalleryDatasetIdRoute: typeof GalleryDatasetIdRoute + GalleryIndexRoute: typeof GalleryIndexRoute + PerspectivesIndexRoute: typeof PerspectivesIndexRoute + PerspectivesModuleModuleNameRoute: typeof PerspectivesModuleModuleNameRouteWithChildren } const rootRouteChildren: RootRouteChildren = { - IndexRoute: IndexRoute, - AboutRoute: AboutRoute, - GalleryDatasetIdRoute: GalleryDatasetIdRoute, - GalleryIndexRoute: GalleryIndexRoute, -}; + IndexRoute: IndexRoute, + AboutRoute: AboutRoute, + GalleryDatasetIdRoute: GalleryDatasetIdRoute, + GalleryIndexRoute: GalleryIndexRoute, + PerspectivesIndexRoute: PerspectivesIndexRoute, + PerspectivesModuleModuleNameRoute: + PerspectivesModuleModuleNameRouteWithChildren, +} export const routeTree = rootRoute - ._addFileChildren(rootRouteChildren) - ._addFileTypes(); + ._addFileChildren(rootRouteChildren) + ._addFileTypes() /* ROUTE_MANIFEST_START { @@ -137,7 +232,9 @@ export const routeTree = rootRoute "/", "/about", "/gallery/$datasetId", - "/gallery/" + "/gallery/", + "/perspectives/", + "/perspectives/module/$moduleName" ] }, "/": { @@ -151,6 +248,19 @@ export const routeTree = rootRoute }, "/gallery/": { "filePath": "gallery/index.tsx" + }, + "/perspectives/": { + "filePath": "perspectives/index.tsx" + }, + "/perspectives/module/$moduleName": { + "filePath": "perspectives/module/$moduleName.tsx", + "children": [ + "/perspectives/module/$moduleName/perspective/$perspectiveName" + ] + }, + "/perspectives/module/$moduleName/perspective/$perspectiveName": { + "filePath": "perspectives/module/$moduleName/perspective/$perspectiveName.tsx", + "parent": "/perspectives/module/$moduleName" } } } diff --git a/graphcap_studio/src/routes/perspectives/index.tsx b/graphcap_studio/src/routes/perspectives/index.tsx new file mode 100644 index 00000000..6b39f973 --- /dev/null +++ b/graphcap_studio/src/routes/perspectives/index.tsx @@ -0,0 +1,115 @@ +import { + SelectContent, + SelectItem, + SelectRoot, + SelectTrigger, + SelectValueText, +} from "@/components/ui/select"; +import { usePerspectiveModules } from "@/features/perspectives/hooks"; +import { Box, Flex, Heading, Spinner, Text } from "@chakra-ui/react"; +import { + type SelectValueChangeDetails, + createListCollection, +} from "@chakra-ui/react"; +import { Link, createFileRoute, useNavigate } from "@tanstack/react-router"; + +export const Route = createFileRoute("/perspectives/")({ + component: PerspectivesPage, +}); + +// Define the module item interface +interface ModuleItem { + label: string; + value: string; +} + +function PerspectivesPage() { + const { modules, isLoading, error } = usePerspectiveModules(); + const navigate = useNavigate(); + + // Create items collection for the select + const moduleItems: ModuleItem[] = + modules?.map((module) => ({ + label: `${module.name} (${module.perspectives.length} perspectives)`, + value: module.id, + })) || []; + + const collection = createListCollection({ + items: moduleItems, + }); + + // Handle module selection + const handleModuleChange = ( + details: SelectValueChangeDetails, + ) => { + if (details.value.length > 0) { + navigate({ + to: "/perspectives/module/$moduleName", + params: { moduleName: details.value[0] }, + }); + } + }; + + if (isLoading) { + return ( + + + Loading perspective modules... + + ); + } + + if (error) { + return ( + + + Error + {error.message} + + + ); + } + + return ( + + + Perspectives Management + + + + + + + Select a Module + + + Choose a module to browse its perspectives or manage configurations. + + + + + + + + + {moduleItems.map((item) => ( + + {item.label} + + ))} + + + + + + Total modules: {modules?.length || 0} + + + + ); +} diff --git a/graphcap_studio/src/routes/perspectives/module/$moduleName.tsx b/graphcap_studio/src/routes/perspectives/module/$moduleName.tsx new file mode 100644 index 00000000..bd0eb4d8 --- /dev/null +++ b/graphcap_studio/src/routes/perspectives/module/$moduleName.tsx @@ -0,0 +1,11 @@ +import { ModulePage } from "@/pages/perspectives"; +import { createFileRoute } from "@tanstack/react-router"; + +export const Route = createFileRoute("/perspectives/module/$moduleName")({ + component: ModulePageWrapper, +}); + +function ModulePageWrapper() { + const { moduleName } = Route.useParams(); + return ; +} diff --git a/graphcap_studio/src/routes/perspectives/module/$moduleName/perspective/$perspectiveName.tsx b/graphcap_studio/src/routes/perspectives/module/$moduleName/perspective/$perspectiveName.tsx new file mode 100644 index 00000000..4deee943 --- /dev/null +++ b/graphcap_studio/src/routes/perspectives/module/$moduleName/perspective/$perspectiveName.tsx @@ -0,0 +1,11 @@ +import { PerspectiveEditorPage } from "@/pages/perspectives"; +import { createFileRoute } from "@tanstack/react-router"; + +export const Route = createFileRoute("/perspectives/module/$moduleName/perspective/$perspectiveName")({ + component: PerspectiveEditorPageWrapper, +}); + +function PerspectiveEditorPageWrapper() { + const { moduleName, perspectiveName } = Route.useParams(); + return ; +} \ No newline at end of file diff --git a/pyproject.toml b/pyproject.toml index f029d36c..99491de5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -49,7 +49,7 @@ exclude = [ ] # Source directories -src = ["./src"] +src = ["./servers/inference_server"] # Same as Black line-length = 120 @@ -61,7 +61,7 @@ target-version = "py311" [tool.ruff.lint] # Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default select = ["E", "F", "W", "I001"] -ignore = [] +ignore = ["W293"] # Allow fix for all enabled rules (when `--fix`) is provided fixable = ["ALL"] diff --git a/servers/inference_server/graphcap/perspectives/__init__.py b/servers/inference_server/graphcap/perspectives/__init__.py index 189aba86..fffa256b 100644 --- a/servers/inference_server/graphcap/perspectives/__init__.py +++ b/servers/inference_server/graphcap/perspectives/__init__.py @@ -1,36 +1,58 @@ """ # SPDX-License-Identifier: Apache-2.0 -Caption Perspectives Module +Perspectives Package -Collection of different perspectives for analyzing and captioning images. -Each perspective provides a unique way of understanding and describing visual content. - -Perspectives are defined in JSON configuration files in workspace/config/perspectives directory. -Each perspective configuration specifies its analysis approach and prompting strategy. -Perspectives can be organized in subdirectories for better organization. +Provides utilities for working with different perspectives/views of data. """ from pathlib import Path -from typing import Dict, List +from typing import Any, Dict, List, Optional from loguru import logger from .constants import WORKSPACE_PERSPECTIVES_DIR -from .perspective_loader import JsonPerspectiveProcessor, load_all_perspectives +from .perspective_loader import ( + # Classes + JsonPerspectiveProcessor, + ModuleConfig, + PerspectiveConfig, + PerspectiveModule, + PerspectiveSettings, + # Models + SchemaField, + get_all_modules, + # Functions + get_perspective_directories, + load_all_perspectives, + load_module_settings, + load_perspective_config, + load_perspective_from_json, +) # Load JSON-based perspectives from workspace config _json_perspectives: Dict[str, JsonPerspectiveProcessor] = {} +_modules: Dict[str, PerspectiveModule] = {} + +# Get all perspective directories +perspective_dirs = get_perspective_directories() +logger.info(f"Looking for perspectives in: {perspective_dirs}") + +# Load module settings +settings = load_module_settings() + +# Load perspectives organized by modules +try: + _modules = get_all_modules(perspective_dirs, settings) + _json_perspectives = load_all_perspectives(perspective_dirs, settings) -logger.info(f"Looking for perspectives in: {WORKSPACE_PERSPECTIVES_DIR}") -logger.info(f"Directory exists: {WORKSPACE_PERSPECTIVES_DIR.exists()}") -if WORKSPACE_PERSPECTIVES_DIR.exists(): - logger.info(f"Directory contents: {list(WORKSPACE_PERSPECTIVES_DIR.glob('*.json'))}") - _json_perspectives = load_all_perspectives(WORKSPACE_PERSPECTIVES_DIR) - logger.info( - f"Loaded {len(_json_perspectives)} perspectives from workspace configurations: {list(_json_perspectives.keys())}" - ) -else: - logger.warning(f"No perspective configurations found at {WORKSPACE_PERSPECTIVES_DIR}") + # Log loaded modules and perspectives + enabled_modules = [name for name, module in _modules.items() if module.enabled] + logger.info(f"Loaded {len(_modules)} modules: {list(_modules.keys())}") + logger.info(f"Enabled modules: {enabled_modules}") + logger.info(f"Loaded {len(_json_perspectives)} perspectives: {list(_json_perspectives.keys())}") +except Exception as e: + logger.error(f"Error loading perspectives: {str(e)}") + logger.exception(e) def get_perspective(perspective_name: str, **kwargs): @@ -52,7 +74,7 @@ def get_perspective(perspective_name: str, **kwargs): available = list(_json_perspectives.keys()) logger.error(f"Perspective {perspective_name} not found. Available: {available}") - logger.error(f"Perspectives directory contents: {list(WORKSPACE_PERSPECTIVES_DIR.glob('*.json'))}") + logger.error(f"Perspectives directories: {perspective_dirs}") raise ValueError(f"Unknown perspective: {perspective_name}. Available perspectives: {available}") @@ -66,6 +88,103 @@ def get_perspective_list() -> List[str]: return list(_json_perspectives.keys()) +def get_perspective_modules() -> Dict[str, PerspectiveModule]: + """ + Get all perspective modules. + + Returns: + Dictionary mapping module names to module objects + """ + return _modules + + +def get_module_perspectives(module_name: str) -> Dict[str, JsonPerspectiveProcessor]: + """ + Get all perspectives in a specific module. + + Args: + module_name: The name of the module + + Returns: + Dictionary mapping perspective names to processors + + Raises: + ValueError: If the module name is unknown + """ + if module_name in _modules: + return _modules[module_name].get_perspectives() + + available = list(_modules.keys()) + raise ValueError(f"Unknown module: {module_name}. Available modules: {available}") + + +def get_perspectives_by_tag(tag: str) -> Dict[str, JsonPerspectiveProcessor]: + """ + Get all perspectives with a specific tag. + + Args: + tag: The tag to filter by + + Returns: + Dictionary mapping perspective names to processors + """ + result = {} + for name, perspective in _json_perspectives.items(): + if tag in perspective.tags: + result[name] = perspective + return result + + +def get_perspectives_by_tags(tags: List[str], match_all: bool = False) -> Dict[str, JsonPerspectiveProcessor]: + """ + Get all perspectives matching the specified tags. + + Args: + tags: List of tags to filter by + match_all: If True, perspective must have all tags; if False, any matching tag is sufficient + + Returns: + Dictionary mapping perspective names to processors + """ + result = {} + for name, perspective in _json_perspectives.items(): + if match_all: + # All tags must match + if all(tag in perspective.tags for tag in tags): + result[name] = perspective + else: + # Any tag match is sufficient + if any(tag in perspective.tags for tag in tags): + result[name] = perspective + return result + + +def toggle_module(module_name: str, enabled: bool) -> None: + """ + Toggle a module on or off. + + Args: + module_name: The name of the module to toggle + enabled: Whether to enable or disable the module + + Raises: + ValueError: If the module name is unknown + """ + if module_name not in _modules: + available = list(_modules.keys()) + raise ValueError(f"Unknown module: {module_name}. Available modules: {available}") + + _modules[module_name].toggle(enabled) + + # Reload perspectives to reflect changes + global _json_perspectives + _json_perspectives = {} + for name, module in _modules.items(): + if module.enabled: + for perspective_name, perspective in module.perspectives.items(): + _json_perspectives[perspective_name] = perspective + + def get_synthesizer() -> JsonPerspectiveProcessor: """ Get the synthesized caption processor. @@ -82,9 +201,58 @@ def get_synthesizer() -> JsonPerspectiveProcessor: raise ValueError("Synthesized caption perspective not found in JSON configurations") +def get_non_deprecated_perspectives() -> Dict[str, JsonPerspectiveProcessor]: + """ + Get all non-deprecated perspectives. + + Returns: + Dictionary mapping perspective names to processors + """ + return {name: p for name, p in _json_perspectives.items() if not p.is_deprecated} + + +def get_perspective_metadata() -> List[Dict[str, Any]]: + """ + Get metadata for all perspectives. + + Returns: + List of dictionaries with perspective metadata + """ + result = [] + for name, perspective in _json_perspectives.items(): + result.append( + { + "name": name, + "display_name": perspective.display_name, + "version": perspective.version, + "module": perspective.module_name, + "tags": perspective.tags, + "description": perspective.description, + "deprecated": perspective.is_deprecated, + "replacement": perspective.replacement, + "priority": perspective.priority, + } + ) + + # Sort by priority (lowest first) + result.sort(key=lambda x: x["priority"]) + return result + + __all__ = [ + # Models + "SchemaField", + "PerspectiveConfig", + "ModuleConfig", + "PerspectiveSettings", + # Classes "JsonPerspectiveProcessor", - "get_perspective", - "get_perspective_list", - "get_synthesizer", + "PerspectiveModule", + # Functions + "get_perspective_directories", + "load_perspective_config", + "load_perspective_from_json", + "get_all_modules", + "load_all_perspectives", + "load_module_settings", ] diff --git a/servers/inference_server/graphcap/perspectives/loaders/__init__.py b/servers/inference_server/graphcap/perspectives/loaders/__init__.py new file mode 100644 index 00000000..779e0628 --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/loaders/__init__.py @@ -0,0 +1,20 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Perspective Loaders Package + +Contains utilities for loading perspectives from various sources. +""" + +from .directory import get_perspective_directories +from .json_file import load_perspective_config, load_perspective_from_json +from .modules import get_all_modules, load_all_perspectives +from .settings import load_module_settings + +__all__ = [ + "get_perspective_directories", + "load_perspective_config", + "load_perspective_from_json", + "get_all_modules", + "load_all_perspectives", + "load_module_settings", +] diff --git a/servers/inference_server/graphcap/perspectives/loaders/directory.py b/servers/inference_server/graphcap/perspectives/loaders/directory.py new file mode 100644 index 00000000..76d9d583 --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/loaders/directory.py @@ -0,0 +1,28 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Directory Loader Module + +Provides functions for finding directories that contain perspective files. +""" + +from pathlib import Path +from typing import List + + +def get_perspective_directories() -> List[Path]: + """Get all directories where perspectives can be found.""" + from ..constants import WORKSPACE_PERSPECTIVES_DIR + + dirs = [WORKSPACE_PERSPECTIVES_DIR] + + # Check for local perspective directory in user's home + local_dir = Path.home() / ".graphcap" / "perspectives" + if local_dir.exists(): + dirs.append(local_dir) + + # Also check for "perspective_library" directory that seems to be used + perspective_library = Path("/workspace") / "perspective_library" + if perspective_library.exists(): + dirs.append(perspective_library) + + return dirs diff --git a/servers/inference_server/graphcap/perspectives/loaders/json_file.py b/servers/inference_server/graphcap/perspectives/loaders/json_file.py new file mode 100644 index 00000000..316c2ebd --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/loaders/json_file.py @@ -0,0 +1,46 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +JSON File Loader + +Provides utilities for loading perspective configurations from JSON files. +""" + +import json +from pathlib import Path +from typing import Union + +from ..models import PerspectiveConfig +from ..processor import JsonPerspectiveProcessor + + +def load_perspective_config(config_path: Union[str, Path]) -> PerspectiveConfig: + """Load a perspective configuration from a JSON file.""" + with open(config_path, "r") as f: + config_data = json.load(f) + + return PerspectiveConfig(**config_data) + + +def load_perspective_from_json(config_path: Union[str, Path]) -> JsonPerspectiveProcessor: + """Load and instantiate a perspective from a JSON configuration file.""" + config = load_perspective_config(config_path) + return JsonPerspectiveProcessor(config) + + +def prepare_module_from_file_path(config_data: dict, json_path: Path, config_dir: Path) -> None: + """ + Prepare module information from file path if not specified in config. + + Args: + config_data: The perspective configuration data + json_path: Path to the JSON file + config_dir: Base directory for perspectives + """ + # If module is not specified in the file, use directory name as module + if "module" not in config_data: + # Use parent directory name as module, or "default" if at root + rel_path = json_path.relative_to(config_dir) + if len(rel_path.parts) > 1: + config_data["module"] = rel_path.parts[0] + else: + config_data["module"] = "default" diff --git a/servers/inference_server/graphcap/perspectives/loaders/modules.py b/servers/inference_server/graphcap/perspectives/loaders/modules.py new file mode 100644 index 00000000..52e8c727 --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/loaders/modules.py @@ -0,0 +1,303 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Module Loader + +Provides utilities for loading perspective modules and organizing +perspectives by module. +""" + +import json +from pathlib import Path +from typing import Dict, List, Optional, Set, Tuple + +from loguru import logger + +from ..models import PerspectiveConfig, PerspectiveSettings +from ..module import PerspectiveModule +from ..processor import JsonPerspectiveProcessor +from .directory import get_perspective_directories +from .json_file import prepare_module_from_file_path +from .settings import load_module_settings + + +def find_json_files(config_dir: Path) -> List[Path]: + """ + Find all JSON files in a directory recursively. + + Args: + config_dir: Directory to search + + Returns: + List of paths to JSON files + """ + if not config_dir.exists(): + logger.warning(f"Perspective directory does not exist: {config_dir}") + return [] + + logger.info(f"Scanning for JSON files in: {config_dir}") + json_files = list(config_dir.rglob("*.json")) + logger.info(f"Found {len(json_files)} JSON files in {config_dir}") + return json_files + + +def load_json_file(json_path: Path) -> Optional[dict]: + """ + Load a JSON file safely. + + Args: + json_path: Path to the JSON file + + Returns: + Loaded JSON data as a dictionary, or None if loading failed + """ + logger.info(f"Attempting to load perspective from: {json_path}") + try: + with open(json_path, "r", encoding="utf-8") as f: + config_data = json.load(f) + logger.info(f"Successfully read JSON from: {json_path}") + return config_data + except Exception as e: + logger.error(f"Failed to load perspective from {json_path}: {str(e)}") + logger.exception(e) # This will log the full stack trace + return None + + +def validate_perspective_name(config_data: dict, json_path: Path, processed_names: Set[str]) -> Optional[str]: + """ + Validate the perspective name and check for duplicates. + + Args: + config_data: The perspective configuration data + json_path: Path to the JSON file + processed_names: Set of already processed perspective names + + Returns: + The perspective name if valid, None otherwise + """ + name = config_data.get("name") + if not name: + logger.warning(f"Skipping {json_path}: Missing 'name' field") + return None + + # Check for duplicates - later files override earlier ones + if name in processed_names: + logger.info(f"Duplicate perspective name '{name}' found in {json_path}, overriding previous definition") + + return name + + +def create_perspective_processor( + config_data: dict, json_path: Path, config_dir: Path +) -> Optional[JsonPerspectiveProcessor]: + """ + Create a perspective processor from config data. + + Args: + config_data: The perspective configuration data + json_path: Path to the JSON file + config_dir: Base directory for perspectives + + Returns: + JsonPerspectiveProcessor instance, or None if creation failed + """ + try: + # Prepare module information from file path if not specified + prepare_module_from_file_path(config_data, json_path, config_dir) + + # Create perspective config and processor + config = PerspectiveConfig(**config_data) + perspective = JsonPerspectiveProcessor(config) + + return perspective + except Exception as e: + logger.error(f"Failed to create PerspectiveConfig from {json_path}: {str(e)}") + return None + + +def prepare_module(module_name: str, settings: PerspectiveSettings) -> PerspectiveModule: + """ + Prepare a module with appropriate settings. + + Args: + module_name: The name of the module + settings: Perspective settings + + Returns: + New PerspectiveModule instance + """ + # Create module with default settings or from config + display_name = module_name.replace("_", " ").title() + if module_name in settings.modules and settings.modules[module_name].display_name: + module_display_name = settings.modules[module_name].display_name + # Ensure display_name is not None + if module_display_name is not None: + display_name = module_display_name + + enabled = True + if module_name in settings.modules: + enabled = settings.modules[module_name].enabled + + return PerspectiveModule( + name=module_name, + display_name=display_name, + enabled=enabled, + ) + + +def process_json_file( + json_path: Path, + config_dir: Path, + modules: Dict[str, PerspectiveModule], + processed_names: Set[str], + settings: PerspectiveSettings, +) -> Tuple[Optional[str], Optional[JsonPerspectiveProcessor]]: + """ + Process a single JSON file and create perspective processor. + + Args: + json_path: Path to the JSON file + config_dir: Base directory for perspectives + modules: Dictionary of existing modules + processed_names: Set of already processed perspective names + settings: Perspective settings + + Returns: + Tuple of (perspective_name, perspective_processor) or (None, None) if processing failed + """ + # Load the JSON file + config_data = load_json_file(json_path) + if not config_data: + return None, None + + # Validate the perspective name + name = validate_perspective_name(config_data, json_path, processed_names) + if not name: + return None, None + + # Create the perspective processor + perspective = create_perspective_processor(config_data, json_path, config_dir) + if not perspective: + return None, None + + # Get or create the module + module_name = perspective.module_name + if module_name not in modules: + modules[module_name] = prepare_module(module_name, settings) + + # Add perspective to module + modules[module_name].add_perspective(perspective) + + logger.info(f"Successfully loaded perspective '{name}' from {json_path} in module '{module_name}'") + return name, perspective + + +def process_config_directory( + config_dir: Path, + modules: Dict[str, PerspectiveModule], + processed_names: Set[str], + perspectives_by_name: Dict[str, JsonPerspectiveProcessor], + settings: PerspectiveSettings, +) -> None: + """ + Process all JSON files in a config directory. + + Args: + config_dir: Directory to search for JSON files + modules: Dictionary of modules to update + processed_names: Set of processed perspective names + perspectives_by_name: Dictionary of perspectives by name + settings: Perspective settings + """ + # Find all JSON files + json_files = find_json_files(config_dir) + + # Process each JSON file + for json_path in json_files: + name, perspective = process_json_file(json_path, config_dir, modules, processed_names, settings) + + if name and perspective: + processed_names.add(name) + perspectives_by_name[name] = perspective + + +def get_all_modules( + config_dirs: Optional[List[Path]] = None, + settings: Optional[PerspectiveSettings] = None, +) -> Dict[str, PerspectiveModule]: + """ + Load all perspective configurations and return organized by modules. + + Args: + config_dirs: List of directories to search for perspective JSON files + settings: Optional settings controlling which modules are enabled + + Returns: + Dictionary mapping module names to module objects + """ + if config_dirs is None: + config_dirs = get_perspective_directories() + + if settings is None: + settings = load_module_settings() + + # Track loaded modules and processed perspective names + modules: Dict[str, PerspectiveModule] = {} + processed_names: Set[str] = set() + + # Process each config directory + for config_dir in config_dirs: + process_config_directory( + config_dir, + modules, + processed_names, + {}, # Dummy perspectives_by_name - not used by get_all_modules + settings, + ) + + logger.info(f"Total modules loaded: {len(modules)}") + logger.info(f"Module names: {list(modules.keys())}") + + return modules + + +def load_all_perspectives( + config_dirs: Optional[List[Path]] = None, + settings: Optional[PerspectiveSettings] = None, +) -> Dict[str, JsonPerspectiveProcessor]: + """ + Load all perspective configurations from directories and organize them by modules. + + Args: + config_dirs: List of directories to search for perspective JSON files + settings: Optional settings controlling which modules are enabled + + Returns: + Dictionary mapping perspective names to their processors + """ + if config_dirs is None: + config_dirs = get_perspective_directories() + + if settings is None: + settings = load_module_settings() + + # Track loaded modules and perspectives + modules: Dict[str, PerspectiveModule] = {} + perspectives_by_name: Dict[str, JsonPerspectiveProcessor] = {} + processed_names: Set[str] = set() + + # Process each config directory + for config_dir in config_dirs: + process_config_directory(config_dir, modules, processed_names, perspectives_by_name, settings) + + # Return flat dictionary for backward compatibility + result = {} + for module_name, module in modules.items(): + if module.enabled: + for name, perspective in module.perspectives.items(): + result[name] = perspective + + logger.info(f"Total perspectives loaded: {len(result)}") + logger.info(f"Loaded perspective names: {list(result.keys())}") + logger.info(f"Modules: {list(modules.keys())}") + + return result diff --git a/servers/inference_server/graphcap/perspectives/loaders/settings.py b/servers/inference_server/graphcap/perspectives/loaders/settings.py new file mode 100644 index 00000000..06d7ec37 --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/loaders/settings.py @@ -0,0 +1,43 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Perspective Settings Loader + +Provides utilities for loading perspective module settings from configuration files. +""" + +import json +from pathlib import Path +from typing import Union + +from loguru import logger + +from ..models import PerspectiveSettings + + +def load_module_settings(settings_path: Union[str, Path, None] = None) -> PerspectiveSettings: + """ + Load module settings from a JSON file. + + Args: + settings_path: Path to the settings file, or None to use the default + + Returns: + PerspectiveSettings model with loaded settings or defaults + """ + default_settings = PerspectiveSettings() + + if settings_path is None: + # Default settings path + settings_path = Path("/workspace") / "config" / "perspective_settings.json" + + if not Path(settings_path).exists(): + logger.info(f"No module settings found at {settings_path}, using defaults") + return default_settings + + try: + with open(settings_path, "r") as f: + settings_data = json.load(f) + return PerspectiveSettings(**settings_data) + except Exception as e: + logger.error(f"Failed to load module settings from {settings_path}: {str(e)}") + return default_settings diff --git a/servers/inference_server/graphcap/perspectives/models.py b/servers/inference_server/graphcap/perspectives/models.py new file mode 100644 index 00000000..09631863 --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/models.py @@ -0,0 +1,58 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Perspective Models Module + +Contains the data models used for perspective configuration and management. +""" + +from typing import Any, Dict, List, Optional, Union + +from pydantic import BaseModel, Field +from typing_extensions import override + +from .base import PerspectiveData + + +class SchemaField(BaseModel): + """Definition of a field in a perspective schema.""" + + name: str + type: Union[str, Dict[str, Any]] # Changed from str to accept either string or dictionary + description: str + is_list: bool = False + is_complex: bool = False + fields: Optional[List['SchemaField']] = None + + +class PerspectiveConfig(BaseModel): + """Configuration for a perspective loaded from JSON.""" + + name: str + display_name: str + version: str + prompt: str + schema_fields: List[SchemaField] + table_columns: List[Dict[str, str]] + context_template: str + + # Metadata fields + module: str = Field(default="default", description="Module this perspective belongs to") + tags: List[str] = Field(default_factory=list, description="Tags for categorizing perspectives") + description: str = Field(default="", description="Detailed description of the perspective") + deprecated: bool = Field(default=False, description="Whether this perspective is deprecated") + replacement: Optional[str] = Field(default=None, description="Name of perspective that replaces this one") + priority: int = Field(default=100, description="Priority for sorting (lower is higher priority)") + + +class ModuleConfig(BaseModel): + """Configuration for module settings.""" + + enabled: bool = True + display_name: Optional[str] = None + + +class PerspectiveSettings(BaseModel): + """Settings for perspectives configuration.""" + + modules: Dict[str, ModuleConfig] = {} + local_override: bool = True diff --git a/servers/inference_server/graphcap/perspectives/module.py b/servers/inference_server/graphcap/perspectives/module.py new file mode 100644 index 00000000..755d3774 --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/module.py @@ -0,0 +1,38 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Perspective Module + +Provides module functionality for organizing perspectives into groups +that can be enabled or disabled. +""" + +from typing import Dict + +from pydantic import BaseModel + +from .processor import JsonPerspectiveProcessor + + +class PerspectiveModule(BaseModel): + """Represents a module of related perspectives.""" + + name: str + display_name: str + enabled: bool = True + perspectives: Dict[str, JsonPerspectiveProcessor] = {} + + model_config = {"arbitrary_types_allowed": True} + + def toggle(self, enabled: bool) -> None: + """Toggle this module on or off.""" + self.enabled = enabled + + def get_perspectives(self) -> Dict[str, JsonPerspectiveProcessor]: + """Get all perspectives in this module if enabled.""" + if not self.enabled: + return {} + return self.perspectives + + def add_perspective(self, perspective: JsonPerspectiveProcessor) -> None: + """Add a perspective to this module.""" + self.perspectives[perspective.config.name] = perspective diff --git a/servers/inference_server/graphcap/perspectives/perspective_loader.py b/servers/inference_server/graphcap/perspectives/perspective_loader.py index 1ef870d1..c44a5e23 100644 --- a/servers/inference_server/graphcap/perspectives/perspective_loader.py +++ b/servers/inference_server/graphcap/perspectives/perspective_loader.py @@ -6,251 +6,43 @@ This module reduces code duplication by allowing perspectives to be defined declaratively. """ -import json -from pathlib import Path -from typing import Any, Dict, List, Union - -from loguru import logger -from pydantic import BaseModel, Field, create_model -from rich.table import Table -from typing_extensions import override - -from .base import BasePerspective, PerspectiveData - - -class SchemaField(BaseModel): - """Definition of a field in a perspective schema.""" - - name: str - type: str - description: str - is_list: bool = False - - -class PerspectiveConfig(BaseModel): - """Configuration for a perspective loaded from JSON.""" - - name: str - display_name: str - version: str - prompt: str - schema_fields: List[SchemaField] - table_columns: List[Dict[str, str]] - context_template: str - - -class JsonPerspectiveProcessor(BasePerspective): - """ - Processor for perspectives defined in JSON configuration files. - - This class dynamically creates schema models and implements methods - based on the configuration provided in the JSON file. - """ - - def __init__(self, config: PerspectiveConfig): - """Initialize the processor with a JSON configuration.""" - self.config = config - self.display_name = config.display_name - - # Dynamically create schema model from JSON definition - schema_fields = {} - for field in config.schema_fields: - field_type = self._get_field_type(field.type, field.is_list) - schema_fields[field.name] = (field_type, Field(description=field.description)) - - # Create the schema model dynamically - schema_class = create_model(f"{config.name.capitalize()}Schema", __base__=PerspectiveData, **schema_fields) - - super().__init__( - config_name=config.name, - version=config.version, - prompt=config.prompt, - schema=schema_class, - ) - - def _get_field_type(self, type_name: str, is_list: bool) -> Any: - """Convert string type name to actual Python type.""" - type_map = { - "str": str, - "int": int, - "float": float, - "bool": bool, - } - - base_type = type_map.get(type_name, str) - if is_list: - return List[base_type] - return base_type - - @override - def create_rich_table(self, caption_data: Dict[str, Any]) -> Table: - """Create Rich table for displaying caption data based on JSON config.""" - result = caption_data["parsed"] - - table = Table(show_header=True, header_style="bold magenta", expand=True) - - # Add columns based on configuration - for column in self.config.table_columns: - table.add_column(column["name"], style=column.get("style", "default")) - - # Add rows based on schema fields - rows = [] - for field in self.config.schema_fields: - field_name = field.name - field_value = result.get(field_name, "") - - # Format list values if needed - if field.is_list and isinstance(field_value, list): - field_value = "\n".join(f"• {item}" for item in field_value) - - rows.append(field_value) - - table.add_row(*rows) - logger.info(f"Generated {self.display_name} for {caption_data['filename']}") - return table - - @override - def write_outputs(self, job_dir: Path, caption_data: Dict[str, Any]) -> None: - """Write perspective outputs to the job directory.""" - result = caption_data["parsed"] - - # Create output dictionary - output = {"filename": caption_data["filename"], "analysis": {}} - - # Add each field from the parsed result - for field in self.config.schema_fields: - field_name = field.name - output["analysis"][field_name] = result.get(field_name, "") - - # Write to JSON file - response_file = job_dir / f"{self.config.name}_response.json" - with response_file.open("a") as f: - json.dump(output, f, indent=2) - f.write("\n") # Separate entries by newline - - @override - def to_table(self, caption_data: Dict[str, Any]) -> Dict[str, Any]: - """Convert perspective data to a flat dictionary.""" - result = caption_data.get("parsed", {}) - - # Check for error - if "error" in result: - return { - "filename": caption_data.get("filename", "unknown"), - "error": result["error"], - } - - # Create output dictionary with filename - output = {"filename": caption_data.get("filename", "unknown")} - - # Add each field from the parsed result - for field in self.config.schema_fields: - field_name = field.name - field_value = result.get(field_name, "") - - # Format list values if needed - if field.is_list and isinstance(field_value, list): - field_value = ", ".join(field_value) - - output[field_name] = field_value - - return output - - @override - def to_context(self, caption_data: Dict[str, Any]) -> str: - """Convert perspective data to a context string using the template.""" - result = caption_data.get("parsed", {}) - - # Replace placeholders in the template with actual values - context = self.config.context_template - for field in self.config.schema_fields: - field_name = field.name - field_value = result.get(field_name, "") - - # Format list values if needed - if field.is_list and isinstance(field_value, list): - field_value = ", ".join(field_value) - - # Replace placeholder with value - placeholder = f"{{{field_name}}}" - context = context.replace(placeholder, str(field_value)) - - return context - - @property - def config_name(self) -> str: - """Get the configuration name.""" - return self.config.name - - @property - def version(self) -> str: - """Get the perspective version.""" - return self.config.version - - -def load_perspective_config(config_path: Union[str, Path]) -> PerspectiveConfig: - """Load a perspective configuration from a JSON file.""" - with open(config_path, "r") as f: - config_data = json.load(f) - - return PerspectiveConfig(**config_data) - - -def load_perspective_from_json(config_path: Union[str, Path]) -> JsonPerspectiveProcessor: - """Load and instantiate a perspective from a JSON configuration file.""" - config = load_perspective_config(config_path) - return JsonPerspectiveProcessor(config) - - -def load_all_perspectives(config_dir: Union[str, Path]) -> Dict[str, JsonPerspectiveProcessor]: - """ - Load all perspective configurations from a directory and its subdirectories. - - Args: - config_dir: Path to the directory containing perspective JSON configurations - - Returns: - Dictionary mapping perspective names to their processors - """ - config_dir = Path(config_dir) - perspectives: Dict[str, JsonPerspectiveProcessor] = {} - - logger.info(f"Scanning for JSON files in: {config_dir}") - # Recursively find all JSON files - json_files = list(config_dir.rglob("*.json")) - logger.info(f"Found JSON files: {json_files}") - - for json_path in json_files: - logger.info(f"Attempting to load perspective from: {json_path}") - try: - with open(json_path, "r", encoding="utf-8") as f: - config_data = json.load(f) - logger.info(f"Successfully read JSON from: {json_path}") - - # Each perspective must have a unique name - name = config_data.get("name") - if not name: - logger.warning(f"Skipping {json_path}: Missing 'name' field") - continue - - if name in perspectives: - logger.warning(f"Duplicate perspective name '{name}' found in {json_path}, skipping") - continue - - # Convert the dictionary to a PerspectiveConfig model - try: - config = PerspectiveConfig(**config_data) - perspectives[name] = JsonPerspectiveProcessor(config) - logger.info(f"Successfully loaded perspective '{name}' from {json_path}") - except Exception as e: - logger.error(f"Failed to create PerspectiveConfig from {json_path}: {str(e)}") - continue - - except Exception as e: - logger.error(f"Failed to load perspective from {json_path}: {str(e)}") - logger.exception(e) # This will log the full stack trace - continue - - logger.info(f"Total perspectives loaded: {len(perspectives)}") - logger.info(f"Loaded perspective names: {list(perspectives.keys())}") - return perspectives +# Re-export models for backward compatibility +# Re-export loader functions +from .loaders import ( + get_all_modules, + get_perspective_directories, + load_all_perspectives, + load_module_settings, + load_perspective_config, + load_perspective_from_json, +) +from .models import ( + ModuleConfig, + PerspectiveConfig, + PerspectiveSettings, + SchemaField, +) + +# Re-export module +from .module import PerspectiveModule + +# Re-export processor +from .processor import JsonPerspectiveProcessor + +__all__ = [ + # Models + "SchemaField", + "PerspectiveConfig", + "ModuleConfig", + "PerspectiveSettings", + # Classes + "JsonPerspectiveProcessor", + "PerspectiveModule", + # Functions + "get_perspective_directories", + "load_perspective_config", + "load_perspective_from_json", + "get_all_modules", + "load_all_perspectives", + "load_module_settings", +] diff --git a/servers/inference_server/graphcap/perspectives/processor.py b/servers/inference_server/graphcap/perspectives/processor.py new file mode 100644 index 00000000..482c4d9c --- /dev/null +++ b/servers/inference_server/graphcap/perspectives/processor.py @@ -0,0 +1,198 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Perspective Processor Module + +Provides implementation of the perspective processor class that handles +dynamically created schema models and implements conversion methods. +""" + +import json +from pathlib import Path +from typing import Any, Dict, List, Optional + +from loguru import logger +from pydantic import Field, create_model +from rich.table import Table +from typing_extensions import override + +from .base import BasePerspective, PerspectiveData +from .models import PerspectiveConfig + + +class JsonPerspectiveProcessor(BasePerspective): + """ + Processor for perspectives defined in JSON configuration files. + + This class dynamically creates schema models and implements methods + based on the configuration provided in the JSON file. + """ + + def __init__(self, config: PerspectiveConfig): + """Initialize the processor with a JSON configuration.""" + self.config = config + self.display_name = config.display_name + + # Dynamically create schema model from JSON definition + schema_fields = {} + for field in config.schema_fields: + field_type = self._get_field_type(field.type, field.is_list) + schema_fields[field.name] = (field_type, Field(description=field.description)) + + # Create the schema model dynamically + schema_class = create_model(f"{config.name.capitalize()}Schema", __base__=PerspectiveData, **schema_fields) + + super().__init__( + config_name=config.name, + version=config.version, + prompt=config.prompt, + schema=schema_class, + ) + + def _get_field_type(self, type_name: str, is_list: bool) -> Any: + """Convert string type name to actual Python type.""" + type_map = { + "str": str, + "int": int, + "float": float, + "bool": bool, + } + + base_type = type_map.get(type_name, str) + if is_list: + return List[base_type] + return base_type + + @override + def create_rich_table(self, caption_data: Dict[str, Any]) -> Table: + """Create Rich table for displaying caption data based on JSON config.""" + result = caption_data["parsed"] + + table = Table(show_header=True, header_style="bold magenta", expand=True) + + # Add columns based on configuration + for column in self.config.table_columns: + table.add_column(column["name"], style=column.get("style", "default")) + + # Add rows based on schema fields + rows = [] + for field in self.config.schema_fields: + field_name = field.name + field_value = result.get(field_name, "") + + # Format list values if needed + if field.is_list and isinstance(field_value, list): + field_value = "\n".join(f"• {item}" for item in field_value) + + rows.append(field_value) + + table.add_row(*rows) + logger.info(f"Generated {self.display_name} for {caption_data['filename']}") + return table + + @override + def write_outputs(self, job_dir: Path, caption_data: Dict[str, Any]) -> None: + """Write perspective outputs to the job directory.""" + result = caption_data["parsed"] + + # Create output dictionary + output = {"filename": caption_data["filename"], "analysis": {}} + + # Add each field from the parsed result + for field in self.config.schema_fields: + field_name = field.name + output["analysis"][field_name] = result.get(field_name, "") + + # Write to JSON file + response_file = job_dir / f"{self.config.name}_response.json" + with response_file.open("a") as f: + json.dump(output, f, indent=2) + f.write("\n") # Separate entries by newline + + @override + def to_table(self, caption_data: Dict[str, Any]) -> Dict[str, Any]: + """Convert perspective data to a flat dictionary.""" + result = caption_data.get("parsed", {}) + + # Check for error + if "error" in result: + return { + "filename": caption_data.get("filename", "unknown"), + "error": result["error"], + } + + # Create output dictionary with filename + output = {"filename": caption_data.get("filename", "unknown")} + + # Add each field from the parsed result + for field in self.config.schema_fields: + field_name = field.name + field_value = result.get(field_name, "") + + # Format list values if needed + if field.is_list and isinstance(field_value, list): + field_value = ", ".join(field_value) + + output[field_name] = field_value + + return output + + @override + def to_context(self, caption_data: Dict[str, Any]) -> str: + """Convert perspective data to a context string using the template.""" + result = caption_data.get("parsed", {}) + + # Replace placeholders in the template with actual values + context = self.config.context_template + for field in self.config.schema_fields: + field_name = field.name + field_value = result.get(field_name, "") + + # Format list values if needed + if field.is_list and isinstance(field_value, list): + field_value = ", ".join(field_value) + + # Replace placeholder with value + placeholder = f"{{{field_name}}}" + context = context.replace(placeholder, str(field_value)) + + return context + + @property + def config_name(self) -> str: + """Get the configuration name.""" + return self.config.name + + @property + def version(self) -> str: + """Get the perspective version.""" + return self.config.version + + @property + def module_name(self) -> str: + """Get the module name.""" + return self.config.module + + @property + def tags(self) -> List[str]: + """Get the perspective tags.""" + return self.config.tags + + @property + def description(self) -> str: + """Get the perspective description.""" + return self.config.description + + @property + def is_deprecated(self) -> bool: + """Check if the perspective is deprecated.""" + return self.config.deprecated + + @property + def replacement(self) -> Optional[str]: + """Get the replacement perspective name if deprecated.""" + return self.config.replacement + + @property + def priority(self) -> int: + """Get the perspective priority.""" + return self.config.priority diff --git a/servers/inference_server/graphcap/providers/clients/__init__.py b/servers/inference_server/graphcap/providers/clients/__init__.py index 96b1a13b..6666d098 100644 --- a/servers/inference_server/graphcap/providers/clients/__init__.py +++ b/servers/inference_server/graphcap/providers/clients/__init__.py @@ -29,7 +29,7 @@ from .vllm_client import VLLMClient -def get_client(kind: str, **kwargs) -> BaseClient: +def get_client(kind: str, **kwargs) -> BaseClient: client: BaseClient logger.info(f"Creating client for {kind} with args: {kwargs}") if kind == "openai": @@ -46,6 +46,7 @@ def get_client(kind: str, **kwargs) -> BaseClient: raise ValueError(f"Unknown provider kind: {kind}") return client + __all__ = [ "BaseClient", "GeminiClient", diff --git a/servers/inference_server/graphcap/providers/clients/base_client.py b/servers/inference_server/graphcap/providers/clients/base_client.py index 01d804e1..f2401492 100644 --- a/servers/inference_server/graphcap/providers/clients/base_client.py +++ b/servers/inference_server/graphcap/providers/clients/base_client.py @@ -133,13 +133,13 @@ async def vision( """Create a vision completion with rate limiting""" logger.info(f"Starting vision request for model: {model}") logger.debug(f"Vision parameters - max_tokens: {max_tokens}, temperature: {temperature}, top_p: {top_p}") - + # Estimate token count - this is approximate estimated_tokens = len(prompt.split()) + 1000 # Base tokens + image tokens logger.debug(f"Estimated token count: {estimated_tokens}") await self._enforce_rate_limits(estimated_tokens) - + # Handle image input if isinstance(image, (str, Path)) and not str(image).startswith("data:"): logger.debug(f"Loading image from path: {image}") diff --git a/servers/inference_server/graphcap/providers/clients/gemini_client.py b/servers/inference_server/graphcap/providers/clients/gemini_client.py index 18b5ebda..4e232c56 100644 --- a/servers/inference_server/graphcap/providers/clients/gemini_client.py +++ b/servers/inference_server/graphcap/providers/clients/gemini_client.py @@ -54,7 +54,6 @@ def create_structured_completion( json_schema = self._get_schema_from_input(schema) try: - completion = self.chat.completions.create( model=model, messages=messages, response_format={"type": "json_schema", "schema": json_schema}, **kwargs ) diff --git a/servers/inference_server/graphcap/providers/clients/ollama_client.py b/servers/inference_server/graphcap/providers/clients/ollama_client.py index aea52294..fe42c180 100644 --- a/servers/inference_server/graphcap/providers/clients/ollama_client.py +++ b/servers/inference_server/graphcap/providers/clients/ollama_client.py @@ -33,10 +33,10 @@ def __init__(self, name: str, kind: str, environment: str, env_var: str, base_ur logger.info(f" - environment: {environment}") logger.info(f" - base_url: {base_url}") logger.info(f" - default_model: {default_model}") - + # Store the raw base URL for Ollama-specific endpoints base_url = base_url.rstrip("/") - + # For OpenAI compatibility, we need /v1 in the URL # But we need to handle cases where it's already there if "/v1" in base_url: @@ -50,7 +50,7 @@ def __init__(self, name: str, kind: str, environment: str, env_var: str, base_ur self._raw_base_url = base_url openai_base_url = f"{base_url}/v1" logger.debug("Adding /v1 to base URL for OpenAI compatibility") - + # Initialize with OpenAI-compatible base URL super().__init__( name=name, diff --git a/servers/inference_server/graphcap/providers/factory.py b/servers/inference_server/graphcap/providers/factory.py index 90a806f7..efbac5ab 100644 --- a/servers/inference_server/graphcap/providers/factory.py +++ b/servers/inference_server/graphcap/providers/factory.py @@ -22,15 +22,15 @@ def initialize_provider_manager(config_path: Optional[str | Path] = None) -> ProviderManager: """Initialize the global provider manager with the given config path. - + Args: config_path: Path to the provider configuration file. If None, uses default locations. - + Returns: ProviderManager: The initialized provider manager """ global _provider_manager - + if config_path is None: # Try to find config in standard locations possible_paths = [ @@ -40,19 +40,19 @@ def initialize_provider_manager(config_path: Optional[str | Path] = None) -> Pro "/app/provider.config.toml", "/app/config/provider.config.toml", ] - + for path in possible_paths: if path and Path(path).exists(): config_path = path break - + if not config_path or not Path(str(config_path)).exists(): logger.warning(f"No provider config found at {config_path}. Using empty configuration.") # Create a temporary empty config file with tempfile.NamedTemporaryFile(delete=False, suffix=".toml") as temp: temp.write(b"# Empty provider config\n") config_path = temp.name - + # At this point, config_path should not be None _provider_manager = ProviderManager(str(config_path)) return _provider_manager @@ -60,26 +60,26 @@ def initialize_provider_manager(config_path: Optional[str | Path] = None) -> Pro def get_provider_client(provider_name: str = "default") -> BaseClient: """Get a provider client by name. - + Args: provider_name: Name of the provider to get. Defaults to "default". - + Returns: BaseClient: The provider client - + Raises: ValueError: If the provider is not found """ global _provider_manager - + if _provider_manager is None: initialize_provider_manager() - + if _provider_manager is None: raise ValueError("Failed to initialize provider manager") - + try: return _provider_manager.get_client(provider_name) except ValueError as e: logger.error(f"Failed to get provider client: {e}") - raise \ No newline at end of file + raise diff --git a/servers/inference_server/graphcap/providers/provider_manager.py b/servers/inference_server/graphcap/providers/provider_manager.py index 405cada6..8db97294 100644 --- a/servers/inference_server/graphcap/providers/provider_manager.py +++ b/servers/inference_server/graphcap/providers/provider_manager.py @@ -62,7 +62,7 @@ def get_client(self, provider_name: str) -> BaseClient: logger.info(f" - environment: {config.environment}") logger.info(f" - base_url: {config.base_url}") logger.info(f" - default_model: {config.default_model}") - + try: client = get_client( name=provider_name, @@ -75,7 +75,9 @@ def get_client(self, provider_name: str) -> BaseClient: # Set rate limits if configured if config.rate_limits: - logger.debug(f"Setting rate limits for {provider_name} - requests: {config.rate_limits.requests_per_minute}/min, tokens: {config.rate_limits.tokens_per_minute}/min") + logger.debug( + f"Setting rate limits for {provider_name} - requests: {config.rate_limits.requests_per_minute}/min, tokens: {config.rate_limits.tokens_per_minute}/min" + ) client.requests_per_minute = config.rate_limits.requests_per_minute client.tokens_per_minute = config.rate_limits.tokens_per_minute @@ -109,4 +111,4 @@ def get_provider_config(self, provider_name: str) -> ProviderConfig: logger.error(f"Requested config for unknown provider: {provider_name}") raise ValueError(f"Unknown provider: {provider_name}") logger.debug(f"Returning config for provider: {provider_name}") - return self.providers[provider_name] \ No newline at end of file + return self.providers[provider_name] diff --git a/servers/inference_server/pipelines/pipelines/common/constants.py b/servers/inference_server/pipelines/pipelines/common/constants.py index 0baad527..dff54528 100644 --- a/servers/inference_server/pipelines/pipelines/common/constants.py +++ b/servers/inference_server/pipelines/pipelines/common/constants.py @@ -8,4 +8,4 @@ # Workspace configuration WORKSPACE_ROOT = Path("/workspace") WORKSPACE_CONFIG_DIR = WORKSPACE_ROOT / "config" -WORKSPACE_PERSPECTIVES_DIR = WORKSPACE_CONFIG_DIR / "perspectives" \ No newline at end of file +WORKSPACE_PERSPECTIVES_DIR = WORKSPACE_CONFIG_DIR / "perspectives" diff --git a/servers/inference_server/pipelines/pipelines/common/resources.py b/servers/inference_server/pipelines/pipelines/common/resources.py index 4b2b6b92..ab0003b7 100644 --- a/servers/inference_server/pipelines/pipelines/common/resources.py +++ b/servers/inference_server/pipelines/pipelines/common/resources.py @@ -28,6 +28,7 @@ class ProviderConfigFile(dg.ConfigurableResource): provider_config: str = "/workspace/config/provider.config.toml" default_provider: str = "gemini" + class PerspectiveConfig(dg.ConfigurableResource): """Configuration for perspective operations.""" diff --git a/servers/inference_server/pipelines/pipelines/definitions.py b/servers/inference_server/pipelines/pipelines/definitions.py index a99d9449..c2b86142 100644 --- a/servers/inference_server/pipelines/pipelines/definitions.py +++ b/servers/inference_server/pipelines/pipelines/definitions.py @@ -14,6 +14,7 @@ from .huggingface import huggingface_client from .huggingface.types import HfUploadManifestConfig from .perspectives.jobs import PerspectivePipelineRunConfig + # Import jobs from .jobs import JOBS diff --git a/servers/inference_server/pipelines/pipelines/io/image/load_images.py b/servers/inference_server/pipelines/pipelines/io/image/load_images.py index 8c662a19..b66dd3f8 100644 --- a/servers/inference_server/pipelines/pipelines/io/image/load_images.py +++ b/servers/inference_server/pipelines/pipelines/io/image/load_images.py @@ -141,14 +141,9 @@ def image_dataset_config(context: dg.AssetExecutionContext, config: DatasetIOCon return config -@asset( - group_name="image_load", - compute_kind="python", - deps=["perspective_pipeline_run_config"] -) +@asset(group_name="image_load", compute_kind="python", deps=["perspective_pipeline_run_config"]) def perspective_image_list( - context: dg.AssetExecutionContext, - perspective_pipeline_run_config: PerspectivePipelineConfig + context: dg.AssetExecutionContext, perspective_pipeline_run_config: PerspectivePipelineConfig ) -> list[str]: """ Load raw images from directory using the perspective pipeline configuration. @@ -160,7 +155,7 @@ def perspective_image_list( list[str]: A list of image paths. """ io_config = perspective_pipeline_run_config.io - + context.log.info(f"Loading image list for dataset {io_config.dataset_name}") image_files = get_image_list( @@ -170,14 +165,16 @@ def perspective_image_list( sampling_strategy=SamplingStrategy(io_config.sampling_strategy), num_samples=io_config.num_samples, ) - + copied_image_files: list[str] = copy_images(context, image_files, io_config.output_dir) - - context.add_output_metadata({ - "dataset_name": io_config.dataset_name, - "input_dir": io_config.input_dir, - "output_dir": io_config.output_dir, - "num_images": len(copied_image_files) - }) - + + context.add_output_metadata( + { + "dataset_name": io_config.dataset_name, + "input_dir": io_config.input_dir, + "output_dir": io_config.output_dir, + "num_images": len(copied_image_files), + } + ) + return copied_image_files diff --git a/servers/inference_server/pipelines/pipelines/perspectives/assets.py b/servers/inference_server/pipelines/pipelines/perspectives/assets.py index f08dc4c4..ef1f4a2e 100644 --- a/servers/inference_server/pipelines/pipelines/perspectives/assets.py +++ b/servers/inference_server/pipelines/pipelines/perspectives/assets.py @@ -34,10 +34,7 @@ async def perspective_caption( perspective_config = perspective_pipeline_run_config.perspective # Get enabled perspectives - enabled_perspectives = [ - name for name, enabled in perspective_config.enabled_perspectives.items() - if enabled - ] + enabled_perspectives = [name for name, enabled in perspective_config.enabled_perspectives.items() if enabled] context.log.info(f"Processing enabled perspectives: {enabled_perspectives}") # Instantiate the client @@ -51,9 +48,11 @@ async def perspective_caption( # Process images in batch image_paths = [Path(image) for image in perspective_image_list] caption_data_list = await processor.process_batch( - client, image_paths, output_dir=Path(io_config.run_dir), + client, + image_paths, + output_dir=Path(io_config.run_dir), global_context=perspective_config.global_context, - name=perspective + name=perspective, ) # Aggregate results @@ -81,6 +80,7 @@ async def perspective_caption( context.add_output_metadata(metadata) return all_results + @dg.asset( group_name="perspectives", compute_kind="python", @@ -92,7 +92,7 @@ def caption_contexts( perspective_pipeline_run_config: PerspectivePipelineConfig, ) -> Dict[str, List[str]]: """Extracts contexts from the perspective caption data and adds to a dictionary of path:List[str]. - if the path exists, the context is appended to the list.""" + if the path exists, the context is appended to the list.""" contexts = {} for item in perspective_caption: context.log.info(f"Processing {item['image_filename']} ({item['perspective']})") @@ -106,6 +106,7 @@ def caption_contexts( context.log.info(contexts) return contexts + @dg.asset( group_name="perspectives", compute_kind="python", @@ -125,26 +126,28 @@ async def synthesizer_caption( client = get_provider(provider_config.provider_config_file, provider_config.default) synthesizer = get_synthesizer() - image_dir = Path(io_config.output_dir)/"images" + image_dir = Path(io_config.output_dir) / "images" paths = [image_dir / path for path in caption_contexts.keys()] - results = await synthesizer.process_batch(client, paths, - output_dir=Path(io_config.run_dir), - contexts=caption_contexts, - name="synthesized_caption") + results = await synthesizer.process_batch( + client, paths, output_dir=Path(io_config.run_dir), contexts=caption_contexts, name="synthesized_caption" + ) # Format the results to match the perspective_caption output formatted_results = [] for path, caption_data in zip(paths, results): image_filename = path.name - formatted_results.append({ - "perspective": "synthesized_caption", - "image_filename": image_filename, - "caption_data": caption_data, - "context": synthesizer.to_context(caption_data), - }) + formatted_results.append( + { + "perspective": "synthesized_caption", + "image_filename": image_filename, + "caption_data": caption_data, + "context": synthesizer.to_context(caption_data), + } + ) context.log.info(f"Synthesizer caption results: {formatted_results}") return formatted_results + @dg.asset( group_name="perspectives", compute_kind="python", @@ -164,8 +167,7 @@ def caption_output_files( run_dir.mkdir(parents=True, exist_ok=True) # Get enabled perspectives enabled_perspectives = [ - name for name, enabled in perspective_pipeline_run_config.perspective.enabled_perspectives.items() - if enabled + name for name, enabled in perspective_pipeline_run_config.perspective.enabled_perspectives.items() if enabled ] timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") # Prepare data for DataFrame @@ -232,4 +234,3 @@ def caption_output_files( "parquet_output_path": str(parquet_path), } ) - diff --git a/servers/inference_server/pipelines/pipelines/perspectives/jobs/__init__.py b/servers/inference_server/pipelines/pipelines/perspectives/jobs/__init__.py index 1059be32..e5c5ab3a 100644 --- a/servers/inference_server/pipelines/pipelines/perspectives/jobs/__init__.py +++ b/servers/inference_server/pipelines/pipelines/perspectives/jobs/__init__.py @@ -30,4 +30,3 @@ "perspective_pipeline_run_config", "basic_perspective_pipeline", ] - diff --git a/servers/inference_server/pipelines/pipelines/perspectives/jobs/basic_perspective_pipeline.py b/servers/inference_server/pipelines/pipelines/perspectives/jobs/basic_perspective_pipeline.py index a2809c35..dc45af08 100644 --- a/servers/inference_server/pipelines/pipelines/perspectives/jobs/basic_perspective_pipeline.py +++ b/servers/inference_server/pipelines/pipelines/perspectives/jobs/basic_perspective_pipeline.py @@ -15,5 +15,5 @@ "synthesizer_caption", "caption_output_files", ], - description="Basic perspective pipeline, graphcap's \"hello world\" example", + description='Basic perspective pipeline, graphcap\'s "hello world" example', ) diff --git a/servers/inference_server/pipelines/pipelines/perspectives/jobs/config.py b/servers/inference_server/pipelines/pipelines/perspectives/jobs/config.py index 666221a6..ddf01c8f 100644 --- a/servers/inference_server/pipelines/pipelines/perspectives/jobs/config.py +++ b/servers/inference_server/pipelines/pipelines/perspectives/jobs/config.py @@ -48,6 +48,7 @@ class PerspectiveConfig: global_context: str enabled_perspectives: Dict[str, bool] = Field(default_factory=dict) + class PerspectivePipelineConfig(BaseModel): """Configuration for pipeline runs loaded from TOML file.""" @@ -75,7 +76,7 @@ def load_dataset(self, context: dg.AssetExecutionContext) -> PerspectivePipeline # Create perspective config with enabled perspectives perspective = PerspectiveConfig( global_context=config["perspective"]["global_context"], - enabled_perspectives=config["perspective"]["enabled"] + enabled_perspectives=config["perspective"]["enabled"], ) timestamp = datetime.now().strftime("%Y%m%d_%H%M%S") run_dir = config["io"]["output_dir"] + "/" + timestamp @@ -106,12 +107,7 @@ def load_dataset(self, context: dg.AssetExecutionContext) -> PerspectivePipeline ) # Return the complete config object - return PerspectivePipelineConfig( - perspective=perspective, - io=io, - provider=provider, - filesystem=filesystem - ) + return PerspectivePipelineConfig(perspective=perspective, io=io, provider=provider, filesystem=filesystem) @dg.asset diff --git a/servers/inference_server/pipelines/pipelines/providers/util.py b/servers/inference_server/pipelines/pipelines/providers/util.py index f1a3a4b7..5f058ba2 100644 --- a/servers/inference_server/pipelines/pipelines/providers/util.py +++ b/servers/inference_server/pipelines/pipelines/providers/util.py @@ -2,6 +2,7 @@ from graphcap.providers.clients import get_client from ..perspectives.jobs.config import PerspectivePipelineConfig + def get_provider(config_path: str, default_provider: str): """Instantiates the client based on the provider configuration. diff --git a/servers/inference_server/pyproject.toml b/servers/inference_server/pyproject.toml index b68d20dd..af588a00 100644 --- a/servers/inference_server/pyproject.toml +++ b/servers/inference_server/pyproject.toml @@ -10,6 +10,7 @@ dependencies = [ "openai>=1.63.2", "pydantic>=2.10.6", "rich>=13.9.4", + "tenacity>=9.0.0", ] [dependency-groups] @@ -71,7 +72,7 @@ target-version = "py311" [tool.ruff.lint] # Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default select = ["E", "F", "W", "I001"] -ignore = [] +ignore = ["W293"] # Allow fix for all enabled rules (when `--fix`) is provided fixable = ["ALL"] diff --git a/servers/inference_server/scripts/__init__.py b/servers/inference_server/scripts/__init__.py index 7c922a24..d642b489 100644 --- a/servers/inference_server/scripts/__init__.py +++ b/servers/inference_server/scripts/__init__.py @@ -13,4 +13,4 @@ from .config_writer import write_toml_config from .setup import cli -__all__ = ['write_toml_config', 'cli'] +__all__ = ["write_toml_config", "cli"] diff --git a/servers/inference_server/scripts/__main__.py b/servers/inference_server/scripts/__main__.py index fe9d2b28..100824ca 100644 --- a/servers/inference_server/scripts/__main__.py +++ b/servers/inference_server/scripts/__main__.py @@ -1,4 +1,4 @@ from .setup import cli if __name__ == "__main__": - cli() \ No newline at end of file + cli() diff --git a/servers/inference_server/scripts/setup.py b/servers/inference_server/scripts/setup.py index 605953d1..451bcef8 100644 --- a/servers/inference_server/scripts/setup.py +++ b/servers/inference_server/scripts/setup.py @@ -106,14 +106,12 @@ def check_existing_files() -> tuple[bool, bool]: if os.path.exists(dotenv_path): overwrite_env = Confirm.ask( - "[bold blue].env file already exists. Do you want to overwrite it?[/]", - default=False + "[bold blue].env file already exists. Do you want to overwrite it?[/]", default=False ) if os.path.exists(provider_config_path): overwrite_config = Confirm.ask( - "[bold blue]provider.config.toml file already exists. Do you want to overwrite it?[/]", - default=False + "[bold blue]provider.config.toml file already exists. Do you want to overwrite it?[/]", default=False ) return overwrite_env, overwrite_config @@ -130,6 +128,7 @@ def get_provider_selections() -> tuple[bool, bool, bool, bool, bool]: return enable_hugging_face, enable_openai, enable_google, enable_vllm, enable_ollama + def get_postgres_selections() -> tuple[str, str, str]: """Collect user selections for what values to use for Postgres variables.""" console.print("\n[bold]Set the following values for the Postgres installation:[/]") @@ -139,7 +138,10 @@ def get_postgres_selections() -> tuple[str, str, str]: return postgres_user_value, postgres_password_value, postgres_database_value -def collect_env_variables(providers: tuple[bool, bool, bool, bool, bool], postgres: tuple[str, str, str]) -> Mapping[str, str | None]: + +def collect_env_variables( + providers: tuple[bool, bool, bool, bool, bool], postgres: tuple[str, str, str] +) -> Mapping[str, str | None]: """Collect environment variables based on enabled providers.""" enable_hugging_face, enable_openai, enable_google, enable_vllm, enable_ollama = providers postgres_user_value, postgres_password_value, postgres_database_value = postgres @@ -162,8 +164,7 @@ def collect_env_variables(providers: tuple[bool, bool, bool, bool, bool], postgr if enable_vllm and not env_vars["VLLM_BASE_URL"]: cprint("VLLM_BASE_URL not set.", "yellow") env_vars["VLLM_BASE_URL"] = Prompt.ask( - "[bold blue]Please enter the vLLM base URL[/]", - default="http://localhost:12434" + "[bold blue]Please enter the vLLM base URL[/]", default="http://localhost:12434" ) os.environ["VLLM_BASE_URL"] = env_vars["VLLM_BASE_URL"] @@ -205,7 +206,7 @@ def cli(): # Get provider selections provider_selections = get_provider_selections() - + # Get postgres selections postgres_selections = get_postgres_selections() @@ -225,9 +226,11 @@ def cli(): console.print("[bold green]Provider configuration has been saved to workspace/config/provider.config.toml![/]") console.print("[bold magenta]Graphcap instance is ready to be launched![/]") + def main(): """Entry point for the script""" cli() + if __name__ == "__main__": main() diff --git a/servers/inference_server/server/server/config.py b/servers/inference_server/server/server/config.py index ad83fbe1..024ac23d 100644 --- a/servers/inference_server/server/server/config.py +++ b/servers/inference_server/server/server/config.py @@ -42,7 +42,7 @@ def __init__(self, **kwargs): # If provider config not set, default to config directory if not self.PROVIDER_CONFIG_PATH: self.PROVIDER_CONFIG_PATH = self.CONFIG_PATH / "provider.config.toml" - + # Construct DATABASE_URL if not provided if not self.DATABASE_URL: self.DATABASE_URL = f"postgresql+asyncpg://{self.POSTGRES_USER}:{self.POSTGRES_PASSWORD}@{self.POSTGRES_HOST}:{self.POSTGRES_PORT}/{self.POSTGRES_DB}" diff --git a/servers/inference_server/server/server/features/perspectives/__init__.py b/servers/inference_server/server/server/features/perspectives/__init__.py index 78528bb7..c2fa4fea 100644 --- a/servers/inference_server/server/server/features/perspectives/__init__.py +++ b/servers/inference_server/server/server/features/perspectives/__init__.py @@ -3,4 +3,4 @@ Perspectives Feature Module Provides API endpoints for accessing and using perspective captions. -""" \ No newline at end of file +""" diff --git a/servers/inference_server/server/server/features/perspectives/models.py b/servers/inference_server/server/server/features/perspectives/models.py index 37bdf0de..f975e49c 100644 --- a/servers/inference_server/server/server/features/perspectives/models.py +++ b/servers/inference_server/server/server/features/perspectives/models.py @@ -5,7 +5,7 @@ Defines data models for the perspectives API endpoints. """ -from typing import List, Optional, Union +from typing import Any, Dict, List, Optional, Union from fastapi import File, Form, UploadFile from pydantic import BaseModel, Field @@ -19,8 +19,7 @@ DESC_GLOBAL_CONTEXT = "Global context for the caption" DESC_ADDITIONAL_CONTEXT = "Additional context for the caption" DESC_RESIZE_RESOLUTION = ( - "Resolution to resize to (None to disable, or SD_VGA, HD_720P, " - "FHD_1080P, QHD_1440P, UHD_4K, UHD_8K)" + "Resolution to resize to (None to disable, or SD_VGA, HD_720P, FHD_1080P, QHD_1440P, UHD_4K, UHD_8K)" ) @@ -28,11 +27,11 @@ class SchemaField(BaseModel): """Schema field information for a perspective.""" name: str = Field(..., description="Name of the field") - type: str = Field(..., description="Type of the field (str, float)") + type: Union[str, Dict[str, Any]] = Field(..., description="Type of the field (str, float, or complex object)") description: str = Field(..., description="Description of the field") is_list: bool = Field(False, description="Whether the field is a list") is_complex: bool = Field(False, description="Whether the field is a complex type") - fields: Optional[List['SchemaField']] = Field(None, description="Fields for complex types") + fields: Optional[List["SchemaField"]] = Field(None, description="Fields for complex types") class TableColumn(BaseModel): @@ -62,6 +61,13 @@ class PerspectiveInfo(BaseModel): version: str = Field(..., description="Version of the perspective") description: str = Field("", description="Description of what the perspective analyzes") schema: Optional[PerspectiveSchema] = Field(None, description="Schema information for the perspective") + + # New metadata fields + module: str = Field("default", description="Module this perspective belongs to") + tags: List[str] = Field(default_factory=list, description="Tags for categorizing perspectives") + deprecated: bool = Field(False, description="Whether this perspective is deprecated") + replacement: Optional[str] = Field(None, description="Name of perspective that replaces this one") + priority: int = Field(100, description="Priority for sorting (lower is higher priority)") class PerspectiveListResponse(BaseModel): @@ -70,6 +76,29 @@ class PerspectiveListResponse(BaseModel): perspectives: List[PerspectiveInfo] = Field(..., description="List of available perspectives") +class ModuleInfo(BaseModel): + """Information about a perspective module.""" + + name: str = Field(..., description="Unique identifier for the module") + display_name: str = Field(..., description="Human-readable name for the module") + description: str = Field("", description="Description of the module") + enabled: bool = Field(True, description="Whether the module is enabled") + perspective_count: int = Field(0, description="Number of perspectives in this module") + + +class ModuleListResponse(BaseModel): + """Response model for listing available modules.""" + + modules: List[ModuleInfo] = Field(..., description="List of available modules") + + +class ModulePerspectivesResponse(BaseModel): + """Response model for perspectives in a module.""" + + module: ModuleInfo = Field(..., description="Information about the module") + perspectives: List[PerspectiveInfo] = Field(..., description="List of perspectives in the module") + + class ImageSource(BaseModel): """Source of an image for captioning.""" @@ -77,11 +106,7 @@ class ImageSource(BaseModel): base64: Optional[str] = Field(None, description="Base64-encoded image data") class Config: - schema_extra = { - "example": { - "url": "https://example.com/image.jpg" - } - } + schema_extra = {"example": {"url": "https://example.com/image.jpg"}} class CaptionRequest(BaseModel): @@ -101,12 +126,10 @@ class Config: schema_extra = { "example": { "perspective": "custom_caption", - "image": { - "url": "https://example.com/image.jpg" - }, + "image": {"url": "https://example.com/image.jpg"}, "max_tokens": 4096, "temperature": 0.8, - "resize_resolution": "HD_720P" + "resize_resolution": "HD_720P", } } @@ -136,7 +159,7 @@ def __init__( repetition_penalty: Optional[float] = Form(1.15, description=DESC_REPETITION_PENALTY), global_context: Optional[str] = Form(None, description=DESC_GLOBAL_CONTEXT), context: Optional[str] = Form(None, description="Additional context for the caption (JSON array string)"), - resize_resolution: Optional[str] = Form(None, description=DESC_RESIZE_RESOLUTION) + resize_resolution: Optional[str] = Form(None, description=DESC_RESIZE_RESOLUTION), ): self.perspective = perspective self.file = file @@ -153,6 +176,7 @@ def __init__( self.context = None if context: import json + try: self.context = json.loads(context) except json.JSONDecodeError: @@ -182,6 +206,6 @@ class Config: "provider": "gemini", "max_tokens": 4096, "temperature": 0.8, - "resize_resolution": "HD_720P" + "resize_resolution": "HD_720P", } } diff --git a/servers/inference_server/server/server/features/perspectives/router.py b/servers/inference_server/server/server/features/perspectives/router.py index 39cf94b7..3671273e 100644 --- a/servers/inference_server/server/server/features/perspectives/router.py +++ b/servers/inference_server/server/server/features/perspectives/router.py @@ -9,6 +9,8 @@ - POST /perspectives/caption - Generate a caption for an image using file upload - GET /perspectives/debug/{perspective_name} - Get debug information about a perspective - POST /perspectives/caption-from-path - Generate a caption for an image using a file path +- GET /perspectives/modules - List all available perspective modules +- GET /perspectives/modules/{module_name} - Get perspectives for a specific module """ import json @@ -21,10 +23,18 @@ from loguru import logger from ...utils.resizing import ResolutionPreset, log_resize_options, resize_image -from .models import CaptionPathRequest, CaptionResponse, PerspectiveListResponse +from .models import ( + CaptionPathRequest, + CaptionResponse, + ModuleListResponse, + ModulePerspectivesResponse, + PerspectiveListResponse, +) from .service import ( generate_caption, + get_available_modules, get_available_perspectives, + get_perspectives_by_module, save_uploaded_file, ) @@ -43,45 +53,6 @@ async def list_perspectives() -> PerspectiveListResponse: return PerspectiveListResponse(perspectives=perspectives) -@router.get("/debug/{perspective_name}") -async def debug_perspective(perspective_name: str) -> dict: - """ - Get debug information about a perspective. - Args: - perspective_name: Name of the perspective to debug - Returns: - Debug information about the perspective - Raises: - HTTPException: If the perspective is not found - """ - from graphcap.perspectives import get_perspective - try: - perspective = get_perspective(perspective_name) - # Get perspective attributes - attributes = { - "config_name": perspective.config_name, - "display_name": perspective.display_name, - "version": perspective.version, - "has_process_single": hasattr(perspective, "process_single"), - "has_process_batch": hasattr(perspective, "process_batch"), - } - # Get method signatures if available - if attributes["has_process_single"]: - import inspect - attributes["process_single_signature"] = str(inspect.signature(perspective.process_single)) - if attributes["has_process_batch"]: - import inspect - attributes["process_batch_signature"] = str(inspect.signature(perspective.process_batch)) - return { - "perspective": perspective_name, - "attributes": attributes, - "type": str(type(perspective)), - } - except Exception as e: - logger.error(f"Error getting perspective debug info: {str(e)}") - raise HTTPException(status_code=404, detail=f"Error getting perspective debug info: {str(e)}") - - @router.post("/caption", response_model=CaptionResponse, status_code=status.HTTP_200_OK) async def create_caption( background_tasks: BackgroundTasks, @@ -94,7 +65,7 @@ async def create_caption( repetition_penalty: Optional[float] = Form(1.15, description="Repetition penalty"), global_context: Optional[str] = Form(None, description="Global context for the caption"), context: Optional[str] = Form(None, description="Additional context for the caption as JSON array string"), - resize_resolution: Optional[str] = Form(None, description="Resolution to resize to (None to disable resizing)") + resize_resolution: Optional[str] = Form(None, description="Resolution to resize to (None to disable resizing)"), ) -> CaptionResponse: """ Generate a caption for an image using a perspective. @@ -128,9 +99,7 @@ async def create_caption( image_path = await save_uploaded_file(file) # Log resize options - options = { - 'resize_resolution': resize_resolution - } + options = {"resize_resolution": resize_resolution} log_resize_options(options) # Resize the image if resize_resolution is provided @@ -230,10 +199,7 @@ async def create_caption_from_path( temp_path = None # Handle image resizing if requested - image_path, temp_path = await _resize_image_if_needed( - image_path, - request.resize_resolution - ) + image_path, temp_path = await _resize_image_if_needed(image_path, request.resize_resolution) # Process context context = _process_context(request.context) @@ -255,11 +221,7 @@ async def create_caption_from_path( _cleanup_temp_file(temp_path) # Prepare the response - return _prepare_caption_response( - caption_data, - request.perspective, - request.provider - ) + return _prepare_caption_response(caption_data, request.perspective, request.provider) except Exception as e: logger.error(f"Error creating caption from path: {str(e)}") if isinstance(e, HTTPException): @@ -275,15 +237,12 @@ def _validate_image_path(image_path_str: str) -> Path: return image_path -async def _resize_image_if_needed( - image_path: Path, - resize_resolution: Optional[str] -) -> tuple[Path, Optional[Path]]: +async def _resize_image_if_needed(image_path: Path, resize_resolution: Optional[str]) -> tuple[Path, Optional[Path]]: """Resize the image if a resize resolution is provided.""" temp_path = None # Log resize options - options = {'resize_resolution': resize_resolution} + options = {"resize_resolution": resize_resolution} log_resize_options(options) if not resize_resolution: @@ -348,11 +307,7 @@ def _cleanup_temp_file(temp_path: Optional[Path]) -> None: logger.error(f"Error removing temporary file: {str(e)}") -def _prepare_caption_response( - caption_data: dict, - perspective: str, - provider: str -) -> CaptionResponse: +def _prepare_caption_response(caption_data: dict, perspective: str, provider: str) -> CaptionResponse: """Prepare the caption response from the caption data.""" # Log the caption data for debugging logger.debug(f"Caption data: {caption_data}") @@ -387,3 +342,57 @@ def _parse_context(context_str) -> Optional[List[str]]: return [context_str] except json.JSONDecodeError: return [context_str] + + + + +@router.get("/modules", response_model=ModuleListResponse) +async def list_modules() -> ModuleListResponse: + """ + List all available perspective modules. + + Returns: + List of available modules with metadata + """ + modules = get_available_modules() + return ModuleListResponse(modules=modules) + + +@router.get("/modules/{module_name}", response_model=ModulePerspectivesResponse) +async def get_module_perspectives(module_name: str) -> ModulePerspectivesResponse: + """ + Get all perspectives that belong to a specific module. + + Args: + module_name: Name of the module to get perspectives for + + Returns: + Module information and list of perspectives in the module + + Raises: + HTTPException: If the module is not found + """ + try: + # Get perspectives for the module + perspectives = get_perspectives_by_module(module_name) + + # Get module info + modules = get_available_modules() + module = next((m for m in modules if m.name == module_name), None) + + if not module: + raise HTTPException(status_code=404, detail=f"Module '{module_name}' not found") + + return ModulePerspectivesResponse( + module=module, + perspectives=perspectives + ) + except HTTPException: + # Re-raise HTTP exceptions + raise + except Exception as e: + logger.error(f"Error getting perspectives for module '{module_name}': {str(e)}") + raise HTTPException( + status_code=500, + detail=f"Error getting perspectives for module '{module_name}': {str(e)}" + ) diff --git a/servers/inference_server/server/server/features/perspectives/service.py b/servers/inference_server/server/server/features/perspectives/service.py index 8a94fb01..d52d969b 100644 --- a/servers/inference_server/server/server/features/perspectives/service.py +++ b/servers/inference_server/server/server/features/perspectives/service.py @@ -6,12 +6,12 @@ """ import base64 -import json import os import socket import tempfile from pathlib import Path from typing import Dict, List, Optional +from collections import defaultdict import aiohttp from fastapi import HTTPException, UploadFile @@ -23,19 +23,16 @@ from loguru import logger from ..providers.service import get_provider_manager -from .models import PerspectiveInfo, PerspectiveSchema, SchemaField, TableColumn +from .models import ModuleInfo, PerspectiveInfo, PerspectiveSchema, SchemaField, TableColumn async def download_image(url: str) -> Path: """ Download an image from a URL to a temporary file. - Args: url: URL of the image to download - Returns: Path to the downloaded image - Raises: HTTPException: If the image cannot be downloaded """ @@ -49,10 +46,7 @@ async def download_image(url: str) -> Path: async with aiohttp.ClientSession() as session: async with session.get(url) as response: if response.status != 200: - raise HTTPException( - status_code=400, - detail=f"Failed to download image from URL: {response.status}" - ) + raise HTTPException(status_code=400, detail=f"Failed to download image from URL: {response.status}") content = await response.read() with open(temp_file, "wb") as f: @@ -67,13 +61,13 @@ async def download_image(url: str) -> Path: async def save_base64_image(base64_data: str) -> Path: """ Save a base64-encoded image to a temporary file. - + Args: base64_data: Base64-encoded image data - + Returns: Path to the saved image - + Raises: HTTPException: If the image cannot be saved """ @@ -104,68 +98,61 @@ async def save_base64_image(base64_data: str) -> Path: def load_perspective_schema(perspective_name: str) -> Optional[PerspectiveSchema]: """ Load the schema for a perspective from its configuration file. - + Args: perspective_name: Name of the perspective - + Returns: Schema information for the perspective, or None if not found """ try: - # Construct the path to the schema file - schema_path = Path("/workspace/config/perspectives") / f"{perspective_name}.json" + # Import perspective function + from graphcap.perspectives import get_perspective - # Check if the schema file exists - if not schema_path.exists(): - logger.warning(f"Schema file not found for perspective {perspective_name}") - return None - - # Load and parse the schema file - with open(schema_path, "r") as f: - schema_data = json.load(f) + # Get the perspective processor + perspective = get_perspective(perspective_name) + if perspective and hasattr(perspective, 'config'): + # Extract config directly from the processor + config = perspective.config - # Convert the schema data to our models - schema_fields = [] - for field_data in schema_data.get("schema_fields", []): - # Convert nested fields for complex types - nested_fields = None - if field_data.get("fields"): - nested_fields = [ + # Create schema fields + schema_fields = [] + for field in config.schema_fields: + schema_fields.append( SchemaField( - name=f["name"], - type=f["type"], - description=f["description"], - is_list=f.get("is_list", False), - is_complex=f.get("is_complex", False) + name=field.name, + type=field.type, + description=field.description, + is_list=getattr(field, "is_list", False), + is_complex=getattr(field, "is_complex", False), + fields=None ) - for f in field_data["fields"] - ] - - schema_fields.append( - SchemaField( - name=field_data["name"], - type=field_data["type"], - description=field_data["description"], - is_list=field_data.get("is_list", False), - is_complex=field_data.get("is_complex", False), - fields=nested_fields ) + + # Create table columns + table_columns = [ + TableColumn(name=col["name"], style=col["style"]) + for col in config.table_columns + ] + + # Create the schema + schema = PerspectiveSchema( + name=config.name, + display_name=config.display_name, + version=config.version, + prompt=config.prompt, + schema_fields=schema_fields, + table_columns=table_columns, + context_template=config.context_template ) - table_columns = [ - TableColumn(name=col["name"], style=col["style"]) - for col in schema_data.get("table_columns", []) - ] + logger.info(f"Successfully loaded schema for perspective '{perspective_name}'") + return schema - return PerspectiveSchema( - name=schema_data["name"], - display_name=schema_data["display_name"], - version=schema_data["version"], - prompt=schema_data["prompt"], - schema_fields=schema_fields, - table_columns=table_columns, - context_template=schema_data["context_template"] - ) + logger.warning(f"""Could not load schema for perspective '{perspective_name}': + Perspective not found or does not have config attribute""") + return None + except Exception as e: logger.error(f"Error loading schema for perspective {perspective_name}: {str(e)}") return None @@ -174,7 +161,7 @@ def load_perspective_schema(perspective_name: str) -> Optional[PerspectiveSchema def get_available_perspectives() -> List[PerspectiveInfo]: """ Get a list of available perspectives with their schemas. - + Returns: List of perspective information including schemas """ @@ -185,14 +172,19 @@ def get_available_perspectives() -> List[PerspectiveInfo]: try: perspective = get_perspective(name) schema = load_perspective_schema(perspective.config_name) - + perspectives.append( PerspectiveInfo( name=perspective.config_name, display_name=perspective.display_name, version=perspective.version, - description="", # Could be added to the perspective config in the future - schema=schema + description=perspective.description, + schema=schema, + module=perspective.module_name, + tags=perspective.tags, + deprecated=perspective.is_deprecated, + replacement=perspective.replacement, + priority=perspective.priority, ) ) except Exception as e: @@ -201,6 +193,73 @@ def get_available_perspectives() -> List[PerspectiveInfo]: return perspectives +def get_available_modules() -> List[ModuleInfo]: + """ + Get a list of available modules and their metadata. + + Returns: + List of module information + """ + perspectives = get_available_perspectives() + + # Group perspectives by module + module_map = defaultdict(list) + for perspective in perspectives: + module_map[perspective.module].append(perspective) + + # Create module info objects + modules = [] + for module_name, module_perspectives in module_map.items(): + # Try to find a display name, falling back to the module name + display_name = module_name.replace("_", " ").title() + + modules.append( + ModuleInfo( + name=module_name, + display_name=display_name, + description=f"Contains {len(module_perspectives)} perspectives", + enabled=True, # Default to enabled + perspective_count=len(module_perspectives) + ) + ) + + # Sort modules by name + modules.sort(key=lambda m: m.name) + + return modules + + +def get_perspectives_by_module(module_name: str) -> List[PerspectiveInfo]: + """ + Get a list of perspectives that belong to a specific module. + + Args: + module_name: Name of the module to filter by + + Returns: + List of perspective information in the specified module + + Raises: + HTTPException: If the module is not found + """ + perspectives = get_available_perspectives() + + # Filter perspectives by module name + module_perspectives = [p for p in perspectives if p.module == module_name] + + # If no perspectives were found for this module, the module might not exist + if not module_perspectives: + # Check if the module exists + all_modules = get_available_modules() + if not any(m.name == module_name for m in all_modules): + raise HTTPException(status_code=404, detail=f"Module '{module_name}' not found") + + # Sort perspectives by priority, then name + module_perspectives.sort(key=lambda p: (p.priority, p.name)) + + return module_perspectives + + async def generate_caption( perspective_name: str, image_path: Path, @@ -214,7 +273,7 @@ async def generate_caption( ) -> Dict: """ Generate a caption for an image using a perspective. - + Args: perspective_name: Name of the perspective to use image_path: Path to the image file @@ -225,10 +284,10 @@ async def generate_caption( context: Additional context for the caption global_context: Global context for the caption provider_name: Name of the provider to use (default: "gemini") - + Returns: Caption data - + Raises: HTTPException: If the caption cannot be generated """ @@ -238,22 +297,22 @@ async def generate_caption( # Get the provider client from the provider manager provider_manager = get_provider_manager() - + # Debug: Log available providers available_providers = provider_manager.available_providers() logger.debug(f"Available providers: {available_providers}") - + # Debug: Try to resolve host.docker.internal try: - host_ip = socket.gethostbyname('host.docker.internal') + host_ip = socket.gethostbyname("host.docker.internal") logger.debug(f"host.docker.internal resolves to: {host_ip}") except socket.gaierror as e: logger.warning(f"Could not resolve host.docker.internal: {e}") - + try: provider: BaseClient = provider_manager.get_client(provider_name) # Debug: Log provider details - logger.debug(f"Provider details:") + logger.debug("Provider details:") logger.debug(f" - Name: {provider_name}") logger.debug(f" - Kind: {provider.kind}") logger.debug(f" - Environment: {provider.environment}") @@ -263,16 +322,19 @@ async def generate_caption( logger.error(f"Provider '{provider_name}' not found: {str(e)}") raise HTTPException( status_code=404, - detail=f"Provider '{provider_name}' not found. Available providers: {', '.join(provider_manager.available_providers())}" + detail=f"""Provider '{provider_name}' not found. + Available providers: {', '.join(provider_manager.available_providers())}""", ) - + # Create a temporary output directory with tempfile.TemporaryDirectory() as temp_dir: output_dir = Path(temp_dir) - + # Generate the caption - logger.info(f"Generating caption for {image_path} using {perspective_name} perspective and {provider_name} provider") - + logger.info( + f"Generating caption for {image_path} using {perspective_name} perspective and {provider_name} provider" + ) + # Check if the perspective has process_batch method if hasattr(perspective, "process_batch"): logger.info(f"Using process_batch method for {perspective_name}") @@ -286,14 +348,14 @@ async def generate_caption( top_p=top_p, repetition_penalty=repetition_penalty, global_context=global_context, - name=perspective_name + name=perspective_name, ) - + # Get the first (and only) result if not caption_data_list or len(caption_data_list) == 0: logger.error(f"No caption data returned for {image_path}") raise HTTPException(status_code=500, detail="No caption data returned") - + caption_data = caption_data_list[0] else: # Fallback to process_single if process_batch is not available @@ -308,10 +370,10 @@ async def generate_caption( context=context, global_context=global_context, ) - + # Log the result logger.info(f"Caption generated successfully: {caption_data.keys() if caption_data else 'None'}") - + return caption_data except ValueError as e: logger.error(f"Error getting perspective: {str(e)}") @@ -324,13 +386,13 @@ async def generate_caption( async def save_uploaded_file(file: UploadFile) -> Path: """ Save an uploaded file to a temporary location. - + Args: file: Uploaded file object - + Returns: Path to the saved file - + Raises: HTTPException: If the file cannot be saved """ diff --git a/servers/inference_server/server/server/features/providers/__init__.py b/servers/inference_server/server/server/features/providers/__init__.py index 2999ceaa..ac7be2ec 100644 --- a/servers/inference_server/server/server/features/providers/__init__.py +++ b/servers/inference_server/server/server/features/providers/__init__.py @@ -7,4 +7,4 @@ from .router import router -__all__ = ["router"] \ No newline at end of file +__all__ = ["router"] diff --git a/servers/inference_server/server/server/features/providers/models.py b/servers/inference_server/server/server/features/providers/models.py index 75b0ec84..506f7298 100644 --- a/servers/inference_server/server/server/features/providers/models.py +++ b/servers/inference_server/server/server/features/providers/models.py @@ -26,7 +26,7 @@ class ProviderListResponse(BaseModel): class ModelInfo(BaseModel): """Information about a model.""" - + id: str = Field(..., description="Unique identifier for the model") name: str = Field(..., description="Display name of the model") is_default: bool = Field(False, description="Whether this is the default model for the provider") @@ -34,6 +34,6 @@ class ModelInfo(BaseModel): class ProviderModelsResponse(BaseModel): """Response model for listing available models for a provider.""" - + provider: str = Field(..., description="Name of the provider") - models: List[ModelInfo] = Field(..., description="List of available models") \ No newline at end of file + models: List[ModelInfo] = Field(..., description="List of available models") diff --git a/servers/inference_server/server/server/features/providers/router.py b/servers/inference_server/server/server/features/providers/router.py index 663641cd..4e71725a 100644 --- a/servers/inference_server/server/server/features/providers/router.py +++ b/servers/inference_server/server/server/features/providers/router.py @@ -34,28 +34,28 @@ async def list_providers() -> ProviderListResponse: async def check_provider(provider_name: str) -> dict: """ Check if a specific provider is available. - + Args: provider_name: Name of the provider to check - + Returns: Status of the provider - + Raises: HTTPException: If the provider is not found """ provider_manager = get_provider_manager() available_providers = provider_manager.available_providers() - + if provider_name not in available_providers: raise HTTPException( status_code=404, - detail=f"Provider '{provider_name}' not found. Available providers: {', '.join(available_providers)}" + detail=f"Provider '{provider_name}' not found. Available providers: {', '.join(available_providers)}", ) - + # Get the provider config provider_config = provider_manager.get_provider_config(provider_name) - + return { "status": "available", "provider": provider_name, @@ -69,13 +69,13 @@ async def check_provider(provider_name: str) -> dict: async def list_provider_models(provider_name: str) -> ProviderModelsResponse: """ List available models for a specific provider. - + Args: provider_name: Name of the provider to get models for - + Returns: List of available models for the provider - + Raises: HTTPException: If the provider is not found """ @@ -85,4 +85,4 @@ async def list_provider_models(provider_name: str) -> ProviderModelsResponse: except ValueError as e: raise HTTPException(status_code=404, detail=str(e)) except Exception as e: - raise HTTPException(status_code=500, detail=f"Error getting models: {str(e)}") \ No newline at end of file + raise HTTPException(status_code=500, detail=f"Error getting models: {str(e)}") diff --git a/servers/inference_server/server/server/features/providers/service.py b/servers/inference_server/server/server/features/providers/service.py index 84f053c2..c4186d7a 100644 --- a/servers/inference_server/server/server/features/providers/service.py +++ b/servers/inference_server/server/server/features/providers/service.py @@ -93,11 +93,8 @@ def get_available_providers() -> List[ProviderInfo]: def _create_model_info(model_id: str, default_model: str) -> ModelInfo: """Create a ModelInfo instance with the given ID and default model.""" - return ModelInfo( - id=model_id, - name=model_id, - is_default=(model_id == default_model) - ) + return ModelInfo(id=model_id, name=model_id, is_default=(model_id == default_model)) + def _extract_model_id(model: Any) -> str: """Extract model ID from a model object.""" @@ -105,6 +102,7 @@ def _extract_model_id(model: Any) -> str: return model.id return model.name if hasattr(model, "name") else str(model) + async def _fetch_models_from_available_models(client: Any, default_model: str) -> List[ModelInfo]: """Fetch models using get_available_models method.""" models = [] @@ -117,6 +115,7 @@ async def _fetch_models_from_available_models(client: Any, default_model: str) - return models + async def _fetch_models_from_get_models(client: Any, default_model: str) -> List[ModelInfo]: """Fetch models using get_models method.""" models = [] @@ -129,10 +128,12 @@ async def _fetch_models_from_get_models(client: Any, default_model: str) -> List return models + def _get_configured_models(config: Any) -> List[ModelInfo]: """Get models from configuration.""" return [_create_model_info(model_id, config.default_model) for model_id in config.models] + async def _fetch_provider_models(client: Any, provider_name: str, config: Any) -> List[ModelInfo]: """Attempt to fetch models from the provider.""" models = [] @@ -152,10 +153,11 @@ async def _fetch_provider_models(client: Any, provider_name: str, config: Any) - return models + async def get_provider_models(provider_name: str) -> List[ModelInfo]: """ Get a list of available models for a specific provider. - + Args: provider_name: Name of the provider to get models for Returns: diff --git a/servers/inference_server/server/server/features/repositories/types.py b/servers/inference_server/server/server/features/repositories/types.py index 05bb458c..d781488b 100644 --- a/servers/inference_server/server/server/features/repositories/types.py +++ b/servers/inference_server/server/server/features/repositories/types.py @@ -1,8 +1,8 @@ from pydantic import BaseModel + class RepositoryParams(BaseModel): repo_location_name: str = "pipelines.definitions" repo_name: str = "__repository__" dagster_host: str = "graphcap_pipelines" # Docker service name for Dagster dagster_port: int = 32300 - \ No newline at end of file diff --git a/servers/inference_server/server/server/pipelines/dagster_client.py b/servers/inference_server/server/server/pipelines/dagster_client.py index 214282b0..da421159 100644 --- a/servers/inference_server/server/server/pipelines/dagster_client.py +++ b/servers/inference_server/server/server/pipelines/dagster_client.py @@ -6,6 +6,7 @@ params = RepositoryParams() + class DagsterClientWrapper: def __init__(self, params: RepositoryParams = RepositoryParams()): """ diff --git a/servers/inference_server/server/server/utils/resizing.py b/servers/inference_server/server/server/utils/resizing.py index ef786ee9..822779fb 100644 --- a/servers/inference_server/server/server/utils/resizing.py +++ b/servers/inference_server/server/server/utils/resizing.py @@ -2,17 +2,19 @@ """ Image Resizing Utilities -This module provides utility functions for resizing images to standard display +This module provides utility functions for resizing images to standard display and video resolutions while maintaining aspect ratio. """ -from loguru import logger from enum import Enum -from typing import Tuple, Union, Optional, Dict, Any from pathlib import Path +from typing import Any, Dict, Optional, Tuple, Union + +from loguru import logger try: from PIL import Image, ImageFile + ImageFile.LOAD_TRUNCATED_IMAGES = True except ImportError: raise ImportError("This module requires Pillow. Install it with 'pip install Pillow'.") @@ -21,12 +23,12 @@ def log_resize_options(options: Dict[str, Any]) -> None: """ Log the resize options to help with debugging. - + Args: options: The options dictionary containing resize settings """ - resize_resolution = options.get('resize_resolution') - + resize_resolution = options.get("resize_resolution") + if resize_resolution: logger.info(f"Image resizing is ENABLED with resolution: {resize_resolution}") else: @@ -35,13 +37,13 @@ def log_resize_options(options: Dict[str, Any]) -> None: class ResolutionPreset(Enum): """Standard display and video resolution presets.""" - - SD_VGA = (640, 480) # Standard Definition VGA - HD_720P = (1280, 720) # High Definition 720p - FHD_1080P = (1920, 1080) # Full High Definition 1080p - QHD_1440P = (2560, 1440) # Quad HD 1440p - UHD_4K = (3840, 2160) # Ultra HD 4K - UHD_8K = (7680, 4320) # 8K UHD + + SD_VGA = (640, 480) # Standard Definition VGA + HD_720P = (1280, 720) # High Definition 720p + FHD_1080P = (1920, 1080) # Full High Definition 1080p + QHD_1440P = (2560, 1440) # Quad HD 1440p + UHD_4K = (3840, 2160) # Ultra HD 4K + UHD_8K = (7680, 4320) # 8K UHD def resize_image( @@ -49,21 +51,19 @@ def resize_image( resolution: Union[ResolutionPreset, Tuple[int, int]], maintain_aspect_ratio: bool = True, upscale: bool = False, - output_format: Optional[str] = None ) -> Image.Image: """ Resize an image to a target resolution while maintaining aspect ratio. - + Args: image: Path to the image file or PIL Image object resolution: Target resolution as ResolutionPreset enum or (width, height) tuple maintain_aspect_ratio: If True, preserve aspect ratio; if False, force to exact dimensions upscale: If True, allow upscaling of images smaller than target resolution - output_format: Output format (e.g., 'JPEG', 'PNG'). If None, uses input format. - + Returns: Resized PIL Image object - + Raises: ValueError: If the image path is invalid or resolution is not recognized IOError: If there's an error opening or processing the image @@ -87,19 +87,23 @@ def resize_image( # Get original dimensions orig_width, orig_height = img.size - - logger.info(f"Resizing image: original dimensions {orig_width}x{orig_height}, target {target_width}x{target_height}") - + + logger.info( + f"Resizing image: original dimensions {orig_width}x{orig_height}, target {target_width}x{target_height}" + ) + # Skip resizing if the image is already smaller than target and upscale is False if not upscale and orig_width <= target_width and orig_height <= target_height: - logger.info(f"Skipping resize: image {orig_width}x{orig_height} already smaller than target {target_width}x{target_height}") + logger.info( + f"Skipping resize: image {orig_width}x{orig_height} already smaller than target {target_width}x{target_height}" + ) return img - + if maintain_aspect_ratio: # Calculate aspect ratios target_ratio = target_width / target_height image_ratio = orig_width / orig_height - + if image_ratio > target_ratio: # Image is wider than target: fit to width new_width = target_width @@ -111,27 +115,25 @@ def resize_image( else: # Force to exact dimensions without maintaining aspect ratio new_width, new_height = target_width, target_height - + # Perform the resize operation resized_img = img.resize((new_width, new_height), Image.LANCZOS) logger.info(f"Image resized: original {orig_width}x{orig_height} → final {new_width}x{new_height}") - + return resized_img def resize_to_fit_max_dimension( - image: Union[str, Path, Image.Image], - max_dimension: int, - upscale: bool = False + image: Union[str, Path, Image.Image], max_dimension: int, upscale: bool = False ) -> Image.Image: """ Resize an image so its largest dimension does not exceed max_dimension. - + Args: image: Path to the image file or PIL Image object max_dimension: Maximum size for the largest dimension upscale: If True, allow upscaling of images smaller than target size - + Returns: Resized PIL Image object """ @@ -140,17 +142,17 @@ def resize_to_fit_max_dimension( img = Image.open(str(image)) else: img = image - + # Get original dimensions width, height = img.size - + logger.info(f"Resizing to max dimension: original dimensions {width}x{height}, max dimension {max_dimension}") - + # Skip if no resizing needed if not upscale and max(width, height) <= max_dimension: logger.info(f"Skipping resize: largest dimension {max(width, height)} already smaller than max {max_dimension}") return img - + # Calculate new dimensions if width >= height: new_width = max_dimension @@ -158,11 +160,11 @@ def resize_to_fit_max_dimension( else: new_height = max_dimension new_width = int(width * (max_dimension / height)) - + # Perform the resize resized_img = img.resize((new_width, new_height), Image.LANCZOS) logger.info(f"Image resized: original {width}x{height} → final {new_width}x{new_height}") - + return resized_img @@ -171,11 +173,11 @@ def save_with_quality( output_path: Union[str, Path], format: Optional[str] = None, quality: int = 90, - optimize: bool = True + optimize: bool = True, ) -> None: """ Save an image with specified quality settings. - + Args: image: PIL Image object to save output_path: Path where the image should be saved @@ -183,13 +185,10 @@ def save_with_quality( quality: Quality setting (0-100, higher is better quality) optimize: Whether to optimize the image """ - save_params = { - 'quality': quality, - 'optimize': optimize - } - + save_params = {"quality": quality, "optimize": optimize} + # Remove quality parameter for formats that don't support it - if format and format.upper() in ('PNG', 'GIF'): - del save_params['quality'] - + if format and format.upper() in ("PNG", "GIF"): + del save_params["quality"] + image.save(str(output_path), format=format, **save_params) diff --git a/servers/inference_server/tests/test_perspective_modules.py b/servers/inference_server/tests/test_perspective_modules.py new file mode 100644 index 00000000..44e479f8 --- /dev/null +++ b/servers/inference_server/tests/test_perspective_modules.py @@ -0,0 +1,187 @@ +""" +# SPDX-License-Identifier: Apache-2.0 +Tests for perspective module loading functionality. +""" + +import json +import tempfile +from pathlib import Path + +import pytest +from graphcap.perspectives.perspective_loader import ( + ModuleConfig, + PerspectiveSettings, + get_all_modules, + load_all_perspectives, +) + + +@pytest.fixture +def temp_perspective_dir(): + """Create a temporary directory for test perspectives.""" + with tempfile.TemporaryDirectory() as temp_dir: + # Create module directories + core_dir = Path(temp_dir) / "core" + core_dir.mkdir() + + experimental_dir = Path(temp_dir) / "experimental" + experimental_dir.mkdir() + + # Create a few test perspectives + core_perspective = { + "name": "test_core", + "display_name": "Test Core", + "version": "1", + "prompt": "Test prompt", + "schema_fields": [{"name": "caption", "type": "str", "description": "A test caption", "is_list": False}], + "table_columns": [{"name": "Caption", "style": "green"}], + "context_template": "\n{caption}\n\n", + "module": "core", + "tags": ["test", "core"], + "description": "A test perspective", + "deprecated": False, + "priority": 10, + } + + with open(core_dir / "test_core.json", "w") as f: + json.dump(core_perspective, f, indent=2) + + # Create a perspective without explicit module (should use directory name) + auto_module_perspective = { + "name": "test_experimental", + "display_name": "Test Experimental", + "version": "0.1", + "prompt": "Test experimental prompt", + "schema_fields": [{"name": "caption", "type": "str", "description": "A test caption", "is_list": False}], + "table_columns": [{"name": "Caption", "style": "yellow"}], + "context_template": "\n{caption}\n\n", + "tags": ["test", "experimental"], + "description": "An experimental test perspective", + "deprecated": False, + "priority": 20, + } + + with open(experimental_dir / "test_experimental.json", "w") as f: + json.dump(auto_module_perspective, f, indent=2) + + # Also add a perspective in the root directory + root_perspective = { + "name": "test_root", + "display_name": "Test Root", + "version": "1", + "prompt": "Test root prompt", + "schema_fields": [{"name": "caption", "type": "str", "description": "A test caption", "is_list": False}], + "table_columns": [{"name": "Caption", "style": "blue"}], + "context_template": "\n{caption}\n\n", + "module": "default", + "tags": ["test", "default"], + "description": "A default module test perspective", + "deprecated": False, + "priority": 30, + } + + with open(Path(temp_dir) / "test_root.json", "w") as f: + json.dump(root_perspective, f, indent=2) + + yield temp_dir + + +@pytest.fixture +def settings(): + """Create test settings.""" + return PerspectiveSettings( + modules={ + "core": ModuleConfig(enabled=True, display_name="Core Test"), + "experimental": ModuleConfig(enabled=False, display_name="Experimental Test"), + "default": ModuleConfig(enabled=True, display_name="Default Test"), + }, + local_override=True, + ) + + +def test_load_perspectives_from_modules(temp_perspective_dir, settings): + """Test loading perspectives from module directories.""" + # Test loading with all modules + all_modules = get_all_modules([Path(temp_perspective_dir)], settings) + + # Check that we have the expected modules + assert set(all_modules.keys()) == {"core", "experimental", "default"} + + # Check module display names from settings + assert all_modules["core"].display_name == "Core Test" + assert all_modules["experimental"].display_name == "Experimental Test" + assert all_modules["default"].display_name == "Default Test" + + # Check module enabled status + assert all_modules["core"].enabled is True + assert all_modules["experimental"].enabled is False + assert all_modules["default"].enabled is True + + # Check that each module has the correct perspectives + assert "test_core" in all_modules["core"].perspectives + assert "test_experimental" in all_modules["experimental"].perspectives + assert "test_root" in all_modules["default"].perspectives + + # Test loading perspectives respecting module enabled status + perspectives = load_all_perspectives([Path(temp_perspective_dir)], settings) + + # Experimental module is disabled, so its perspective shouldn't be included + assert "test_core" in perspectives + assert "test_experimental" not in perspectives + assert "test_root" in perspectives + + +def test_perspective_module_toggle(temp_perspective_dir, settings): + """Test toggling modules on and off.""" + all_modules = get_all_modules([Path(temp_perspective_dir)], settings) + + # Initially, experimental is disabled + assert all_modules["experimental"].enabled is False + + # Toggle experimental on + all_modules["experimental"].toggle(True) + assert all_modules["experimental"].enabled is True + + # Create a new settings object with experimental enabled + updated_settings = PerspectiveSettings( + modules={ + "core": ModuleConfig(enabled=True, display_name="Core Test"), + "experimental": ModuleConfig(enabled=True, display_name="Experimental Test"), + "default": ModuleConfig(enabled=True, display_name="Default Test"), + }, + local_override=True, + ) + + # Now load perspectives again, should include experimental + perspectives = load_all_perspectives([Path(temp_perspective_dir)], updated_settings) + + assert "test_experimental" in perspectives + + +def test_auto_module_detection(temp_perspective_dir): + """Test that perspectives without explicit module are assigned based on directory.""" + # Load without providing settings + all_modules = get_all_modules([Path(temp_perspective_dir)]) + + # The experimental perspective should be in the experimental module + assert "test_experimental" in all_modules["experimental"].perspectives + + # The perspective itself should have the right module + assert all_modules["experimental"].perspectives["test_experimental"].module_name == "experimental" + + +def test_perspective_metadata(temp_perspective_dir, settings): + """Test that perspective metadata is correctly accessible.""" + perspectives = load_all_perspectives([Path(temp_perspective_dir)], settings) + + # Check that core perspective has the right metadata + assert perspectives["test_core"].module_name == "core" + assert perspectives["test_core"].tags == ["test", "core"] + assert perspectives["test_core"].description == "A test perspective" + assert perspectives["test_core"].is_deprecated is False + assert perspectives["test_core"].priority == 10 + + # Check root perspective + assert perspectives["test_root"].module_name == "default" + assert "default" in perspectives["test_root"].tags + assert perspectives["test_root"].priority == 30 diff --git a/servers/inference_server/uv.lock b/servers/inference_server/uv.lock index 9a23d2b1..6636e2a5 100644 --- a/servers/inference_server/uv.lock +++ b/servers/inference_server/uv.lock @@ -24,6 +24,20 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/46/eb/e7f063ad1fec6b3178a3cd82d1a3c4de82cccf283fc42746168188e1cdd5/anyio-4.8.0-py3-none-any.whl", hash = "sha256:b5011f270ab5eb0abf13385f851315585cc37ef330dd88e27ec3d34d651fd47a", size = 96041 }, ] +[[package]] +name = "build" +version = "1.2.2.post1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "os_name == 'nt'" }, + { name = "packaging" }, + { name = "pyproject-hooks" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/7d/46/aeab111f8e06793e4f0e421fcad593d547fb8313b50990f31681ee2fb1ad/build-1.2.2.post1.tar.gz", hash = "sha256:b36993e92ca9375a219c99e606a122ff365a760a2d4bba0caa09bd5278b608b7", size = 46701 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/84/c2/80633736cd183ee4a62107413def345f7e6e3c01563dbca1417363cf957e/build-1.2.2.post1-py3-none-any.whl", hash = "sha256:1d61c0887fa860c01971625baae8bdd338e517b836a2f70dd1f7aa3a6b2fc5b5", size = 22950 }, +] + [[package]] name = "certifi" version = "2025.1.31" @@ -61,6 +75,14 @@ dependencies = [ { name = "openai" }, { name = "pydantic" }, { name = "rich" }, + { name = "tenacity" }, +] + +[package.dev-dependencies] +dev = [ + { name = "build" }, + { name = "pytest" }, + { name = "ruff" }, ] [package.metadata] @@ -70,6 +92,14 @@ requires-dist = [ { name = "openai", specifier = ">=1.63.2" }, { name = "pydantic", specifier = ">=2.10.6" }, { name = "rich", specifier = ">=13.9.4" }, + { name = "tenacity", specifier = ">=9.0.0" }, +] + +[package.metadata.requires-dev] +dev = [ + { name = "build", specifier = ">=1.2.2.post1" }, + { name = "pytest", specifier = ">=8.3.4" }, + { name = "ruff", specifier = ">=0.9.6" }, ] [[package]] @@ -118,6 +148,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl", hash = "sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3", size = 70442 }, ] +[[package]] +name = "iniconfig" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/4b/cbd8e699e64a6f16ca3a8220661b5f83792b3017d0f79807cb8708d33913/iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3", size = 4646 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ef/a6/62565a6e1cf69e10f5727360368e451d4b7f58beeac6173dc9db836a5b46/iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374", size = 5892 }, +] + [[package]] name = "jiter" version = "0.8.2" @@ -218,6 +257,24 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/15/64/db3462b358072387b8e93e6e6a38d3c741a17b4a84171ef01d6c85c63f25/openai-1.63.2-py3-none-any.whl", hash = "sha256:1f38b27b5a40814c2b7d8759ec78110df58c4a614c25f182809ca52b080ff4d4", size = 472282 }, ] +[[package]] +name = "packaging" +version = "24.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d0/63/68dbb6eb2de9cb10ee4c9c14a0148804425e13c4fb20d61cce69f53106da/packaging-24.2.tar.gz", hash = "sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f", size = 163950 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", hash = "sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", size = 65451 }, +] + +[[package]] +name = "pluggy" +version = "1.5.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/96/2d/02d4312c973c6050a18b314a5ad0b3210edb65a906f868e31c111dede4a6/pluggy-1.5.0.tar.gz", hash = "sha256:2cffa88e94fdc978c4c574f15f9e59b7f4201d439195c3715ca9e2486f1d0cf1", size = 67955 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/88/5f/e351af9a41f866ac3f1fac4ca0613908d9a41741cfcf2228f4ad853b697d/pluggy-1.5.0-py3-none-any.whl", hash = "sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669", size = 20556 }, +] + [[package]] name = "pydantic" version = "2.10.6" @@ -294,6 +351,30 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/8a/0b/9fcc47d19c48b59121088dd6da2488a49d5f72dacf8262e2790a1d2c7d15/pygments-2.19.1-py3-none-any.whl", hash = "sha256:9ea1544ad55cecf4b8242fab6dd35a93bbce657034b0611ee383099054ab6d8c", size = 1225293 }, ] +[[package]] +name = "pyproject-hooks" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/82/28175b2414effca1cdac8dc99f76d660e7a4fb0ceefa4b4ab8f5f6742925/pyproject_hooks-1.2.0.tar.gz", hash = "sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8", size = 19228 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", hash = "sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", size = 10216 }, +] + +[[package]] +name = "pytest" +version = "8.3.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "iniconfig" }, + { name = "packaging" }, + { name = "pluggy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/3c/c9d525a414d506893f0cd8a8d0de7706446213181570cdbd766691164e40/pytest-8.3.5.tar.gz", hash = "sha256:f4efe70cc14e511565ac476b57c279e12a855b11f48f212af1080ef2263d3845", size = 1450891 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/30/3d/64ad57c803f1fa1e963a7946b6e0fea4a70df53c1a7fed304586539c2bac/pytest-8.3.5-py3-none-any.whl", hash = "sha256:c69214aa47deac29fad6c2a4f590b9c4a9fdb16a403176fe154b79c0b4d4d820", size = 343634 }, +] + [[package]] name = "rich" version = "13.9.4" @@ -307,6 +388,31 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/19/71/39c7c0d87f8d4e6c020a393182060eaefeeae6c01dab6a84ec346f2567df/rich-13.9.4-py3-none-any.whl", hash = "sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90", size = 242424 }, ] +[[package]] +name = "ruff" +version = "0.11.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/77/2b/7ca27e854d92df5e681e6527dc0f9254c9dc06c8408317893cf96c851cdd/ruff-0.11.0.tar.gz", hash = "sha256:e55c620690a4a7ee6f1cccb256ec2157dc597d109400ae75bbf944fc9d6462e2", size = 3799407 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/48/40/3d0340a9e5edc77d37852c0cd98c5985a5a8081fc3befaeb2ae90aaafd2b/ruff-0.11.0-py3-none-linux_armv6l.whl", hash = "sha256:dc67e32bc3b29557513eb7eeabb23efdb25753684b913bebb8a0c62495095acb", size = 10098158 }, + { url = "https://files.pythonhosted.org/packages/ec/a9/d8f5abb3b87b973b007649ac7bf63665a05b2ae2b2af39217b09f52abbbf/ruff-0.11.0-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:38c23fd9bdec4eb437b4c1e3595905a0a8edfccd63a790f818b28c78fe345639", size = 10879071 }, + { url = "https://files.pythonhosted.org/packages/ab/62/aaa198614c6211677913ec480415c5e6509586d7b796356cec73a2f8a3e6/ruff-0.11.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:7c8661b0be91a38bd56db593e9331beaf9064a79028adee2d5f392674bbc5e88", size = 10247944 }, + { url = "https://files.pythonhosted.org/packages/9f/52/59e0a9f2cf1ce5e6cbe336b6dd0144725c8ea3b97cac60688f4e7880bf13/ruff-0.11.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b6c0e8d3d2db7e9f6efd884f44b8dc542d5b6b590fc4bb334fdbc624d93a29a2", size = 10421725 }, + { url = "https://files.pythonhosted.org/packages/a6/c3/dcd71acc6dff72ce66d13f4be5bca1dbed4db678dff2f0f6f307b04e5c02/ruff-0.11.0-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3c3156d3f4b42e57247275a0a7e15a851c165a4fc89c5e8fa30ea6da4f7407b8", size = 9954435 }, + { url = "https://files.pythonhosted.org/packages/a6/9a/342d336c7c52dbd136dee97d4c7797e66c3f92df804f8f3b30da59b92e9c/ruff-0.11.0-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:490b1e147c1260545f6d041c4092483e3f6d8eba81dc2875eaebcf9140b53905", size = 11492664 }, + { url = "https://files.pythonhosted.org/packages/84/35/6e7defd2d7ca95cc385ac1bd9f7f2e4a61b9cc35d60a263aebc8e590c462/ruff-0.11.0-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:1bc09a7419e09662983b1312f6fa5dab829d6ab5d11f18c3760be7ca521c9329", size = 12207856 }, + { url = "https://files.pythonhosted.org/packages/22/78/da669c8731bacf40001c880ada6d31bcfb81f89cc996230c3b80d319993e/ruff-0.11.0-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:bcfa478daf61ac8002214eb2ca5f3e9365048506a9d52b11bea3ecea822bb844", size = 11645156 }, + { url = "https://files.pythonhosted.org/packages/ee/47/e27d17d83530a208f4a9ab2e94f758574a04c51e492aa58f91a3ed7cbbcb/ruff-0.11.0-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6fbb2aed66fe742a6a3a0075ed467a459b7cedc5ae01008340075909d819df1e", size = 13884167 }, + { url = "https://files.pythonhosted.org/packages/9f/5e/42ffbb0a5d4b07bbc642b7d58357b4e19a0f4774275ca6ca7d1f7b5452cd/ruff-0.11.0-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:92c0c1ff014351c0b0cdfdb1e35fa83b780f1e065667167bb9502d47ca41e6db", size = 11348311 }, + { url = "https://files.pythonhosted.org/packages/c8/51/dc3ce0c5ce1a586727a3444a32f98b83ba99599bb1ebca29d9302886e87f/ruff-0.11.0-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:e4fd5ff5de5f83e0458a138e8a869c7c5e907541aec32b707f57cf9a5e124445", size = 10305039 }, + { url = "https://files.pythonhosted.org/packages/60/e0/475f0c2f26280f46f2d6d1df1ba96b3399e0234cf368cc4c88e6ad10dcd9/ruff-0.11.0-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:96bc89a5c5fd21a04939773f9e0e276308be0935de06845110f43fd5c2e4ead7", size = 9937939 }, + { url = "https://files.pythonhosted.org/packages/e2/d3/3e61b7fd3e9cdd1e5b8c7ac188bec12975c824e51c5cd3d64caf81b0331e/ruff-0.11.0-py3-none-musllinux_1_2_i686.whl", hash = "sha256:a9352b9d767889ec5df1483f94870564e8102d4d7e99da52ebf564b882cdc2c7", size = 10923259 }, + { url = "https://files.pythonhosted.org/packages/30/32/cd74149ebb40b62ddd14bd2d1842149aeb7f74191fb0f49bd45c76909ff2/ruff-0.11.0-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:049a191969a10897fe052ef9cc7491b3ef6de79acd7790af7d7897b7a9bfbcb6", size = 11406212 }, + { url = "https://files.pythonhosted.org/packages/00/ef/033022a6b104be32e899b00de704d7c6d1723a54d4c9e09d147368f14b62/ruff-0.11.0-py3-none-win32.whl", hash = "sha256:3191e9116b6b5bbe187447656f0c8526f0d36b6fd89ad78ccaad6bdc2fad7df2", size = 10310905 }, + { url = "https://files.pythonhosted.org/packages/ed/8a/163f2e78c37757d035bd56cd60c8d96312904ca4a6deeab8442d7b3cbf89/ruff-0.11.0-py3-none-win_amd64.whl", hash = "sha256:c58bfa00e740ca0a6c43d41fb004cd22d165302f360aaa56f7126d544db31a21", size = 11411730 }, + { url = "https://files.pythonhosted.org/packages/4e/f7/096f6efabe69b49d7ca61052fc70289c05d8d35735c137ef5ba5ef423662/ruff-0.11.0-py3-none-win_arm64.whl", hash = "sha256:868364fc23f5aa122b00c6f794211e85f7e78f5dffdf7c590ab90b8c4e69b657", size = 10538956 }, +] + [[package]] name = "sniffio" version = "1.3.1" @@ -316,6 +422,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235 }, ] +[[package]] +name = "tenacity" +version = "9.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/94/91fccdb4b8110642462e653d5dcb27e7b674742ad68efd146367da7bdb10/tenacity-9.0.0.tar.gz", hash = "sha256:807f37ca97d62aa361264d497b0e31e92b8027044942bfa756160d908320d73b", size = 47421 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b6/cb/b86984bed139586d01532a587464b5805f12e397594f19f931c4c2fbfa61/tenacity-9.0.0-py3-none-any.whl", hash = "sha256:93de0c98785b27fcf659856aa9f54bfbd399e29969b0621bc7f762bd441b4539", size = 28169 }, +] + [[package]] name = "tqdm" version = "4.67.1" diff --git a/test/inference_tests/constants.py b/test/inference_tests/constants.py index 62e27c49..7c7afc73 100644 --- a/test/inference_tests/constants.py +++ b/test/inference_tests/constants.py @@ -19,7 +19,3 @@ class StructuredCaption(BaseModel): scratchpad: str caption: str - - - - diff --git a/test/inference_tests/ollama_client_test.py b/test/inference_tests/ollama_client_test.py index c86bd475..2662876e 100644 --- a/test/inference_tests/ollama_client_test.py +++ b/test/inference_tests/ollama_client_test.py @@ -23,40 +23,40 @@ async def test_structured_vision(): """Test structured vision capabilities with Ollama client.""" # Set up provider configuration config_path = os.environ.get("PROVIDER_CONFIG_PATH", PROVIDER_CONFIG_PATH) - + # Create a minimal provider config if it doesn't exist if not Path(config_path).exists(): logger.warning(f"Provider config not found at {config_path}, creating minimal config") os.makedirs(os.path.dirname(config_path), exist_ok=True) with open(config_path, "w") as f: f.write(EXAMPLE_CONFIG) - + # Initialize provider manager provider_manager = initialize_provider_manager(config_path) - + # Get the Ollama client try: client = get_provider_client("ollama") logger.info(f"Successfully initialized Ollama client with model: {client.default_model}") - + # Debug the base URL logger.info(f"Client base URL: {client.base_url}") except Exception as e: logger.error(f"Failed to initialize Ollama client: {e}") return - + # Define the image path image_path = IMAGE_PATH if not Path(image_path).exists(): logger.error(f"Test image not found at {image_path}") return - + # Define the prompt prompt = ( "Analyze the provided image and generate a structured caption. " "Include a brief scratchpad of your thought process and a final concise caption." ) - + try: # Call the vision method with structured output completion = await client.vision( @@ -64,14 +64,14 @@ async def test_structured_vision(): image=image_path, model=client.default_model, schema=StructuredCaption, - temperature=.7, + temperature=0.7, ) - + # Process the response - if hasattr(completion, 'choices') and hasattr(completion.choices[0].message, 'parsed'): + if hasattr(completion, "choices") and hasattr(completion.choices[0].message, "parsed"): # Access the parsed response parsed_response = completion.choices[0].message.parsed - + # Display as formatted JSON logger.info("Structured Caption (JSON):") formatted_json = json.dumps(parsed_response.model_dump(), indent=2) @@ -79,11 +79,11 @@ async def test_structured_vision(): else: logger.error("Unexpected response format") print("Raw response:", completion) - + except Exception as e: logger.error(f"Error during structured vision test: {e}") if __name__ == "__main__": # Run the async test - asyncio.run(test_structured_vision()) \ No newline at end of file + asyncio.run(test_structured_vision()) diff --git a/test/inference_tests/ollama_graphcap_REST.py b/test/inference_tests/ollama_graphcap_REST.py index 076d0fe7..5937f9cb 100644 --- a/test/inference_tests/ollama_graphcap_REST.py +++ b/test/inference_tests/ollama_graphcap_REST.py @@ -16,4 +16,4 @@ ], "global_context": "You are a structured captioning model." }' -""" \ No newline at end of file +""" diff --git a/test/inference_tests/ollama_structured_test.py b/test/inference_tests/ollama_structured_test.py index d058934b..bbbfed3a 100644 --- a/test/inference_tests/ollama_structured_test.py +++ b/test/inference_tests/ollama_structured_test.py @@ -4,14 +4,17 @@ import openai from pydantic import BaseModel from constants import PROVIDER_CONFIG_PATH, EXAMPLE_CONFIG, StructuredCaption, IMAGE_PATH, OLLAMA_DEFAULT_MODEL + # Initialize the client using the OpenAI compatibility layer. client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") + # Define the StructuredCaption schema. class StructuredCaption(BaseModel): scratchpad: str caption: str + # Define the prompt for generating a structured caption. prompt = ( "Analyze the provided image and generate a structured caption. " @@ -33,7 +36,7 @@ class StructuredCaption(BaseModel): "content": [ {"type": "text", "text": prompt}, {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{b64_image}"}}, - ] + ], } ], response_format=StructuredCaption, @@ -50,4 +53,4 @@ class StructuredCaption(BaseModel): except openai.LengthFinishReasonError as e: print(f"Too many tokens: {e}") except Exception as e: - print(f"Error during API call: {e}") \ No newline at end of file + print(f"Error during API call: {e}") diff --git a/test/inference_tests/rest_api_test.py b/test/inference_tests/rest_api_test.py index cc6604ed..4b7a710f 100644 --- a/test/inference_tests/rest_api_test.py +++ b/test/inference_tests/rest_api_test.py @@ -21,7 +21,7 @@ logger.add( sys.stderr, level="DEBUG", - format="{time:YYYY-MM-DD HH:mm:ss.SSS} | {level: <8} | {name}:{function}:{line} - {message}" + format="{time:YYYY-MM-DD HH:mm:ss.SSS} | {level: <8} | {name}:{function}:{line} - {message}", ) # Constants @@ -34,8 +34,10 @@ logger.debug(f"Using Ollama host: {OLLAMA_HOST}") logger.debug(f"Full Ollama URL: {OLLAMA_BASE_URL}") + class CaptionRequest(BaseModel): """Request model for caption generation.""" + perspective: str image_path: str provider: str @@ -46,11 +48,14 @@ class CaptionRequest(BaseModel): context: Optional[List[str]] = ["string"] global_context: str = "You are a structured captioning model." + class StructuredCaption(BaseModel): """Response model for structured caption.""" + scratchpad: str caption: str + async def check_ollama_health(): """Check if Ollama is running and healthy.""" try: @@ -80,13 +85,14 @@ async def check_ollama_health(): logger.debug(f"Exception details: {e.__cause__ if hasattr(e, '__cause__') else 'No cause info'}") return False + async def test_caption_generation(provider: str): """Test the caption generation REST API endpoint. - + Args: provider: The provider to use for caption generation (e.g., 'ollama' or 'gemini') """ - logger.info(f"\n{'='*50}\nTesting with {provider.upper()} provider\n{'='*50}") + logger.info(f"\n{'=' * 50}\nTesting with {provider.upper()} provider\n{'=' * 50}") # For Ollama provider, check health first if provider == "ollama": @@ -107,7 +113,7 @@ async def test_caption_generation(provider: str): top_p=0.9, repetition_penalty=1.15, context=["string"], - global_context="You are a structured captioning model." + global_context="You are a structured captioning model.", ) endpoint = f"{API_BASE_URL}/perspectives/caption-from-path" @@ -122,10 +128,7 @@ async def test_caption_generation(provider: str): response = await client.post( endpoint, json=request.model_dump(), - headers={ - "accept": "application/json", - "Content-Type": "application/json" - } + headers={"accept": "application/json", "Content-Type": "application/json"}, ) # Log response details @@ -165,6 +168,7 @@ async def test_caption_generation(provider: str): logger.debug(f"Exception details: {e.__cause__ if hasattr(e, '__cause__') else 'No cause info'}") raise + async def run_tests(): """Run tests with different providers.""" logger.debug("Starting test run") @@ -179,6 +183,7 @@ async def run_tests(): logger.debug("Test run completed") + if __name__ == "__main__": # Run all tests logger.debug("Initializing test script") diff --git a/test/library_tests/provider_tests/test_provider_config.py b/test/library_tests/provider_tests/test_provider_config.py index f144c3de..97a20106 100644 --- a/test/library_tests/provider_tests/test_provider_config.py +++ b/test/library_tests/provider_tests/test_provider_config.py @@ -13,7 +13,6 @@ None (contains test functions only) """ - import pytest from graphcap.providers.provider_config import _load_provider_config, get_providers_config, validate_config diff --git a/test/library_tests/provider_tests/test_providers.py b/test/library_tests/provider_tests/test_providers.py index 25b9d539..7eb57045 100644 --- a/test/library_tests/provider_tests/test_providers.py +++ b/test/library_tests/provider_tests/test_providers.py @@ -48,6 +48,7 @@ class StructuredVisionConfig: prompt: str schema: BaseModel + class TestStructuredOutput(BaseModel): is_cat: bool caption: str @@ -155,7 +156,6 @@ async def test_gemini_structured_vision(self, test_logger, provider_manager, pro await run_structured_vision(client, test_logger, image_path) - @pytest.mark.integration @pytest.mark.vllm class TestVLLMProvider: @@ -227,7 +227,6 @@ async def test_vllm_structured_vision(self, test_logger, provider_manager, provi await run_structured_vision(client, test_logger, image_path) - @pytest.mark.asyncio @pytest.mark.integration @pytest.mark.provider diff --git a/workspace/config/perspective_settings.json b/workspace/config/perspective_settings.json new file mode 100644 index 00000000..942df60a --- /dev/null +++ b/workspace/config/perspective_settings.json @@ -0,0 +1,37 @@ +{ + "modules": { + "core": { + "enabled": true, + "display_name": "Core Perspectives" + }, + "narrative": { + "enabled": true, + "display_name": "Narrative Perspectives" + }, + "emotional": { + "enabled": true, + "display_name": "Emotional Perspectives" + }, + "art": { + "enabled": true, + "display_name": "Art Analysis" + }, + "temporal": { + "enabled": true, + "display_name": "Temporal Analysis" + }, + "speculative": { + "enabled": true, + "display_name": "Speculative Perspectives" + }, + "experimental": { + "enabled": false, + "display_name": "Experimental Perspectives" + }, + "deprecated": { + "enabled": false, + "display_name": "Deprecated Perspectives" + } + }, + "local_override": true +} \ No newline at end of file diff --git a/workspace/perspective_library/README.md b/workspace/perspective_library/README.md new file mode 100644 index 00000000..c413bec8 --- /dev/null +++ b/workspace/perspective_library/README.md @@ -0,0 +1,128 @@ +# GraphCap Perspective Library + +## Overview + +The Perspective Library is a collection of specialized analytical frameworks for image understanding and captioning. Each perspective provides a unique lens through which to analyze and describe visual content, from objective factual descriptions to creative interpretations, temporal analyses, and specialized domain-specific insights. + +Think of a perspective as a unique way of looking at and describing an image - just as a photographer, art critic, and child might describe the same photograph differently. The Perspective Library organizes these different viewpoints into a structured, modular system that can be used individually or in combination to generate rich, multi-faceted image descriptions. + +For a comprehensive conceptual overview of perspectives, see the [Perspectives Concept Documentation](../../doc/concepts/perspectives.rst). + +## Library Structure + +The Perspective Library is organized into modules, each containing perspectives with related focuses: + +- **[Core](./core/)**: Fundamental perspectives forming the backbone of the image analysis pipeline, providing objective, structured analysis from complementary angles. These represent our "gold standard" for image captioning. + +- **[Creative](./creative/)**: Perspectives designed to generate "controlled hallucinations" - creative extensions beyond what's directly visible in the image while remaining plausibly connected to the visual content. + +- **[Video](./video/)**: Specialized perspectives for transforming static images into rich, dynamic prompts for video generation, focusing on motion potential, narrative progression, and cinematic elements. + +- **[Niche](./niche/)**: Focused analytical perspectives targeting specific dimensions of visual content, extracting specialized information optimized for particular use cases or domains. + +Each module contains a README with detailed information about its specific perspectives and their applications. + +## Using Perspectives + +Perspectives can be used in several ways: + +### 1. Through the GraphCap UI + +The easiest way to use perspectives is through the GraphCap web interface: + +1. Select an image from your dataset +2. Choose a perspective from the right hand panel +3. Generate the analysis +4. View the structured output with all fields + +![Perspective Generation](../../doc/static/generate_perspective.png) + +## Creating New Perspectives + +To create a new perspective: + +1. **Determine the Module**: Decide which module your perspective belongs to (core, creative, video, niche), or create a new module if needed. + +2. **Create the JSON File**: Create a new JSON file in the appropriate module directory with the following structure: + +```json +{ + "name": "my_perspective", + "display_name": "My Perspective", + "version": "1", + "prompt": "Your detailed prompt for the perspective...", + "schema_fields": [ + { + "name": "field_name", + "type": "str", + "description": "Description of this field.", + "is_list": false + }, + // Additional fields... + ], + "table_columns": [ + { + "name": "Column Name", + "style": "cyan" + }, + { + "name": "Value", + "style": "green" + } + ], + "context_template": "\n{field_name}\n\n", + "module": "your_module", + "type": "perspective", + "tags": ["relevant", "tags", "here"], + "description": "A concise description of your perspective.", + "deprecated": false, + "priority": 50 +} +``` + +3. **Structure Your Prompt**: Craft a clear, detailed prompt that guides the AI to produce structured output in the desired format. Make sure it: + - Clearly defines the analytical focus + - Specifies what elements to look for + - Provides a structured format for responses + - Maintains objectivity or clearly specifies where interpretation is allowed + +4. **Define Schema Fields**: Create fields that capture the structured output of your perspective. Each field should have: + - A clear name + - Type (str, int, float, dict) + - Description + - Whether it's a list or a single value + - For complex fields, nested field definitions + +5. **Test Your Perspective**: Before committing, test your perspective with a variety of images to ensure it consistently produces high-quality, useful output. + +6. **Document Your Perspective**: Update the module's README.md to include your new perspective, its purpose, and use cases. + +For more detailed guidance on creating effective perspectives, refer to the [Perspectives Concept Documentation](../../doc/concepts/perspectives.rst). + +## Integration with Other Systems + +Perspectives can be integrated with: + +- **Training Data Generation**: Use perspectives to create rich, structured annotations for ML training +- **Content Management Systems**: Enhance image metadata with perspective analyses +- **Creative Tools**: Generate prompts and inspiration from perspective outputs +- **Search Systems**: Enable sophisticated image retrieval based on perspective-generated attributes + +## Best Practices + +- **Combine Complementary Perspectives**: Pair factual perspectives with interpretative ones for comprehensive analysis +- **Match Perspectives to Use Cases**: Select perspectives based on your specific needs +- **Version Control**: When updating perspectives, increment the version number and document changes +- **Validate Outputs**: Periodically review perspective outputs to ensure quality and accuracy + +## Contributing + +Contributions to the Perspective Library are welcome! To contribute: + +1. Fork the repository +2. Create your perspective following the guidelines above +3. Add appropriate documentation +4. Submit a pull request with a clear description of your new perspective + +For more information on contributing, see the [Contributing Guide](../../CONTRIBUTING.md). + diff --git a/workspace/perspective_library/core/README.md b/workspace/perspective_library/core/README.md new file mode 100644 index 00000000..63df1661 --- /dev/null +++ b/workspace/perspective_library/core/README.md @@ -0,0 +1,67 @@ +# Core Perspectives Module + +## Overview + +The Core Perspectives module contains fundamental perspectives that form the backbone of the Open Model Initiative (OMI) pipeline. These perspectives represent our "gold standard" for image analysis and captioning, providing comprehensive, structured, and objective analysis of visual content from complementary angles. Unlike specialized perspectives that focus on specific aspects or creative interpretations, core perspectives are designed to extract factual, verifiable information with high precision and reliability. + +These perspectives are particularly valuable for: + +- Building robust foundation models for image understanding +- Creating high-quality, structured training datasets +- Supporting objective image search and retrieval +- Establishing baseline annotations against which specialized analyses can be compared +- Powering downstream applications that require reliable image understanding + +## Core Perspectives + +| Perspective | Description | Primary Focus | Use Cases | +|-------------|-------------|--------------|-----------| +| **Graph Caption** | Structured analysis generating categorized tags with confidence scores, short captions, verification, and dense descriptions. | Objective content analysis with confidence scoring | Training data generation, searchable image databases, content verification | +| **Locality Graph** | Spatial relationship analysis capturing objects as nodes and their relationships as edges in a visual composition graph. | Explicit spatial relationships between visual elements | Scene understanding, layout analysis, relational datasets | +| **Art Critic** | Formal analysis examining visual elements, technical execution, and stylistic choices with objective, observable language. | Objective formal analysis without interpretation | Visual style datasets, art analysis, aesthetic categorization | +| **Synthesized Caption** | Integration of multiple perspectives into cohesive analysis with explicit reasoning and multiple levels of detail. | Multi-perspective synthesis with structured reasoning | Comprehensive image understanding, dataset enrichment, explainable AI training | +| **Structured Style Presets** | Generation of detailed style specifications with theme, color palette, camera settings, lighting, texture, and mood. | Structured aesthetic style specification | Style transfer, generation guidance, aesthetic categorization | + +## The Value of Core Perspectives + +The core perspectives collectively provide a comprehensive foundation for image understanding through several key principles: + +1. **Structured Objectivity**: Each perspective focuses on observable elements that can be reliably identified and verified, creating annotations with high agreement rates among human evaluators. + +2. **Complementary Analysis Angles**: The perspectives approach visual content from different analytical frameworks (graph-based, spatial, formal, synthetic), ensuring complete coverage of relevant information. + +3. **Confidence Quantification**: Where applicable, elements include confidence scoring to indicate reliability, supporting applications that require understanding of model certainty. + +4. **Multi-granular Analysis**: From concise tags to detailed descriptions, the perspectives provide annotations at multiple levels of detail suitable for different downstream tasks. + +5. **Explicit Reasoning**: The analysis includes clear explanation of the reasoning process, supporting applications that require explainability and transparency. + +## Dataset Applications + +These core perspectives are particularly valuable for dataset creation in several contexts: + +### Foundation Model Training + +The structured, objective annotations provided by core perspectives serve as high-quality training data for foundation models, establishing reliable correlations between visual content and language descriptions. + +### Benchmark Standards + +The multi-faceted nature of core perspectives makes them ideal for creating evaluation benchmarks that assess model performance across different dimensions of image understanding. + +### Multimodal Retrieval Systems + +The granular, structured annotations (particularly tags with confidence scores and spatial relationships) enable sophisticated image retrieval systems that can search based on specific visual attributes and relationships. + +### Explainable AI Development + +The explicit reasoning and structured analysis approach supports the development of explainable AI systems that can articulate the basis for their visual interpretations. + +## Integration Guidelines + +When using core perspectives for dataset creation: + +1. Use Graph Caption and Locality Graph for foundational object and relationship identification +2. Apply Art Critic for formal analysis of visual elements and composition +3. Leverage Synthesized Caption to integrate insights across perspectives with explicit reasoning +4. Utilize Style Presets for aesthetic categorization and style specification +5. Combine core perspectives with specialized perspectives for comprehensive dataset enrichment diff --git a/workspace/config/perspectives/art_critic.json b/workspace/perspective_library/core/art_critic.json similarity index 85% rename from workspace/config/perspectives/art_critic.json rename to workspace/perspective_library/core/art_critic.json index 2c484050..2c08612c 100644 --- a/workspace/config/perspectives/art_critic.json +++ b/workspace/perspective_library/core/art_critic.json @@ -45,5 +45,11 @@ "style": "green" } ], - "context_template": "\n{formal_analysis}\n\n" + "context_template": "\n{formal_analysis}\n\n", + "module": "core", + "type": "perspective", + "tags": ["art", "formal-analysis", "visual-elements", "objective"], + "description": "An art critic perspective that focuses on formal analysis principles, providing objective observations of visual elements, technical aspects, and stylistic choices without subjective interpretation.", + "deprecated": false, + "priority": 30 } \ No newline at end of file diff --git a/workspace/config/perspectives/graph_caption.json b/workspace/perspective_library/core/graph_caption.json similarity index 91% rename from workspace/config/perspectives/graph_caption.json rename to workspace/perspective_library/core/graph_caption.json index d728a71b..05e67c60 100644 --- a/workspace/config/perspectives/graph_caption.json +++ b/workspace/perspective_library/core/graph_caption.json @@ -57,5 +57,11 @@ "style": "green" } ], - "context_template": "\n{short_caption}\n\nTags: {tags_list}\n\n" + "context_template": "\n{short_caption}\n\nTags: {tags_list}\n\n", + "module": "core", + "type": "perspective", + "tags": ["graph", "caption", "analysis"], + "description": "Graph caption analysis", + "deprecated": false, + "priority": 50 } \ No newline at end of file diff --git a/workspace/config/perspectives/locality_graph.json b/workspace/perspective_library/core/locality_graph.json similarity index 92% rename from workspace/config/perspectives/locality_graph.json rename to workspace/perspective_library/core/locality_graph.json index 1341d06a..b9001640 100644 --- a/workspace/config/perspectives/locality_graph.json +++ b/workspace/perspective_library/core/locality_graph.json @@ -62,5 +62,11 @@ "style": "green" } ], - "context_template": "\n{composition_summary}\nNodes: {nodes}\nEdges: {edges}\n" + "context_template": "\n{composition_summary}\nNodes: {nodes}\nEdges: {edges}\n", + "module": "core", + "type": "perspective", + "tags": ["graph", "locality", "analysis"], + "description": "Graph locality analysis", + "deprecated": false, + "priority": 50 } diff --git a/workspace/perspective_library/core/style_preset.json b/workspace/perspective_library/core/style_preset.json new file mode 100644 index 00000000..af15b7e7 --- /dev/null +++ b/workspace/perspective_library/core/style_preset.json @@ -0,0 +1,67 @@ +{ + "name": "style_preset", + "display_name": "Structured Style Presets", + "version": "1", + "prompt": "Generate a **structured style preset** that defines a unique visual and thematic quality. Each preset should be concise yet richly descriptive, covering:\n\n1. **Theme**: Clearly state the overarching aesthetic or concept.\n2. **Color Palette**: Define the primary color tones, contrast, and saturation.\n3. **Camera Type & Film Stock**: Specify the camera style, film stock, or digital settings used to achieve the look.\n4. **Lighting**: Describe the lighting setup that enhances the mood (e.g., high-key, soft diffused, chiaroscuro).\n5. **Texture & Rendering**: Highlight key textures, grain, or surface qualities that define the preset.\n6. **Vibe**: Summarize the emotional tone and intended effect.\n\nThe result should be structured as a **formatted preset block**, ensuring clarity and usability.", + "schema_fields": [ + { + "name": "theme", + "type": "str", + "description": "The central artistic or conceptual theme defining the preset.", + "is_list": false + }, + { + "name": "color_palette", + "type": "str", + "description": "The dominant colors, contrast, and saturation levels that shape the aesthetic.", + "is_list": false + }, + { + "name": "camera_and_film", + "type": "str", + "description": "The type of camera, film stock, or digital settings that define the visual style.", + "is_list": false + }, + { + "name": "lighting", + "type": "str", + "description": "Description of the lighting setup and how it interacts with the scene.", + "is_list": false + }, + { + "name": "texture_and_rendering", + "type": "str", + "description": "Key textures, grain, or material properties that contribute to the look.", + "is_list": false + }, + { + "name": "vibe", + "type": "str", + "description": "The overall emotional tone or impact of the preset.", + "is_list": false + }, + { + "name": "final_preset", + "type": "str", + "description": "A structured, formatted preset encapsulating all key elements.", + "is_list": false + } + ], + "table_columns": [ + { + "name": "Component", + "style": "cyan" + }, + { + "name": "Description", + "style": "green" + } + ], + "context_template": "\n{final_preset}\n", + "module": "core", + "type": "perspective", + "tags": ["style", "preset", "aesthetic", "cinematic"], + "description": "A perspective that generates structured style presets for detailed, thematic image and video generation.", + "deprecated": false, + "priority": 50 +} diff --git a/workspace/config/perspectives/synthesized_caption.json b/workspace/perspective_library/core/synthesized_caption.json similarity index 89% rename from workspace/config/perspectives/synthesized_caption.json rename to workspace/perspective_library/core/synthesized_caption.json index 2d3073af..e69f5690 100644 --- a/workspace/config/perspectives/synthesized_caption.json +++ b/workspace/perspective_library/core/synthesized_caption.json @@ -51,5 +51,11 @@ {"name": "Component", "style": "cyan"}, {"name": "Content", "style": "green"} ], - "context_template": "\n{short_caption}\n\nTags: {synthesis_tags}\n\n" + "context_template": "\n{short_caption}\n\nTags: {synthesis_tags}\n\n", + "module": "core", + "type": "synthesizer", + "tags": ["synthesis", "integration", "comprehensive", "multi-perspective"], + "description": "A meta-perspective that integrates insights from multiple other perspectives to create a comprehensive, cohesive analysis with multiple levels of detail and explicit reasoning.", + "deprecated": false, + "priority": 5 } diff --git a/workspace/perspective_library/creative/README.md b/workspace/perspective_library/creative/README.md new file mode 100644 index 00000000..6411c2c9 --- /dev/null +++ b/workspace/perspective_library/creative/README.md @@ -0,0 +1,61 @@ +# Creative Perspectives Module + +## Overview + +The Creative Perspectives module contains a collection of specialized perspectives designed to generate "controlled hallucinations" from images. Unlike factual perspectives that focus on objectively describing what's visible, creative perspectives extend beyond the frame, exploring imaginative interpretations, narratives, and speculative elements that might not be directly observable but are plausibly connected to the visual content. + +These perspectives are particularly valuable for: + +- Generating rich, imaginative training data for creative AI applications +- Exploring potential narratives and context beyond what's visually present +- Creating diverse interpretations of the same visual content +- Supporting creative writing, storytelling, and content generation workflows +- Developing more nuanced understanding of visual media through multiple interpretative lenses + +## Creative Perspectives + +| Perspective | Description | Primary Focus | Use Cases | +|-------------|-------------|--------------|-----------| +| **Temporarium** | A temporal analysis perspective examining historical or futuristic elements with chain-of-thought reasoning about epochs, continuity, and temporal speculation. | Time-related aspects and temporal dimensions | Historical datasets, timeline creation, period piece generation | +| **Simple Temporarium** | A simplified version of Temporarium focusing on basic temporal analysis with less complexity and reasoning steps. | Simplified time analysis | Educational datasets, basic historical labeling | +| **Out of Frame** | Speculative interpretation of what might exist beyond the visible boundaries of the image. | Spatial and narrative extension | Scene completion, environment building, world-building datasets | +| **Storytelling** | Constructs cohesive narratives from visual elements, including scene setting, character development, plot, and themes. | Narrative construction | Story generation, creative writing prompts, screenplay development | +| **Poetic Metaphor** | Creates poetic interpretations using metaphor, imagery, and figurative language to evoke deeper meaning. | Figurative and symbolic interpretation | Poetry generation, artistic description, emotional datasets | + +## The Value of Controlled Hallucinations + +While the term "hallucination" is often considered negative in AI contexts, controlled and intentional creative extensions can be extremely valuable. The perspectives in this module are designed to produce *plausible* creative interpretations that: + +1. **Remain grounded** in the visual evidence present in the image +2. **Explicitly separate** factual observation from creative extension +3. **Apply consistent reasoning** to develop interpretations +4. **Follow structured frameworks** for different types of creative analysis + +## Dataset Applications + +These creative perspectives are particularly valuable for dataset creation in several contexts: + +### Training Data Diversity + +When training generative models, having diverse interpretations of the same content helps create more nuanced and versatile outputs. Creative perspectives can significantly expand the range of textual descriptions associated with images. + +### Cross-Modal Learning + +For models that need to understand relationships between visual content and creative text (such as image generation from creative prompts), these perspectives provide valuable bridging data. + +### Educational Content + +In educational contexts, these perspectives can help generate explanatory content that goes beyond simple descriptions, helping learners understand historical context, narrative structure, or symbolic interpretation. + +### Creative Applications + +For applications focused on creative writing, storytelling, or artistic content generation, these perspectives provide structured approaches to deriving creative content from visual stimuli. + +## Integration Guidelines + +When using creative perspectives for dataset creation: + +1. Consider combining them with factual perspectives for a balanced approach +2. Use appropriate metadata to indicate which aspects are observational vs. interpretative +3. Apply filtering or human review when very high factual accuracy is required +4. Experiment with perspective combinations to achieve different creative effects diff --git a/workspace/config/perspectives/out_of_frame.json b/workspace/perspective_library/creative/out_of_frame.json similarity index 89% rename from workspace/config/perspectives/out_of_frame.json rename to workspace/perspective_library/creative/out_of_frame.json index 5e3b8af2..d35d470e 100644 --- a/workspace/config/perspectives/out_of_frame.json +++ b/workspace/perspective_library/creative/out_of_frame.json @@ -45,5 +45,11 @@ "style": "green" } ], - "context_template": "\n{narrative_possibilities}\n\n" + "context_template": "\n{narrative_possibilities}\n\n", + "module": "creative", + "type": "perspective", + "tags": ["speculative", "imagination", "beyond-frame"], + "description": "Speculative interpretation of what might exist beyond the boundaries of the image", + "deprecated": false, + "priority": 50 } \ No newline at end of file diff --git a/workspace/config/perspectives/poetic_metaphor.json b/workspace/perspective_library/creative/poetic_metaphor.json similarity index 90% rename from workspace/config/perspectives/poetic_metaphor.json rename to workspace/perspective_library/creative/poetic_metaphor.json index 57c4cae6..75888f7c 100644 --- a/workspace/config/perspectives/poetic_metaphor.json +++ b/workspace/perspective_library/creative/poetic_metaphor.json @@ -45,5 +45,11 @@ "style": "green" } ], - "context_template": "\n{poetic_description}\n\n" + "context_template": "\n{poetic_description}\n\n", + "module": "creative", + "type": "perspective", + "tags": ["poetic", "metaphor", "imagery", "figurative"], + "description": "Poetic metaphor analysis", + "deprecated": false, + "priority": 50 } \ No newline at end of file diff --git a/workspace/config/perspectives/simple_temporarium.json b/workspace/perspective_library/creative/simple_temporarium.json similarity index 86% rename from workspace/config/perspectives/simple_temporarium.json rename to workspace/perspective_library/creative/simple_temporarium.json index 2748c045..efefaa28 100644 --- a/workspace/config/perspectives/simple_temporarium.json +++ b/workspace/perspective_library/creative/simple_temporarium.json @@ -51,5 +51,11 @@ "style": "green" } ], - "context_template": "\n{simple_caption}\n" + "context_template": "\n{temporal_caption}\n\n", + "module": "creative", + "type": "perspective", + "tags": ["time", "historical", "simplified", "era", "indicators"], + "description": "A simplified version of the Temporarium perspective that focuses on basic temporal analysis with less complexity and reasoning steps.", + "deprecated": false, + "priority": 45 } \ No newline at end of file diff --git a/workspace/config/perspectives/storytelling.json b/workspace/perspective_library/creative/storytelling.json similarity index 85% rename from workspace/config/perspectives/storytelling.json rename to workspace/perspective_library/creative/storytelling.json index 00d41ea9..d7504e44 100644 --- a/workspace/config/perspectives/storytelling.json +++ b/workspace/perspective_library/creative/storytelling.json @@ -45,5 +45,11 @@ "style": "green" } ], - "context_template": "\n{plot_elements}\n\n" + "context_template": "\n{plot_elements}\n\n", + "module": "creative", + "type": "perspective", + "tags": ["storytelling", "narrative", "creative", "plot", "characters"], + "description": "A narrative-focused perspective that constructs a cohesive story from visual elements, including scene setting, character development, plot, and thematic analysis.", + "deprecated": false, + "priority": 20 } \ No newline at end of file diff --git a/workspace/config/perspectives/temporarium.json b/workspace/perspective_library/creative/temporarium.json similarity index 89% rename from workspace/config/perspectives/temporarium.json rename to workspace/perspective_library/creative/temporarium.json index 6fc08c19..f902af43 100644 --- a/workspace/config/perspectives/temporarium.json +++ b/workspace/perspective_library/creative/temporarium.json @@ -75,5 +75,11 @@ "style": "green" } ], - "context_template": "\n{detailed_caption}\n\n" + "context_template": "\n{detailed_caption}\n\n", + "module": "creative", + "type": "perspective", + "tags": ["time", "historical", "futuristic", "epoch", "continuity", "speculative"], + "description": "A perspective focused on temporal analysis, examining historical or futuristic elements, epoch context, historical continuity, and temporal speculation with chain-of-thought reasoning.", + "deprecated": false, + "priority": 40 } \ No newline at end of file diff --git a/workspace/perspective_library/niche/README.md b/workspace/perspective_library/niche/README.md new file mode 100644 index 00000000..fecbcc58 --- /dev/null +++ b/workspace/perspective_library/niche/README.md @@ -0,0 +1,54 @@ +# Niche Analysis Perspectives Module + +## Overview + +The Niche Analysis Perspectives module contains specialized perspectives designed for focused, targeted analysis of particular aspects of images. Unlike broader perspectives that provide comprehensive descriptions, niche perspectives drill down into specific dimensions of visual content, extracting specialized information optimized for particular use cases or domains. + +These perspectives are particularly valuable for: + +- Extracting specific types of information from visual content +- Enhancing datasets with specialized metadata +- Supporting domain-specific applications and research +- Enriching broader analyses with targeted insights +- Creating training data for specialized machine learning models + +## Niche Perspectives + +| Perspective | Description | Primary Focus | Use Cases | +|-------------|-------------|--------------|-----------| +| **Emotional Sentiment** | Analyzes the emotional tone conveyed by images, providing insights into mood and sentiment through descriptive language. | Affective dimensions and emotional impact | Sentiment analysis datasets, emotional tagging, mood-based recommendations | +| **Time Tags** | Categorizes images based on time periods, temporal indicators, and time-related qualities for historical or fantasy classification. | Temporal characteristics and historical context | Historical archives, period piece datasets, timeline construction | + +## The Value of Specialized Analysis + +While broader perspectives provide comprehensive coverage, niche perspectives offer several unique advantages: + +1. **Depth over Breadth**: By focusing on a single dimension of analysis, niche perspectives can provide much deeper insights into specific aspects of images. +2. **Specialized Vocabulary**: These perspectives employ domain-specific terminology and frameworks appropriate to their focus area. +3. **Structured Metadata**: The outputs are optimized for classification, tagging, and metadata generation within specific domains. +4. **Complementary Insights**: When combined with broader perspectives, they enrich the overall understanding with specialized knowledge. + +## Dataset Applications + +These niche perspectives are particularly valuable for dataset creation in several contexts: + +### Enhanced Metadata + +The structured outputs from niche perspectives can significantly enhance image datasets with rich, specialized metadata that supports advanced search, filtering, and analysis. + +### Domain-Specific Training Data + +For machine learning models focused on specific tasks (emotion recognition, temporal classification), these perspectives generate high-quality labeled data optimized for the target domain. + +### Multi-dimensional Analysis + +By applying multiple niche perspectives alongside broader ones, dataset creators can build multi-layered understanding of visual content that captures both general characteristics and specialized aspects. + +## Integration Guidelines + +When using niche perspectives for dataset creation: + +1. Consider what specific dimensions of analysis are most relevant to your use case +2. Combine multiple niche perspectives for multi-faceted specialized analysis +3. Pair niche perspectives with broader ones for comprehensive coverage +4. Use the structured outputs to enhance search, filtering, and retrieval capabilities diff --git a/workspace/config/perspectives/custom_caption.json b/workspace/perspective_library/niche/custom_caption.json similarity index 72% rename from workspace/config/perspectives/custom_caption.json rename to workspace/perspective_library/niche/custom_caption.json index 14eeaf89..520a3a37 100644 --- a/workspace/config/perspectives/custom_caption.json +++ b/workspace/perspective_library/niche/custom_caption.json @@ -27,5 +27,11 @@ "style": "green" } ], - "context_template": "\n{caption}\n\n" + "context_template": "\n{caption}\n\n", + "module": "niche", + "type": "custom", + "tags": ["customizable", "general-purpose", "flexible"], + "description": "A customizable perspective that allows users to provide their own instructions for image analysis, supporting versatile use cases and custom analysis approaches.", + "deprecated": false, + "priority": 10 } \ No newline at end of file diff --git a/workspace/config/perspectives/emotional_sentiment.json b/workspace/perspective_library/niche/emotional_sentiment.json similarity index 76% rename from workspace/config/perspectives/emotional_sentiment.json rename to workspace/perspective_library/niche/emotional_sentiment.json index bdaea7ed..151509e4 100644 --- a/workspace/config/perspectives/emotional_sentiment.json +++ b/workspace/perspective_library/niche/emotional_sentiment.json @@ -27,5 +27,11 @@ "style": "green" } ], - "context_template": "\n{sentiment_caption}\n\n" + "context_template": "\n{sentiment_caption}\n\n", + "module": "niche", + "type": "perspective", + "tags": ["emotion", "sentiment", "mood", "affective"], + "description": "A perspective focused on analyzing the emotional tone conveyed by an image, providing insights into mood and sentiment through descriptive language.", + "deprecated": false, + "priority": 25 } \ No newline at end of file diff --git a/workspace/config/perspectives/time_tags.json b/workspace/perspective_library/niche/time_tags.json similarity index 85% rename from workspace/config/perspectives/time_tags.json rename to workspace/perspective_library/niche/time_tags.json index af55b3f7..d24bca34 100644 --- a/workspace/config/perspectives/time_tags.json +++ b/workspace/perspective_library/niche/time_tags.json @@ -57,5 +57,11 @@ } ], - "context_template": "\nWhy Important:\n{why_important}\n\nTags:\n{time_tags}\n\n" + "context_template": "\nPeriod: {time_period_tags}\nCategory: {temporal_category}\nQualities: {time_quality_tags}\n\n", + "module": "niche", + "type": "perspective", + "tags": ["time", "tagging", "classification", "historical", "periods"], + "description": "A tagging-focused perspective that categorizes images based on time periods, temporal indicators, and time-related qualities for easier image classification.", + "deprecated": false, + "priority": 50 } \ No newline at end of file diff --git a/workspace/perspective_library/video/README.md b/workspace/perspective_library/video/README.md new file mode 100644 index 00000000..dba3600e --- /dev/null +++ b/workspace/perspective_library/video/README.md @@ -0,0 +1,62 @@ +# Video Perspectives Module + +## Overview + +The Video Perspectives module contains specialized perspectives designed to transform static images into rich, dynamic prompts for video generation. Unlike traditional image captioning, these perspectives focus on identifying and enhancing motion potential, temporal progression, and cinematic elements that can effectively guide the generation of video content from still images. + +These perspectives are particularly valuable for: + +- Creating high-quality prompts for text-to-video and image-to-video generation models +- Developing storyboards and narrative sequences from single images +- Extracting dynamic elements from static visuals +- Enhancing creative workflows for filmmakers, animators, and content creators +- Building datasets that bridge still imagery with video content + +## Video Perspectives + +| Perspective | Description | Primary Focus | Use Cases | +|-------------|-------------|--------------|-----------| +| **Image to Video Prompt** | Generates dense, visually rich captions optimized for video generation with emphasis on action, depth, and cinematography. | Comprehensive video prompt generation | Video generation from images, motion prompt datasets | +| **Image-to-Storyboard (2-Beat)** | Creates a two-beat narrative progression based on a static image, with clear transitions between moments. | Two-stage narrative progression | Simple storyboarding, before/after sequences, transition datasets | +| **Image-to-Storyboard (3-Beat)** | Builds a three-beat narrative sequence with beginning, middle, and end progression. | Three-stage narrative progression | Complete narrative arcs, sequential storytelling datasets | +| **Image to Video with Custom Instructions** | Integrates user instructions with image analysis to create tailored video generation prompts. | Customized video prompt generation | Guided video creation, specialized motion datasets | + +## The Value of Dynamic Interpretation + +While still images capture a single moment in time, effective video generation requires understanding: + +1. **Implied Motion**: What movements are suggested by the static elements? +2. **Narrative Progression**: How might the scene develop over time? +3. **Cinematic Language**: What camera movements and transitions would enhance the visual storytelling? +4. **Temporal Context**: What came before this moment, and what might follow? + +The perspectives in this module systematically extract and enhance these dynamic elements, creating structured outputs that serve as bridges between static and temporal media. + +## Dataset Applications + +These video perspectives are particularly valuable for dataset creation in several contexts: + +### Video Generation Training + +These perspectives create ideal training pairs that link static images with rich prompts specifically optimized for video generation models. This bridges the gap between image and video domains. + +### Storyboarding Datasets + +For applications focused on sequential visual storytelling, these perspectives generate structured narrative progressions with clear beats and transitions that can be used to train storyboarding or sequential image generation models. + +### Cinematic Education + +In educational contexts, these perspectives help illustrate how static compositions imply motion and how to effectively translate visual elements into cinematic language. + +### Motion Prompt Engineering + +By systematically breaking down the elements that contribute to effective motion description, these perspectives help build datasets for researching and improving prompt engineering for video generation. + +## Integration Guidelines + +When using video perspectives for dataset creation: + +1. Pair with traditional image captions to create multi-modal training examples +2. Consider using different beat-count perspectives for varying complexity levels +3. Use the structured fields to train specialized video generation aspects (e.g., just camera movements) +4. For instruction-tuned video models, the custom instruction perspective provides valuable paired examples diff --git a/workspace/perspective_library/video/i2storyboard_2beat.json b/workspace/perspective_library/video/i2storyboard_2beat.json new file mode 100644 index 00000000..90c2649c --- /dev/null +++ b/workspace/perspective_library/video/i2storyboard_2beat.json @@ -0,0 +1,68 @@ +{ + "name": "i2storyboard_2beat", + "display_name": "Image-to-Storyboard (2-Beat)", + "version": "1", + "prompt": "Analyze the image and generate a **two-beat storyboard prompt**, capturing a compelling narrative progression. Each beat should:\n\n1. **Identify a distinct moment** within the image that can transition into another.\n2. **Describe the key subject(s) and their action or state** in a visually engaging way.\n3. **Establish the setting, mood, and atmosphere** to enhance storytelling.\n4. **Define a clear transition between the two beats**, ensuring narrative flow.\n5. **Use cinematic framing** to guide visual composition.\n\nEach beat should be short and impactful, maintaining strong **visual clarity and storytelling flow**.", + "schema_fields": [ + { + "name": "beat_1", + "type": "str", + "description": "The first major moment in the storyboard—describes the initial state, action, or tension.", + "is_list": false + }, + { + "name": "beat_2", + "type": "str", + "description": "The second major moment—describes the transformation, reaction, or escalation from Beat 1.", + "is_list": false + }, + { + "name": "key_subjects", + "type": "str", + "description": "A concise description of the primary characters or objects driving the scene.", + "is_list": false + }, + { + "name": "setting_and_mood", + "type": "str", + "description": "A description of the environment and its emotional tone to reinforce storytelling.", + "is_list": false + }, + { + "name": "transition", + "type": "str", + "description": "A short description of how Beat 1 transitions into Beat 2 visually or thematically.", + "is_list": false + }, + { + "name": "cinematic_framing", + "type": "str", + "description": "Suggested camera angles, movement, or framing techniques to enhance visual storytelling.", + "is_list": false + }, + { + "name": "final_storyboard_prompt", + "type": "str", + "description": "A refined, structured prompt that integrates both beats and transitions smoothly.", + "is_list": false + } + ], + "table_columns": [ + { + "name": "Component", + "style": "cyan" + }, + { + "name": "Description", + "style": "green" + } + ], + "context_template": "\n{final_storyboard_prompt}\n", + "module": "creative", + "type": "perspective", + "tags": ["storyboard", "video", "narrative"], + "description": "A perspective that generates a two-beat storyboard prompt for video generation.", + "deprecated": false, + "priority": 50 + } + \ No newline at end of file diff --git a/workspace/perspective_library/video/i2storyboard_3beat.json b/workspace/perspective_library/video/i2storyboard_3beat.json new file mode 100644 index 00000000..8ed795a3 --- /dev/null +++ b/workspace/perspective_library/video/i2storyboard_3beat.json @@ -0,0 +1,73 @@ +{ + "name": "i2storyboard_3beat", + "display_name": "Image-to-Storyboard (3-Beat)", + "version": "1", + "prompt": "Analyze the image and generate a **three-beat storyboard prompt**, capturing a compelling narrative sequence. Each beat should:\n\n1. **Define a distinct moment** that progresses naturally into the next.\n2. **Describe the key subject(s), action, and state** for visual clarity.\n3. **Establish setting, mood, and atmosphere** to enhance storytelling.\n4. **Ensure smooth narrative and visual transitions** between beats.\n5. **Use cinematic framing** to guide visual composition.\n\nEach beat must be short yet impactful, maintaining strong **visual clarity, narrative flow, and cinematic appeal**.", + "schema_fields": [ + { + "name": "beat_1", + "type": "str", + "description": "The first key moment—sets the stage, introduces the subject, and establishes context.", + "is_list": false + }, + { + "name": "beat_2", + "type": "str", + "description": "The second key moment—presents a shift, action, or escalation in the scene.", + "is_list": false + }, + { + "name": "beat_3", + "type": "str", + "description": "The final key moment—concludes the sequence, resolving tension or setting up further events.", + "is_list": false + }, + { + "name": "key_subjects", + "type": "str", + "description": "A concise description of the primary characters or objects driving the scene.", + "is_list": false + }, + { + "name": "setting_and_mood", + "type": "str", + "description": "A description of the environment and its emotional tone to reinforce storytelling.", + "is_list": false + }, + { + "name": "transition", + "type": "str", + "description": "A short description of how each beat flows into the next.", + "is_list": false + }, + { + "name": "cinematic_framing", + "type": "str", + "description": "Suggested camera angles, movement, or framing techniques to enhance visual storytelling.", + "is_list": false + }, + { + "name": "final_storyboard_prompt", + "type": "str", + "description": "A refined, structured prompt that integrates all beats and transitions smoothly.", + "is_list": false + } + ], + "table_columns": [ + { + "name": "Component", + "style": "cyan" + }, + { + "name": "Description", + "style": "green" + } + ], + "context_template": "\n{final_storyboard_prompt}\n", + "module": "creative", + "type": "perspective", + "tags": ["storyboard", "video", "narrative"], + "description": "A perspective that generates a three-beat storyboard prompt for video generation.", + "deprecated": false, + "priority": 50 +} diff --git a/workspace/perspective_library/video/i2video.json b/workspace/perspective_library/video/i2video.json new file mode 100644 index 00000000..68e2c446 --- /dev/null +++ b/workspace/perspective_library/video/i2video.json @@ -0,0 +1,73 @@ +{ + "name": "i2video", + "display_name": "Image to Video Prompt", + "version": "1", + "prompt": "Craft a high-quality, dense video caption optimized for video generation. The caption should be vivid, concise (one to two sentences), and include:\n\n1. **Core Scene**: Identify the main subject and setting. Ensure the scene has dynamic potential (e.g., action, transformation, movement).\n\n2. **Subject Details**: Describe key visual traits of the subject(s), including clothing, physical features, and expressions that enhance visual interest.\n\n3. **Environmental Layers**: Capture foreground, midground, and background details to ensure depth and spatial clarity.\n\n4. **Action Sequence**: Define the subject's movement or events in a way that emphasizes realistic physics, momentum, or cinematic flow.\n\n5. **Mood & Atmosphere**: Establish the tone using descriptive words that evoke emotions (e.g., 'ominous storm brewing,' 'serene sunset glow').\n\n6. **Artistic Style & Cinematography**: Specify the visual aesthetic (e.g., 'photorealistic cyberpunk city with neon reflections') and include relevant camera directions (e.g., 'dynamic tracking shot from below').\n\nThe final caption must be **short yet densely packed with visual and cinematic cues**, avoiding generic or vague descriptions. Ensure natural phrasing and an engaging flow.", + "schema_fields": [ + { + "name": "core_scene", + "type": "str", + "description": "A brief but clear definition of the main subject and setting, ensuring visual dynamism.", + "is_list": false + }, + { + "name": "subject_details", + "type": "str", + "description": "Distinctive features of the subject, including appearance, attire, posture, or expressions.", + "is_list": false + }, + { + "name": "environmental_layers", + "type": "str", + "description": "Layered description of foreground, midground, and background to establish depth.", + "is_list": false + }, + { + "name": "action_sequence", + "type": "str", + "description": "Specific, dynamic movements or events occurring in the scene.", + "is_list": false + }, + { + "name": "mood_and_atmosphere", + "type": "str", + "description": "Descriptive elements that establish the emotional tone and ambiance.", + "is_list": false + }, + { + "name": "artistic_style", + "type": "str", + "description": "The visual style and cinematographic choices, such as lighting, color schemes, and perspective.", + "is_list": false + }, + { + "name": "camera_directions", + "type": "str", + "description": "Optional cinematic techniques like tracking shots, zooms, or slow-motion effects.", + "is_list": false + }, + { + "name": "final_caption", + "type": "str", + "description": "A refined, short, yet highly descriptive video caption integrating all elements into a seamless flow.", + "is_list": false + } + ], + "table_columns": [ + { + "name": "Component", + "style": "cyan" + }, + { + "name": "Description", + "style": "green" + } + ], + "context_template": "\n{final_caption}\n", + "module": "creative", + "type": "perspective", + "tags": ["video", "caption", "video_generation"], + "description": "A perspective that generates a high-quality, dense video caption optimized for video generation.", + "deprecated": false, + "priority": 50 +} \ No newline at end of file diff --git a/workspace/perspective_library/video/instruct_i2video.json b/workspace/perspective_library/video/instruct_i2video.json new file mode 100644 index 00000000..d0ceaed1 --- /dev/null +++ b/workspace/perspective_library/video/instruct_i2video.json @@ -0,0 +1,85 @@ +{ + "name": "instruct_i2video", + "display_name": "Image to Video with Custom Instructions", + "version": "1", + "prompt": "Analyze the image and extract key visual elements. Then, integrate the provided custom instructions into the video generation prompt to enhance relevance and specificity. The final caption should be concise (one to two sentences) yet highly descriptive, incorporating:\n\n1. **Core Scene**: Identify the main subject and setting with dynamic potential (e.g., action, transformation, movement).\n\n2. **Subject Details**: Describe key visual traits of the subject(s), including clothing, physical features, and expressions.\n\n3. **Environmental Layers**: Capture foreground, midground, and background details to ensure depth and spatial clarity.\n\n4. **Action Sequence**: Define the subject's movement or events in a way that emphasizes realistic physics, momentum, or cinematic flow.\n\n5. **Mood & Atmosphere**: Establish the tone using descriptive words that evoke emotions (e.g., 'ominous storm brewing,' 'serene sunset glow').\n\n6. **Artistic Style & Cinematography**: Specify the visual aesthetic (e.g., 'photorealistic cyberpunk city with neon reflections') and include relevant camera directions (e.g., 'dynamic tracking shot from below').\n\nThe final caption must seamlessly **merge insights from the custom instructions** with the image analysis, ensuring a **natural, engaging, and highly optimized** prompt for video generation.", + "schema_fields": [ + { + "name": "image_analysis", + "type": "str", + "description": "A breakdown of key visual elements in the image, including main subjects, setting, and composition.", + "is_list": false + }, + { + "name": "instruction_integration", + "type": "str", + "description": "A structured plan for incorporating the custom instructions into the video prompt while preserving coherence.", + "is_list": false + }, + { + "name": "core_scene", + "type": "str", + "description": "A brief but clear definition of the main subject and setting, ensuring visual dynamism.", + "is_list": false + }, + { + "name": "subject_details", + "type": "str", + "description": "Distinctive features of the subject, including appearance, attire, posture, or expressions.", + "is_list": false + }, + { + "name": "environmental_layers", + "type": "str", + "description": "Layered description of foreground, midground, and background to establish depth.", + "is_list": false + }, + { + "name": "action_sequence", + "type": "str", + "description": "Specific, dynamic movements or events occurring in the scene.", + "is_list": false + }, + { + "name": "mood_and_atmosphere", + "type": "str", + "description": "Descriptive elements that establish the emotional tone and ambiance.", + "is_list": false + }, + { + "name": "artistic_style", + "type": "str", + "description": "The visual style and cinematographic choices, such as lighting, color schemes, and perspective.", + "is_list": false + }, + { + "name": "camera_directions", + "type": "str", + "description": "Optional cinematic techniques like tracking shots, zooms, or slow-motion effects.", + "is_list": false + }, + { + "name": "final_caption", + "type": "str", + "description": "A refined, short, yet highly descriptive video caption integrating both the image analysis and custom instructions.", + "is_list": false + } + ], + "table_columns": [ + { + "name": "Component", + "style": "cyan" + }, + { + "name": "Description", + "style": "green" + } + ], + "context_template": "\n{final_caption}\n", + "module": "creative", + "type": "perspective", + "tags": ["video", "caption", "video_generation", "custom_integration"], + "description": "A perspective that generates a high-quality, dense video caption by analyzing an image and incorporating custom user instructions.", + "deprecated": false, + "priority": 50 +}