feat: add ModelsLab as TTS and Video generation provider

[adhikjoshi]

Summary

This PR adds ModelsLab as a new provider for both Text-to-Speech (TTS) and Video generation in Sim Studio.

Changes

New Files

• apps/sim/tools/tts/modelslab.ts — TTS tool routing to /api/tools/tts/unified
• apps/sim/tools/video/modelslab.ts — Video tool routing to /api/tools/video

Modified Files

• apps/sim/tools/tts/index.ts — Export modelsLabTtsTool
• apps/sim/tools/tts/types.ts — Add 'modelslab' to TtsProvider union type
• apps/sim/tools/video/index.ts — Export modelsLabVideoTool
• apps/sim/tools/video/types.ts — Add 'modelslab' to VideoParams.provider + new fields (imageUrl, width, height, num_frames)
• apps/sim/blocks/blocks/tts.ts — Add ModelsLab to provider dropdown with voice, language, and speed sub-blocks
• apps/sim/blocks/blocks/video_generator.ts — Add ModelsLab to provider dropdown with mode, imageUrl, width, height sub-blocks (both V1 and V2 blocks)
• apps/sim/app/api/tools/tts/unified/route.ts — Add synthesizeWithModelsLab() with async polling
• apps/sim/app/api/tools/video/route.ts — Add generateWithModelsLab() with async polling
• apps/sim/tools/registry.ts — Register tts_modelslab and video_modelslab

ModelsLab API

• Base URL: https://modelslab.com/api/v6/
• Auth: { "key": "API_KEY" } in JSON request body
• TTS endpoint: POST /voice/text_to_speech → async polling via POST /voice/fetch
• Video (text2video): POST /video/text2video → async polling via POST /video/fetch/{id}
• Video (img2video): POST /video/img2video → async polling via POST /video/fetch/{id}

TTS Features

• Voice selection (Madison, Joanna, Matthew, Salli, etc.)
• Language support (English, Spanish, French, German, Italian, Portuguese, Hindi, Japanese, Chinese)
• Speed control (0.5–2.0)

Video Features

• Text-to-Video mode
• Image-to-Video mode (with image URL input)
• Configurable width and height (256/512/768/1024px)
• Async job polling with timeout

[vercel]

@adhikjoshi is attempting to deploy a commit to the Sim Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Greptile Summary

Added ModelsLab as a new provider for both TTS and video generation. The integration follows established patterns with async polling for job completion.

• TTS: supports voice selection, language, and speed control via /api/v6/voice/text_to_speech endpoint
• Video: supports text-to-video and image-to-video modes via /api/v6/video/text2video and /api/v6/video/img2video
• Both implementations use async polling with timeouts (30 attempts for TTS, 60 for video)
• API keys correctly use user-only visibility per custom rule 2851870a
• Properly registered in tool registry and exported from index files

Issue found: img2video mode missing required validation - when mode === 'img2video', imageUrl parameter is required but not validated before making the API call, which could result in API errors.

Confidence Score: 4/5

• Safe to merge with one validation fix needed
• Implementation follows existing patterns and adheres to style guides. API keys use correct visibility. One logical issue exists: missing validation for required imageUrl in img2video mode could cause runtime errors. Otherwise, code is well-structured with proper error handling and async polling.
• Pay close attention to apps/sim/app/api/tools/video/route.ts - add validation for img2video mode

Important Files Changed

Filename	Overview
apps/sim/tools/tts/modelslab.ts	new ModelsLab TTS tool with correct API key visibility and proper error handling
apps/sim/tools/video/modelslab.ts	new ModelsLab video tool with proper structure and API key visibility
apps/sim/app/api/tools/tts/unified/route.ts	added ModelsLab TTS synthesis with async polling, proper error handling
apps/sim/app/api/tools/video/route.ts	added ModelsLab video generation but missing required validation for img2video mode
apps/sim/blocks/blocks/tts.ts	added ModelsLab provider option with voice, language, and speed configuration
apps/sim/blocks/blocks/video_generator.ts	added ModelsLab provider to both V1 and V2 blocks with mode, dimensions, and imageUrl config

Sequence Diagram

sequenceDiagram
    participant User
    participant Block as TTS/Video Block
    participant Tool as ModelsLab Tool
    participant API as API Route
    participant ModelsLab as ModelsLab API

    User->>Block: Configure provider=modelslab
    Block->>Tool: Execute with params
    Tool->>API: POST /api/tools/tts/unified or /api/tools/video
    API->>ModelsLab: POST text_to_speech or text2video/img2video
    ModelsLab-->>API: {status: processing, id: xxx}
    
    loop Poll until complete (30-60 attempts)
        API->>ModelsLab: POST /voice/fetch or /video/fetch/{id}
        alt Success
            ModelsLab-->>API: {status: success, output: url}
            API->>ModelsLab: Download audio/video from URL
            ModelsLab-->>API: Binary data
        else Still Processing
            ModelsLab-->>API: {status: processing}
        else Error
            ModelsLab-->>API: {status: error/failed}
        end
    end
    
    API-->>Tool: audioUrl/videoUrl
    Tool-->>Block: Response with file
    Block-->>User: Generated audio/video

Loading

_{Last reviewed commit: 30de635}

[greptile-apps]

_{11 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

missing validation for img2video mode - imageUrl is required when mode === 'img2video' but not validated before API call

Suggested change

	if (isImg2Video && imageUrl) {
	requestBody.init_image = imageUrl
	}
	if (isImg2Video && !imageUrl) {
	throw new Error('imageUrl is required for img2video mode')
	}
	if (isImg2Video && imageUrl) {

[adhikjoshi]

Thanks for the review! I will add the img2video validation fix shortly.

[adhikjoshi]

I created a fix PR for the img2video validation: adhikjoshi#2

The fix adds:

if (isImg2Video && !imageUrl) {
  throw new Error(\"imageUrl is required for img2video mode\")
}

Would you like me to close that PR and you can cherry-pick/merge the commit, or would you prefer another approach?

[adhikjoshi]

Fix Applied

I have pushed the fix for the img2video validation to my fork: adhikjoshi#2

The fix adds this validation before the API call:

if (isImg2Video && !imageUrl) {
  throw new Error('imageUrl is required for img2video mode')
}

You can either:

1. Pull commit 77b846b from adhikjoshi/sim@fix/modelslab-img2video-validation
2. Or let me know if you'd like me to update the PR directly

Sorry for the delay - I had to work around GitHub branch restrictions.