Skip to content

Studio Backend

Generative AI Tools, Video Workbench, AI pipelines etc.

Directory Structure

The application follows a modular architecture where each feature is organized into its own directory with consistent file naming:

studio-backend/
├── src/
│   ├── [feature]/               # Each feature follows this pattern:
│   │   ├── [feature].router.ts    # API route definitions
│   │   ├── [feature].controller.ts # HTTP request/response handling
│   │   ├── [feature].service.ts    # Business logic
│   │   ├── [feature].validator.ts  # Input validation (Zod schemas)
│   │   └── [feature].middleware.ts # Authentication/authorization
│   ├── models/                  # MongoDB schema definitions
│   ├── shared/                  # Common utilities, types, constants
│   ├── thirdPartyServices/      # External API integrations
│   ├── utilityServices/         # Standalone services for common operations
│   ├── [product]Templates/        # Email or Prompt templates etc.
│   └── [other-features]/        # Additional feature modules
├── tests/                       # Test files mirroring src structure
├── dist/                        # Compiled JavaScript output
└── config files                 # TypeScript, ESLint, Jest, etc.

Core Files: index.ts (app entry), router.ts (main router), mongoDBClient.ts, redisClient.ts, mailer.ts, errorLogger.ts

Architecture Overview

Router, Controllers, Validators, and Service Functions

The application follows a layered architecture pattern:

  1. Routers (*.router.ts): Define API endpoints and route requests to controllers
  2. Controllers (*.controller.ts): Handle HTTP requests/responses and input validation
  3. Validators (*.validator.ts): Validate request data using Zod schemas
  4. Services (*.service.ts): Contain business logic and database operations
  5. Middleware (*.middleware.ts): Handle cross-cutting concerns like authentication and authorization

MongoDB Integration

The src/models/ directory contains MongoDB schema definitions for all data entities. The application uses multiple mongoDB databases:

  • Primary Database: Studio database for main application data
  • Story Database: Rendarigo database for stories data (to be updated when storydesk is up)
  • Connection Management: mongoDBClient.ts handles connections to both databases
  • Collection Access: Provides methods for accessing collections in both databases

Webhooks

The webhook system (src/webhooks/) handles external service callbacks:

  • Webhook Router: Defines webhook endpoints
  • Webhook Middleware: Validates webhook signatures and authenticity
  • Webhook Controller: Processes incoming webhook payloads
  • Webhook Service: Contains webhook processing business logic

Test Library

The application uses Jest as the testing framework:

  • Configuration: jest.config.js with TypeScript support
  • Test Structure: Tests organized by module in tests/ directory
  • Mockers: Reusable mock objects in tests/mockers/
  • Test Helpers: Common test utilities in tests/testHelpers.ts
  • Test Constants: Shared test constants in tests/testConstants.ts

Lefthook Checks

Lefthook is configured for pre-commit hooks (lefthook.yml):

pre-commit:
  commands:
    lint: npm run lint
    test: npm run test
    build: npm run build

This ensures code quality by running:

  • Linting: ESLint checks for code style and potential issues
  • Tests: Jest test suite execution
  • Build: TypeScript compilation verification

Error Logging to New Relic

The application integrates with New Relic for error monitoring:

  • Configuration: newrelic.js with license key and settings
  • Error Logger: src/errorLogger.ts with Winston integration
  • Log Forwarding: Errors automatically forwarded to New Relic
  • File Logging: Local error logs stored in .errorLogs/ directory
  • Queue Processing: Asynchronous error log processing

Redis Client for RBAC & Permissions

Redis is used for caching and session management:

  • Redis Client: src/redisClient.ts with singleton pattern
  • Connection Management: Automatic reconnection and error handling
  • RBAC Integration: Caches user permissions and roles
  • Session Storage: Stores user sessions and authentication tokens

Websocket Integration

The application uses Socket.IO for real-time communication with the frontend:

  • Socket Service: src/socketService.ts - Singleton pattern implementation
  • Event Definitions: src/shared/socketEvents.ts - Centralized event constants
  • Authentication: JWT token verification for socket connections
  • CORS Support: Configured for frontend communication
  • User-based Messaging: sendToUser(userId, event, ...) function sends messages to all socket connections for a specific user (handles multiple tabs/devices)

Available Events:

  • assetGenResponse: Response with generation status and results
  • Connection events: connect, disconnect, error

Usage Example:

// Frontend connection
const socket = io('http://localhost:8081', {
  auth: { token: 'your-jwt-token' },
  transports: ['websocket', 'polling'],
});

// Listen for responses
socket.on('assetGenResponse', (response) => {
  console.log('Asset generation result:', response);
});

// Backend usage - send to all user connections
socketService.sendToUser(userId, 'assetGenResponse', 200, 'Job completed', true, { job });

JaduSpine (Centrifugo) Integration

JaduSpine is a pub/sub messaging system built on Centrifugo for real-time backend-to-frontend communication:

  • SDK: @scenarix/jaduspine-sdk (private package)
  • Backend Singleton: JaduSpineBackend - initialized once at server startup
  • Publishing: Backend publishes messages to user-specific channels
  • Frontend: Connects via WebSocket to Centrifugo, receives messages in real-time

Initialization (in src/index.ts):

import { JaduSpineBackend } from '@scenarix/jaduspine-sdk';

// During server startup
await JaduSpineBackend.init({
  apiUrl: process.env.CENTRIFUGO_API_URL!,
  apiKey: process.env.CENTRIFUGO_API_KEY!,
  debug: process.env.NODE_ENV === 'development',
});

Publishing Messages:

import { JaduSpineBackend, FrontendChannel, FrontendChannels } from '@scenarix/jaduspine-sdk';

// Publish to a specific user
await JaduSpineBackend.publish(new FrontendChannel(FrontendChannels.MINIMATICS, userId), { topic: 'guruResponse', data: { message: 'Hello!' } });

Environment Variables:

  • CENTRIFUGO_API_URL: Centrifugo HTTP API URL
  • CENTRIFUGO_API_KEY: API key for authentication
  • GITHUB_PKG_TOKEN: GitHub Personal Access Token for installing private @scenarix/* packages (required for npm install, see .npmrc)

See src/agenticGuru/io/ for the guru chat implementation using JaduSpine.

Deploy Build Commands

# Development
npm run dev          # Start development server with nodemon

# Production
npm run build        # Compile TypeScript to JavaScript
npm start           # Start production server with New Relic

# Code Quality
npm run lint        # Run ESLint
npm run format      # Format code with Prettier
npm run test        # Run Jest tests

# Setup
npm run setup       # Install lefthook hooks

Linting & Code Formatting

The application uses ESLint v9 with flat config format and Prettier for code quality:

  • ESLint Config: eslint.config.mjs (flat config format)
  • Prettier: Integrated via eslint-plugin-prettier
  • TypeScript Support: @typescript-eslint/parser and @typescript-eslint/eslint-plugin

Key Rules:

  • ESLint recommended rules
  • TypeScript recommended and stylistic rules
  • Prettier formatting enforced as errors
  • @typescript-eslint/explicit-function-return-type: Required for all functions
  • @typescript-eslint/no-explicit-any: Disabled (use any when needed)

Commands:

npm run lint        # Run ESLint on all TypeScript files
npm run format      # Format code with Prettier

Mailer Integration

The application uses Resend for email delivery:

  • Mailer Service: src/mailer.ts with Handlebars templating
  • Email Templates: Templates in src/emailTemplates/
  • Resend Integration: src/thirdPartyServices/resend.service.ts
  • Supported Emails: OTP emails, bug report notifications

Third-Party Services

The application integrates with multiple external services:

  • AI Services: OpenAI, FalAI, xAI, Replicate
  • Voice Generation: ElevenLabs
  • File Storage: B2 Cloud Storage
  • Email: Resend
  • Monitoring: New Relic
  • Project Management: Jira
  • Video Processing: Vidu
  • Workflow: Prefect

Utility Services

The application provides utility services for common operations:

  • Asset Scoring: assetScoring.service.ts - AI-powered asset evaluation and scoring
  • Prompt Rewrite: promptRewrite.service.ts - Intelligent prompt optimization and rewriting
  • Voice Generation: voiceGen.service.ts - Text-to-speech and voice synthesis utilities

Environment Variables

See example in .env.local

API Endpoints

  • /api/auth/* - Authentication endpoints
  • /api/rbac/* - Role-based access control
  • /api/assetGen/* - Asset generation
  • /api/stories/* - Story management
  • /api/workbench/* - Workbench functionality
  • /api/webhooks/* - Webhook endpoints
  • /api/pipeline/* - Pipeline management (via Prefect)
  • /api/bugReport/* - Bug reporting
  • /api/imageEdit/* - Image editing
  • /api/llmChat/* - LLM chat functionality

Contributing

  1. Follow the existing code structure (router → controller → validator → service)
  2. Add appropriate tests for new functionality if applicable
  3. Ensure all lefthook checks pass before committing
  4. Follow the established error logging patterns
  5. Update this README for significant architectural changes