Media

@modularityjs/media is the shared core: MediaInput, ProcessedMedia, ProcessOptions, MediaHandler, and MediaHandlersPool. Three sibling contract packages — media-image, media-video, media-audio — narrow the transform and result types per medium. Drivers register their handler in MediaHandlersPool so MediaService.process(input, transforms) (and MediaStreamProcessor) can dispatch by MIME type.

FFmpeg drivers use a temp-file pattern (ffmpeg streams in/out via on-disk paths); sharp runs entirely in process.

Apps can either inject the medium-specific processor (ImageProcessor, VideoProcessor, AudioProcessor) for static typing, or MediaService for MIME-dispatched handling against MediaHandlersPool when the input type is not known up front.

Image

Contract (@modularityjs/media-image)

abstract class ImageProcessor {
  abstract process(
    input: MediaInput,
    transforms: ImageTransform[],
    options?: ProcessOptions,
  ): Promise<ProcessedImage>;
}

MediaInput (from @modularityjs/media):

interface MediaInput {
  readonly filename: string;
  readonly mimetype: string;
  toBuffer(): Promise<Buffer>;
  toStream(): Readable;
}

ImageTransform

type ImageTransform =
  | {
      type: 'resize';
      width?: number;
      height?: number;
      fit?: 'cover' | 'contain' | 'fill' | 'inside' | 'outside';
    }
  | { type: 'format'; format: 'webp' | 'avif' | 'jpeg' | 'png' }
  | { type: 'quality'; quality: number }
  | { type: 'crop'; left: number; top: number; width: number; height: number };

ProcessedImage

interface ProcessedImage {
  readonly filename: string;
  readonly mimetype: string;
  readonly size: number;
  readonly width: number;
  readonly height: number;
  toBuffer(): Promise<Buffer>;
  toStream(): Readable;
}

Driver: Sharp (@modularityjs/media-image-sharp)

Sharp-based image processing.

import { MediaImageModule } from '@modularityjs/media-image';
import { MediaImageSharpModule } from '@modularityjs/media-image-sharp';

const modules = [MediaImageModule, MediaImageSharpModule];

Video

Contract (@modularityjs/media-video)

abstract class VideoProcessor {
  abstract process(
    input: MediaInput,
    transforms: VideoTransform[],
    options?: ProcessOptions,
  ): Promise<ProcessedVideo>;
}

VideoTransform

type VideoTransform =
  | { type: 'transcode'; codec: 'h264' | 'h265' | 'vp9' | 'av1' }
  | { type: 'resolution'; width: number; height: number }
  | { type: 'bitrate'; bitrate: number }
  | { type: 'fps'; fps: number }
  | { type: 'trim'; start: number; end: number }
  | { type: 'thumbnail'; time: number };

ProcessedVideo

interface ProcessedVideo {
  readonly filename: string;
  readonly mimetype: string;
  readonly size: number;
  readonly width: number;
  readonly height: number;
  readonly duration: number;
  readonly codec: string;
  toBuffer(): Promise<Buffer>;
  toStream(): Readable;
}

Driver: FFmpeg (@modularityjs/media-video-ffmpeg)

FFmpeg-based video processing.

import { MediaVideoModule } from '@modularityjs/media-video';
import { MediaVideoFfmpegModule } from '@modularityjs/media-video-ffmpeg';

const modules = [MediaVideoModule, MediaVideoFfmpegModule];

Audio

Contract (@modularityjs/media-audio)

abstract class AudioProcessor {
  abstract process(
    input: MediaInput,
    transforms: AudioTransform[],
    options?: ProcessOptions,
  ): Promise<ProcessedAudio>;
}

AudioTransform

type AudioTransform =
  | { type: 'transcode'; codec: 'aac' | 'mp3' | 'opus' | 'flac' | 'wav' }
  | { type: 'bitrate'; bitrate: number }
  | { type: 'sampleRate'; sampleRate: number }
  | { type: 'channels'; channels: number }
  | { type: 'trim'; start: number; end: number }
  | { type: 'normalize' };

ProcessedAudio

interface ProcessedAudio {
  readonly filename: string;
  readonly mimetype: string;
  readonly size: number;
  readonly duration: number;
  readonly codec: string;
  readonly sampleRate: number;
  readonly channels: number;
  toBuffer(): Promise<Buffer>;
  toStream(): Readable;
}

Driver: FFmpeg (@modularityjs/media-audio-ffmpeg)

FFmpeg-based audio processing.

import { MediaAudioModule } from '@modularityjs/media-audio';
import { MediaAudioFfmpegModule } from '@modularityjs/media-audio-ffmpeg';

const modules = [MediaAudioModule, MediaAudioFfmpegModule];

Storage Stream Bridge (@modularityjs/media-storage-stream)

MediaStorageStreamModule provides MediaStreamProcessor — a service that reads from storage, processes via the appropriate media handler, and writes the result back to storage. This eliminates manual buffer management for storage-backed media pipelines.

import { MediaStorageStreamModule } from '@modularityjs/media-storage-stream';

const modules = [
  StorageModule,
  StorageLocalModule.forRoot({ basePath: './uploads' }),
  MediaImageModule,
  MediaImageSharpModule,
  MediaVideoModule,
  MediaVideoFfmpegModule,
  MediaStorageStreamModule,
];

Usage

import { Inject, Injectable } from '@modularityjs/di';
import { MediaStreamProcessor } from '@modularityjs/media-storage-stream';
import type { StreamProcessResult } from '@modularityjs/media-storage-stream';

@Injectable()
class ImagePipeline {
  constructor(
    @Inject(MediaStreamProcessor)
    private readonly processor: MediaStreamProcessor,
  ) {}

  async resizeAvatar(
    inputKey: string,
    outputKey: string,
  ): Promise<StreamProcessResult> {
    return this.processor.process(inputKey, outputKey, [
      { type: 'resize', width: 256, height: 256, fit: 'cover' },
      { type: 'format', format: 'webp' },
      { type: 'quality', quality: 80 },
    ]);
  }
}

process(inputKey, outputKey, transforms) returns a StreamProcessResult:

interface StreamProcessResult {
  readonly inputKey: string;
  readonly outputKey: string;
  readonly mimetype: string;
  readonly size: number;
  readonly metadata: Record<string, unknown>;
}

The processor reads the content-type of the input object from storage (falling back to application/octet-stream if the storage object has no content-type) and dispatches via MediaHandlersPool. If no handler supports the content type, it throws StateException.

Usage

Resize and convert an uploaded image

import { Inject, Injectable } from '@modularityjs/di';
import { ImageProcessor } from '@modularityjs/media-image';
import { StorageService } from '@modularityjs/storage';
import type { MediaInput } from '@modularityjs/media-image';

@Injectable()
class AvatarService {
  constructor(
    @Inject(ImageProcessor) private readonly images: ImageProcessor,
    @Inject(StorageService) private readonly storage: StorageService,
  ) {}

  async processAvatar(input: MediaInput): Promise<string> {
    const result = await this.images.process(input, [
      { type: 'resize', width: 256, height: 256, fit: 'cover' },
      { type: 'format', format: 'webp' },
      { type: 'quality', quality: 80 },
    ]);

    const key = `avatars/${result.filename}`;
    await this.storage.write(key, await result.toBuffer(), {
      contentType: result.mimetype,
    });
    return key;
  }
}

Transcode an uploaded video

import { Inject, Injectable } from '@modularityjs/di';
import { VideoProcessor } from '@modularityjs/media-video';
import type { MediaInput } from '@modularityjs/media-video';

@Injectable()
class VideoService {
  constructor(
    @Inject(VideoProcessor) private readonly videos: VideoProcessor,
  ) {}

  async transcode(input: MediaInput): Promise<void> {
    const result = await this.videos.process(input, [
      { type: 'transcode', codec: 'h264' },
      { type: 'resolution', width: 1920, height: 1080 },
      { type: 'bitrate', bitrate: 5_000_000 },
    ]);
    console.log(
      `Transcoded: ${result.codec}, ${result.duration}s, ${result.size} bytes`,
    );
  }
}

Normalize and convert audio

import { Inject, Injectable } from '@modularityjs/di';
import { AudioProcessor } from '@modularityjs/media-audio';
import type { MediaInput } from '@modularityjs/media-audio';

@Injectable()
class PodcastService {
  constructor(@Inject(AudioProcessor) private readonly audio: AudioProcessor) {}

  async prepare(input: MediaInput): Promise<void> {
    const result = await this.audio.process(input, [
      { type: 'normalize' },
      { type: 'transcode', codec: 'mp3' },
      { type: 'bitrate', bitrate: 128_000 },
      { type: 'sampleRate', sampleRate: 44_100 },
      { type: 'channels', channels: 2 },
    ]);
    console.log(
      `Prepared: ${result.codec}, ${result.duration}s, ${result.sampleRate}Hz`,
    );
  }
}

Module wiring

import { HttpModule } from '@modularityjs/http';
import { MediaAudioModule } from '@modularityjs/media-audio';
import { MediaImageModule } from '@modularityjs/media-image';
import { MediaVideoModule } from '@modularityjs/media-video';
import { StorageModule } from '@modularityjs/storage';
import { HttpFastifyModule } from '@modularityjs/http-fastify';
import { MediaAudioFfmpegModule } from '@modularityjs/media-audio-ffmpeg';
import { MediaImageSharpModule } from '@modularityjs/media-image-sharp';
import { MediaVideoFfmpegModule } from '@modularityjs/media-video-ffmpeg';
import { StorageLocalModule } from '@modularityjs/storage-local';

const modules = [
  HttpModule.forRoot({ port: 3000 }),
  HttpFastifyModule,
  StorageModule,
  StorageLocalModule.forRoot({ basePath: './uploads' }),
  MediaImageModule,
  MediaImageSharpModule,
  MediaVideoModule,
  MediaVideoFfmpegModule,
  MediaAudioModule,
  MediaAudioFfmpegModule,
];