# Velour - Video Library Application Architecture Plan ## Technology Stack ### Backend - **Framework:** Ruby on Rails 8.x - **Database:** SQLite3 (with potential migration path to PostgreSQL later) - **Background Jobs:** Solid Queue (Rails 8 default) - **Caching:** Solid Cache (Rails 8 default) - **File Storage:** - Active Storage (thumbnails/sprites/previews only) - Direct filesystem paths for video files - S3 SDK (aws-sdk-s3 gem) for remote video storage - **Video Processing:** FFmpeg via streamio-ffmpeg gem ### Frontend - **Framework:** Hotwire (Turbo + Stimulus) - **Video Player:** Video.js 8.x with custom plugins - **Asset Pipeline:** Importmap-rails or esbuild - **Styling:** TailwindCSS ### Authentication (Phase 2) - **OIDC:** omniauth-openid-connect gem - **Session Management:** Rails sessions with encrypted cookies --- ## Database Schema ### Core Models **Works** (canonical representation) - Represents the conceptual "work" (e.g., "Batman [1989]") - Has many Videos (different versions/qualities) ```ruby - title (string, required) - year (integer) - director (string) - description (text) - rating (decimal) - organized (boolean, default: false) - poster_path (string) - backdrop_path (string) - metadata (jsonb) ``` **Videos** (instances of works) - Physical video files across all sources - Belongs to a Work ```ruby - work_id (references works) - storage_location_id (references storage_locations) - title (string) - file_path (string, required) # relative path or S3 key - file_hash (string, indexed) - file_size (bigint) - duration (float) - width (integer) - height (integer) - resolution_label (string) - video_codec (string) - audio_codec (string) - bit_rate (integer) - frame_rate (float) - format (string) - has_subtitles (boolean) - version_type (string) - source_type (string) # "local", "s3", "jellyfin", "web", "velour" - source_url (string) # full URL for remote sources - imported (boolean, default: false) # copied to writable storage? - metadata (jsonb) ``` **StorageLocations** - All video sources (readable and writable) ```ruby - name (string, required) - path (string) # local path or S3 bucket name - location_type (string) # "local", "s3", "jellyfin", "web", "velour" - writable (boolean, default: false) # can we import videos here? - enabled (boolean, default: true) - scan_subdirectories (boolean, default: true) - priority (integer, default: 0) # for unified view ordering - settings (jsonb) # config per type: # S3: {region, access_key_id, secret_access_key, endpoint} # JellyFin: {api_url, api_key, user_id} # Web: {base_url, username, password, auth_type} # Velour: {api_url, api_key} - last_scanned_at (datetime) ``` **VideoAssets** - Generated assets (stored via Active Storage) ```ruby - video_id (references videos) - asset_type (string) # "thumbnail", "preview", "sprite", "vtt" - metadata (jsonb) # Active Storage attachments ``` **PlaybackSessions** ```ruby - video_id (references videos) - user_id (references users, nullable) - position (float) - duration_watched (float) - last_watched_at (datetime) - completed (boolean) - play_count (integer, default: 0) ``` **ImportJobs** (Phase 3) - Track video imports from remote sources ```ruby - video_id (references videos) # source video - destination_location_id (references storage_locations) - destination_path (string) - status (string) # "pending", "downloading", "completed", "failed" - progress (float) # 0-100% - bytes_transferred (bigint) - error_message (text) - started_at (datetime) - completed_at (datetime) ``` **Users** (Phase 2) ```ruby - email (string, required, unique) - name (string) - role (integer, default: 0) # enum: member, admin - provider (string) - uid (string) ``` ### Database Schema Implementation Notes **SQLite Limitations:** - SQLite does NOT support `jsonb` type - must use `text` with `serialize :metadata, coder: JSON` - SQLite does NOT support `enum` types - use integers with Rails enums - Consider PostgreSQL migration path for production deployments **Rails Migration Best Practices:** ```ruby # db/migrate/20240101000001_create_works.rb class CreateWorks < ActiveRecord::Migration[8.1] def change create_table :works do |t| t.string :title, null: false t.integer :year t.string :director t.text :description t.decimal :rating, precision: 3, scale: 1 t.boolean :organized, default: false, null: false t.string :poster_path t.string :backdrop_path t.text :metadata # SQLite: use text, serialize in model t.timestamps end add_index :works, :title add_index :works, [:title, :year], unique: true add_index :works, :organized end end # db/migrate/20240101000002_create_storage_locations.rb class CreateStorageLocations < ActiveRecord::Migration[8.1] def change create_table :storage_locations do |t| t.string :name, null: false t.string :path t.integer :location_type, null: false, default: 0 # enum t.boolean :writable, default: false, null: false t.boolean :enabled, default: true, null: false t.boolean :scan_subdirectories, default: true, null: false t.integer :priority, default: 0, null: false t.text :settings # SQLite: encrypted text column t.datetime :last_scanned_at t.timestamps end add_index :storage_locations, :name, unique: true add_index :storage_locations, :location_type add_index :storage_locations, :enabled add_index :storage_locations, :priority end end # db/migrate/20240101000003_create_videos.rb class CreateVideos < ActiveRecord::Migration[8.1] def change create_table :videos do |t| t.references :work, null: true, foreign_key: true, index: true t.references :storage_location, null: false, foreign_key: true, index: true t.string :title t.string :file_path, null: false t.string :file_hash, index: true t.bigint :file_size t.float :duration t.integer :width t.integer :height t.string :resolution_label t.string :video_codec t.string :audio_codec t.integer :bit_rate t.float :frame_rate t.string :format t.boolean :has_subtitles, default: false t.string :version_type t.integer :source_type, null: false, default: 0 # enum t.string :source_url t.boolean :imported, default: false, null: false t.boolean :processing_failed, default: false t.text :error_message t.text :metadata t.timestamps end add_index :videos, [:storage_location_id, :file_path], unique: true add_index :videos, :source_type add_index :videos, :file_hash add_index :videos, :imported add_index :videos, [:work_id, :resolution_label] end end # db/migrate/20240101000004_create_video_assets.rb class CreateVideoAssets < ActiveRecord::Migration[8.1] def change create_table :video_assets do |t| t.references :video, null: false, foreign_key: true, index: true t.integer :asset_type, null: false # enum t.text :metadata t.timestamps end add_index :video_assets, [:video_id, :asset_type], unique: true end end # db/migrate/20240101000005_create_playback_sessions.rb class CreatePlaybackSessions < ActiveRecord::Migration[8.1] def change create_table :playback_sessions do |t| t.references :video, null: false, foreign_key: true, index: true t.references :user, null: true, foreign_key: true, index: true t.float :position, default: 0.0 t.float :duration_watched, default: 0.0 t.datetime :last_watched_at t.boolean :completed, default: false, null: false t.integer :play_count, default: 0, null: false t.timestamps end add_index :playback_sessions, [:video_id, :user_id], unique: true add_index :playback_sessions, :last_watched_at end end # db/migrate/20240101000006_create_import_jobs.rb class CreateImportJobs < ActiveRecord::Migration[8.1] def change create_table :import_jobs do |t| t.references :video, null: false, foreign_key: true, index: true t.references :destination_location, null: false, foreign_key: { to_table: :storage_locations } t.string :destination_path t.integer :status, null: false, default: 0 # enum t.float :progress, default: 0.0 t.bigint :bytes_transferred, default: 0 t.text :error_message t.datetime :started_at t.datetime :completed_at t.timestamps end add_index :import_jobs, :status add_index :import_jobs, [:video_id, :status] end end # db/migrate/20240101000007_create_users.rb (Phase 2) class CreateUsers < ActiveRecord::Migration[8.1] def change create_table :users do |t| t.string :email, null: false t.string :name t.integer :role, default: 0, null: false # enum t.string :provider t.string :uid t.timestamps end add_index :users, :email, unique: true add_index :users, [:provider, :uid], unique: true add_index :users, :role end end ``` **Enum Definitions:** ```ruby # app/models/video.rb class Video < ApplicationRecord enum source_type: { local: 0, s3: 1, jellyfin: 2, web: 3, velour: 4 } end # app/models/storage_location.rb class StorageLocation < ApplicationRecord enum location_type: { local: 0, s3: 1, jellyfin: 2, web: 3, velour: 4 } end # app/models/video_asset.rb class VideoAsset < ApplicationRecord enum asset_type: { thumbnail: 0, preview: 1, sprite: 2, vtt: 3 } end # app/models/import_job.rb class ImportJob < ApplicationRecord enum status: { pending: 0, downloading: 1, processing: 2, completed: 3, failed: 4, cancelled: 5 } end # app/models/user.rb (Phase 2) class User < ApplicationRecord enum role: { member: 0, admin: 1 } end ``` --- ## Storage Architecture ### Storage Location Types **1. Local Filesystem** (Readable + Writable) ```ruby settings: {} path: "/path/to/videos" writable: true ``` **2. S3 Compatible Storage** (Readable + Writable) ```ruby settings: { region: "us-east-1", access_key_id: "...", secret_access_key: "...", endpoint: "https://s3.amazonaws.com" # or Wasabi, Backblaze, etc. } path: "bucket-name" writable: true ``` **3. JellyFin Server** (Readable only) ```ruby settings: { api_url: "https://jellyfin.example.com", api_key: "...", user_id: "..." } path: null writable: false ``` **4. Web Directory** (Readable only) ```ruby settings: { base_url: "https://videos.example.com", auth_type: "basic", # or "bearer", "none" username: "...", password: "..." } path: null writable: false ``` **5. Velour Instance** (Readable only - Phase 4) ```ruby settings: { api_url: "https://velour.example.com", api_key: "..." } path: null writable: false ``` ### Unified View Strategy **Library Aggregation:** - Videos table contains entries from ALL sources - `storage_location_id` links each video to its source - Unified `/videos` view queries across all enabled locations - Filter/group by `storage_location` to see source breakdown **Video Streaming Strategy:** - Local: `send_file` with byte-range support - S3: Generate presigned URLs (configurable expiry) - JellyFin: Proxy through Rails or redirect to JellyFin stream - Web: Proxy through Rails with auth forwarding - Velour: Proxy or redirect to federated instance ### Storage Adapter Pattern **Architecture:** Strategy pattern for pluggable storage backends. **Base Adapter Interface:** ```ruby # app/services/storage_adapters/base_adapter.rb module StorageAdapters class BaseAdapter def initialize(storage_location) @storage_location = storage_location end # Scan for video files and return array of relative paths def scan raise NotImplementedError, "#{self.class} must implement #scan" end # Generate streaming URL for a video def stream_url(video) raise NotImplementedError, "#{self.class} must implement #stream_url" end # Check if file exists at path def exists?(file_path) raise NotImplementedError, "#{self.class} must implement #exists?" end # Check if storage can be read from def readable? raise NotImplementedError, "#{self.class} must implement #readable?" end # Check if storage can be written to def writable? @storage_location.writable? end # Write/copy file to storage def write(source_path, dest_path) raise NotImplementedError, "#{self.class} must implement #write" end # Download file to local temp path (for processing) def download_to_temp(video) raise NotImplementedError, "#{self.class} must implement #download_to_temp" end end end ``` **Example: Local Adapter** ```ruby # app/services/storage_adapters/local_adapter.rb module StorageAdapters class LocalAdapter < BaseAdapter VIDEO_EXTENSIONS = %w[.mp4 .mkv .avi .mov .wmv .flv .webm .m4v].freeze def scan return [] unless readable? pattern = if @storage_location.scan_subdirectories File.join(@storage_location.path, "**", "*{#{VIDEO_EXTENSIONS.join(',')}}") else File.join(@storage_location.path, "*{#{VIDEO_EXTENSIONS.join(',')}}") end Dir.glob(pattern, File::FNM_CASEFOLD).map do |full_path| full_path.sub(@storage_location.path + "/", "") end end def stream_url(video) full_path(video) end def exists?(file_path) File.exist?(full_path_from_relative(file_path)) end def readable? File.directory?(@storage_location.path) && File.readable?(@storage_location.path) end def writable? super && File.writable?(@storage_location.path) end def write(source_path, dest_path) dest_full_path = full_path_from_relative(dest_path) FileUtils.mkdir_p(File.dirname(dest_full_path)) FileUtils.cp(source_path, dest_full_path) dest_path end def download_to_temp(video) # Already local, return path full_path(video) end private def full_path(video) full_path_from_relative(video.file_path) end def full_path_from_relative(file_path) File.join(@storage_location.path, file_path) end end end ``` **Example: S3 Adapter** ```ruby # app/services/storage_adapters/s3_adapter.rb require 'aws-sdk-s3' module StorageAdapters class S3Adapter < BaseAdapter VIDEO_EXTENSIONS = %w[.mp4 .mkv .avi .mov .wmv .flv .webm .m4v].freeze def scan return [] unless readable? prefix = @storage_location.scan_subdirectories ? "" : nil s3_client.list_objects_v2( bucket: bucket_name, prefix: prefix ).contents.select do |obj| VIDEO_EXTENSIONS.any? { |ext| obj.key.downcase.end_with?(ext) } end.map(&:key) end def stream_url(video) s3_client.presigned_url( :get_object, bucket: bucket_name, key: video.file_path, expires_in: 3600 # 1 hour ) end def exists?(file_path) s3_client.head_object(bucket: bucket_name, key: file_path) true rescue Aws::S3::Errors::NotFound false end def readable? s3_client.head_bucket(bucket: bucket_name) true rescue Aws::S3::Errors::NotFound, Aws::S3::Errors::Forbidden false end def write(source_path, dest_path) File.open(source_path, 'rb') do |file| s3_client.put_object( bucket: bucket_name, key: dest_path, body: file ) end dest_path end def download_to_temp(video) temp_file = Tempfile.new(['velour-video', File.extname(video.file_path)]) s3_client.get_object( bucket: bucket_name, key: video.file_path, response_target: temp_file.path ) temp_file.path end private def s3_client @s3_client ||= Aws::S3::Client.new( region: settings['region'], access_key_id: settings['access_key_id'], secret_access_key: settings['secret_access_key'], endpoint: settings['endpoint'] ) end def bucket_name @storage_location.path end def settings @storage_location.settings end end end ``` **Model Integration:** ```ruby # app/models/storage_location.rb class StorageLocation < ApplicationRecord encrypts :settings serialize :settings, coder: JSON enum location_type: { local: 0, s3: 1, jellyfin: 2, web: 3, velour: 4 } def adapter @adapter ||= adapter_class.new(self) end private def adapter_class "StorageAdapters::#{location_type.classify}Adapter".constantize end end ``` **File Organization:** ``` app/ └── services/ └── storage_adapters/ ├── base_adapter.rb ├── local_adapter.rb ├── s3_adapter.rb ├── jellyfin_adapter.rb # Phase 3 ├── web_adapter.rb # Phase 3 └── velour_adapter.rb # Phase 4 ``` --- ## Video Processing Pipeline ### Asset Generation Jobs **VideoProcessorJob** - Only runs for videos we can access (local, S3, or importable) - Downloads temp copy if remote - Generates: 1. Thumbnail (1920x1080 JPEG at 10% mark) 2. Preview clip (30s MP4 at 720p) 3. VTT sprite sheet (160x90 tiles, 5s intervals) 4. Metadata extraction (FFprobe) - Stores assets via Active Storage (S3 or local) - Cleans up temp files **VideoImportJob** (Phase 3) - Downloads video from remote source - Shows progress (bytes transferred, %) - Saves to writable storage location - Creates new Video record for imported copy - Links to same Work as source - Triggers VideoProcessorJob on completion ### Service Objects Architecture Service objects encapsulate complex business logic that doesn't belong in models or controllers. They follow the single responsibility principle and make code more testable. **1. FileScannerService** Orchestrates scanning a storage location for video files. ```ruby # app/services/file_scanner_service.rb class FileScannerService def initialize(storage_location) @storage_location = storage_location @adapter = storage_location.adapter end def call return Result.failure("Storage location not readable") unless @adapter.readable? @storage_location.update!(last_scanned_at: Time.current) file_paths = @adapter.scan new_videos = [] file_paths.each do |file_path| video = find_or_create_video(file_path) new_videos << video if video.previously_new_record? # Queue processing for new or unprocessed videos VideoProcessorJob.perform_later(video.id) if video.duration.nil? end Result.success(videos_found: file_paths.size, new_videos: new_videos.size) end private def find_or_create_video(file_path) Video.find_or_create_by!( storage_location: @storage_location, file_path: file_path ) do |video| video.title = extract_title_from_path(file_path) video.source_type = @storage_location.location_type end end def extract_title_from_path(file_path) File.basename(file_path, ".*") .gsub(/[\._]/, " ") .gsub(/\[.*?\]/, "") .strip end end # Usage: # result = FileScannerService.new(@storage_location).call # if result.success? # flash[:notice] = "Found #{result.videos_found} videos (#{result.new_videos} new)" # end ``` **2. VideoMetadataExtractor** Extracts video metadata using FFprobe. ```ruby # app/services/video_metadata_extractor.rb require 'streamio-ffmpeg' class VideoMetadataExtractor def initialize(video) @video = video end def call file_path = @video.storage_location.adapter.download_to_temp(@video) movie = FFMPEG::Movie.new(file_path) @video.update!( duration: movie.duration, width: movie.width, height: movie.height, video_codec: movie.video_codec, audio_codec: movie.audio_codec, bit_rate: movie.bitrate, frame_rate: movie.frame_rate, format: movie.container, file_size: movie.size, resolution_label: calculate_resolution_label(movie.height), file_hash: calculate_file_hash(file_path) ) Result.success rescue FFMPEG::Error => e @video.update!(processing_failed: true, error_message: e.message) Result.failure(e.message) ensure # Clean up temp file if it was downloaded File.delete(file_path) if file_path && File.exist?(file_path) && file_path.include?('tmp') end private def calculate_resolution_label(height) case height when 0..480 then "SD" when 481..720 then "720p" when 721..1080 then "1080p" when 1081..1440 then "1440p" when 1441..2160 then "4K" else "8K+" end end def calculate_file_hash(file_path) # Hash first and last 64KB for speed (like Plex/Emby) Digest::MD5.file(file_path).hexdigest end end ``` **3. DuplicateDetectorService** Finds potential duplicate videos based on file hash and title similarity. ```ruby # app/services/duplicate_detector_service.rb class DuplicateDetectorService def initialize(video = nil) @video = video end def call # Find all videos without a work assigned unorganized_videos = Video.where(work_id: nil) # Group by similar titles and file hashes potential_groups = [] unorganized_videos.group_by(&:file_hash).each do |hash, videos| next if videos.size < 2 potential_groups << { type: :exact_duplicate, videos: videos, confidence: :high } end # Find similar titles using Levenshtein distance or fuzzy matching unorganized_videos.find_each do |video| similar = find_similar_titles(video, unorganized_videos) if similar.any? potential_groups << { type: :similar_title, videos: [video] + similar, confidence: :medium } end end Result.success(groups: potential_groups.uniq) end private def find_similar_titles(video, candidates) return [] unless video.title candidates.select do |candidate| next false if candidate.id == video.id next false unless candidate.title similarity_score(video.title, candidate.title) > 0.8 end end def similarity_score(str1, str2) # Simple implementation - could use gems like 'fuzzy_match' or 'levenshtein' str1_clean = normalize_title(str1) str2_clean = normalize_title(str2) return 1.0 if str1_clean == str2_clean # Jaccard similarity on words words1 = str1_clean.split words2 = str2_clean.split intersection = (words1 & words2).size union = (words1 | words2).size intersection.to_f / union end def normalize_title(title) title.downcase .gsub(/\[.*?\]/, "") # Remove brackets .gsub(/\(.*?\)/, "") # Remove parentheses .gsub(/[^\w\s]/, "") # Remove special chars .strip end end ``` **4. WorkGrouperService** Groups videos into a work. ```ruby # app/services/work_grouper_service.rb class WorkGrouperService def initialize(video_ids, work_attributes = {}) @video_ids = video_ids @work_attributes = work_attributes end def call videos = Video.where(id: @video_ids).includes(:work) return Result.failure("No videos found") if videos.empty? # Use existing work if all videos belong to the same one existing_work = videos.first.work if videos.all? { |v| v.work_id == videos.first.work_id } ActiveRecord::Base.transaction do work = existing_work || create_work_from_videos(videos) videos.each do |video| video.update!(work: work) end Result.success(work: work) end end private def create_work_from_videos(videos) representative = videos.max_by(&:height) || videos.first Work.create!( title: @work_attributes[:title] || extract_base_title(representative.title), year: @work_attributes[:year], organized: false ) end def extract_base_title(title) # Extract base title by removing resolution, format markers, etc. title.gsub(/\d{3,4}p/, "") .gsub(/\b(BluRay|WEB-?DL|HDRip|DVDRip|4K|UHD)\b/i, "") .gsub(/\[.*?\]/, "") .strip end end ``` **5. VideoImporterService** (Phase 3) Handles downloading a video from remote source to writable storage. ```ruby # app/services/video_importer_service.rb class VideoImporterService def initialize(video, destination_location, destination_path = nil) @source_video = video @destination_location = destination_location @destination_path = destination_path || video.file_path @import_job = nil end def call return Result.failure("Destination is not writable") unless @destination_location.writable? return Result.failure("Source is not readable") unless @source_video.storage_location.adapter.readable? # Create import job @import_job = ImportJob.create!( video: @source_video, destination_location: @destination_location, destination_path: @destination_path, status: :pending ) # Queue background job VideoImportJob.perform_later(@import_job.id) Result.success(import_job: @import_job) end end ``` **File Organization:** ``` app/ └── services/ ├── storage_adapters/ # Storage backends ├── file_scanner_service.rb ├── video_metadata_extractor.rb ├── duplicate_detector_service.rb ├── work_grouper_service.rb ├── video_importer_service.rb # Phase 3 └── result.rb # Simple result object ``` **Result Object Pattern:** ```ruby # app/services/result.rb class Result attr_reader :data, :error def initialize(success:, data: {}, error: nil) @success = success @data = data @error = error end def success? @success end def failure? !@success end def self.success(data = {}) new(success: true, data: data) end def self.failure(error) new(success: false, error: error) end # Allow accessing data as methods def method_missing(method, *args) return @data[method] if @data.key?(method) super end def respond_to_missing?(method, include_private = false) @data.key?(method) || super end end ``` --- ## Model Organization Rails models follow a specific organization pattern for readability and maintainability. ### Complete Model Examples **Work Model:** ```ruby # app/models/work.rb class Work < ApplicationRecord # 1. Includes/Concerns include Searchable # 2. Serialization serialize :metadata, coder: JSON # 3. Associations has_many :videos, dependent: :nullify has_one :primary_video, -> { order(height: :desc) }, class_name: "Video" # 4. Validations validates :title, presence: true validates :year, numericality: { only_integer: true, greater_than: 1800 }, allow_nil: true validates :rating, numericality: { greater_than_or_equal_to: 0, less_than_or_equal_to: 10 }, allow_nil: true # 5. Scopes scope :organized, -> { where(organized: true) } scope :unorganized, -> { where(organized: false) } scope :recent, -> { order(created_at: :desc) } scope :by_title, -> { order(:title) } scope :with_year, -> { where.not(year: nil) } # 6. Delegations delegate :resolution_label, :duration, to: :primary_video, prefix: true, allow_nil: true # 7. Class methods def self.search(query) where("title LIKE ? OR director LIKE ?", "%#{query}%", "%#{query}%") end # 8. Instance methods def display_title year ? "#{title} (#{year})" : title end def video_count videos.count end def total_duration videos.sum(:duration) end def available_versions videos.group_by(&:resolution_label) end end ``` **Video Model:** ```ruby # app/models/video.rb class Video < ApplicationRecord # 1. Includes/Concerns include Streamable include Processable # 2. Serialization serialize :metadata, coder: JSON # 3. Enums enum source_type: { local: 0, s3: 1, jellyfin: 2, web: 3, velour: 4 } # 4. Associations belongs_to :work, optional: true, touch: true belongs_to :storage_location has_many :video_assets, dependent: :destroy has_many :playback_sessions, dependent: :destroy has_one :import_job, dependent: :nullify has_one_attached :thumbnail has_one_attached :preview_video # 5. Validations validates :title, presence: true validates :file_path, presence: true validates :file_hash, presence: true validates :source_type, presence: true validate :file_exists_on_storage, on: :create # 6. Callbacks before_save :normalize_title after_create :queue_processing # 7. Scopes scope :unprocessed, -> { where(duration: nil, processing_failed: false) } scope :processed, -> { where.not(duration: nil) } scope :failed, -> { where(processing_failed: true) } scope :by_source, ->(type) { where(source_type: type) } scope :imported, -> { where(imported: true) } scope :recent, -> { order(created_at: :desc) } scope :by_resolution, -> { order(height: :desc) } scope :with_work, -> { where.not(work_id: nil) } scope :without_work, -> { where(work_id: nil) } # 8. Delegations delegate :name, :location_type, :adapter, to: :storage_location, prefix: true delegate :display_title, to: :work, prefix: true, allow_nil: true # 9. Class methods def self.search(query) left_joins(:work) .where("videos.title LIKE ? OR works.title LIKE ?", "%#{query}%", "%#{query}%") .distinct end def self.by_duration(min: nil, max: nil) scope = all scope = scope.where("duration >= ?", min) if min scope = scope.where("duration <= ?", max) if max scope end # 10. Instance methods def display_title work_display_title || title || filename end def filename File.basename(file_path) end def file_extension File.extname(file_path).downcase end def formatted_duration return "Unknown" unless duration hours = (duration / 3600).to_i minutes = ((duration % 3600) / 60).to_i seconds = (duration % 60).to_i if hours > 0 "%d:%02d:%02d" % [hours, minutes, seconds] else "%d:%02d" % [minutes, seconds] end end def formatted_file_size return "Unknown" unless file_size units = ['B', 'KB', 'MB', 'GB', 'TB'] size = file_size.to_f unit_index = 0 while size >= 1024 && unit_index < units.length - 1 size /= 1024.0 unit_index += 1 end "%.2f %s" % [size, units[unit_index]] end def processable? !processing_failed && storage_location_adapter.readable? end def streamable? duration.present? && storage_location.enabled? && storage_location_adapter.readable? end private def normalize_title self.title = title.strip if title.present? end def queue_processing VideoProcessorJob.perform_later(id) if processable? end def file_exists_on_storage return if storage_location_adapter.exists?(file_path) errors.add(:file_path, "does not exist on storage") end end ``` **StorageLocation Model:** ```ruby # app/models/storage_location.rb class StorageLocation < ApplicationRecord # 1. Encryption & Serialization encrypts :settings serialize :settings, coder: JSON # 2. Enums enum location_type: { local: 0, s3: 1, jellyfin: 2, web: 3, velour: 4 } # 3. Associations has_many :videos, dependent: :restrict_with_error has_many :destination_imports, class_name: "ImportJob", foreign_key: :destination_location_id # 4. Validations validates :name, presence: true, uniqueness: true validates :location_type, presence: true validates :path, presence: true, if: -> { local? || s3? } validate :adapter_is_valid # 5. Scopes scope :enabled, -> { where(enabled: true) } scope :disabled, -> { where(enabled: false) } scope :writable, -> { where(writable: true) } scope :readable, -> { enabled } scope :by_priority, -> { order(priority: :desc) } scope :by_type, ->(type) { where(location_type: type) } # 6. Instance methods def adapter @adapter ||= adapter_class.new(self) end def scan! FileScannerService.new(self).call end def accessible? adapter.readable? rescue StandardError false end def video_count videos.count end def last_scan_ago return "Never" unless last_scanned_at distance_of_time_in_words(last_scanned_at, Time.current) end private def adapter_class "StorageAdapters::#{location_type.classify}Adapter".constantize end def adapter_is_valid adapter.readable? rescue StandardError => e errors.add(:base, "Cannot access storage: #{e.message}") end end ``` **PlaybackSession Model:** ```ruby # app/models/playback_session.rb class PlaybackSession < ApplicationRecord # 1. Associations belongs_to :video belongs_to :user, optional: true # 2. Validations validates :video, presence: true validates :position, numericality: { greater_than_or_equal_to: 0 }, allow_nil: true validates :duration_watched, numericality: { greater_than_or_equal_to: 0 }, allow_nil: true # 3. Callbacks before_save :check_completion # 4. Scopes scope :recent, -> { order(last_watched_at: :desc) } scope :completed, -> { where(completed: true) } scope :in_progress, -> { where(completed: false).where.not(position: 0) } scope :for_user, ->(user) { where(user: user) } # 5. Class methods def self.update_position(video, user, position, duration_watched = 0) session = find_or_initialize_by(video: video, user: user) session.position = position session.duration_watched = (session.duration_watched || 0) + duration_watched session.last_watched_at = Time.current session.play_count += 1 if session.position.zero? session.save! end # 6. Instance methods def progress_percentage return 0 unless video.duration && position ((position / video.duration) * 100).round(1) end def resume_position completed? ? 0 : position end private def check_completion if video.duration && position self.completed = (position / video.duration) > 0.9 end end end ``` ### Model Concerns **Streamable Concern:** ```ruby # app/models/concerns/streamable.rb module Streamable extend ActiveSupport::Concern included do # Add any class-level includes here end def stream_url storage_location.adapter.stream_url(self) end def streamable? duration.present? && storage_location.enabled? && storage_location.adapter.readable? end def stream_type case source_type when "s3" then :presigned when "local" then :direct else :proxy end end end ``` **Processable Concern:** ```ruby # app/models/concerns/processable.rb module Processable extend ActiveSupport::Concern included do scope :processing_pending, -> { where(duration: nil, processing_failed: false) } end def processed? duration.present? end def processing_pending? !processed? && !processing_failed? end def mark_processing_failed!(error_message) update!(processing_failed: true, error_message: error_message) end def retry_processing! update!(processing_failed: false, error_message: nil) VideoProcessorJob.perform_later(id) end end ``` **Searchable Concern:** ```ruby # app/models/concerns/searchable.rb module Searchable extend ActiveSupport::Concern class_methods do def search(query) return all if query.blank? where("title LIKE ?", "%#{sanitize_sql_like(query)}%") end end end ``` --- ## Frontend Architecture ### Main Views (Hotwire) **Library Views:** - `videos/index` - Unified library grid with filters by source - `videos/show` - Video player page - `works/index` - Works library (grouped by work) - `works/show` - Work details with all versions/sources **Admin Views (Phase 2+):** - `storage_locations/index` - Manage all sources - `storage_locations/new` - Add new source (local/S3/JellyFin/etc) - `import_jobs/index` - Monitor import progress - `videos/:id/import` - Import modal/form ### Stimulus Controllers **VideoPlayerController** - Initialize Video.js with source detection - Handle different streaming strategies (local/S3/proxy) - Track playback position - Quality switching for multiple versions **LibraryScanController** - Trigger scans per storage location - Real-time progress via Turbo Streams - Show scan results and new videos found **VideoImportController** - Select destination storage location - Show import progress - Cancel import jobs **WorkMergeController** - Group videos into works - Drag-and-drop UI - Show all versions/sources for a work ### Video.js Custom Plugins 1. **resume-plugin** - Auto-resume from saved position 2. **track-plugin** - Send playback stats to Rails API 3. **quality-selector** - Switch between versions (same work, different sources/resolutions) 4. **thumbnails-plugin** - VTT sprite preview on seek --- ## Authorization & Security ### Phase 1: No Authentication (MVP) For MVP, all features are accessible without authentication. However, the authorization structure is designed to be auth-ready. **Current Pattern (Request Context):** ```ruby # app/models/current.rb class Current < ActiveSupport::CurrentAttributes attribute :user attribute :request_id end # app/controllers/application_controller.rb class ApplicationController < ActionController::Base before_action :set_current_attributes private def set_current_attributes Current.user = current_user if respond_to?(:current_user) Current.request_id = request.uuid end end ``` ### Phase 2: OIDC Authentication **User Model with Authorization:** ```ruby # app/models/user.rb class User < ApplicationRecord enum role: { member: 0, admin: 1 } validates :email, presence: true, uniqueness: true def admin? role == "admin" end def self.admin_from_env admin_email = ENV['ADMIN_EMAIL'] return nil unless admin_email find_by(email: admin_email) end end ``` **OIDC Configuration:** ```ruby # config/initializers/omniauth.rb Rails.application.config.middleware.use OmniAuth::Builder do provider :openid_connect, name: :oidc, issuer: ENV['OIDC_ISSUER'], client_id: ENV['OIDC_CLIENT_ID'], client_secret: ENV['OIDC_CLIENT_SECRET'], scope: [:openid, :email, :profile] end ``` **Sessions Controller:** ```ruby # app/controllers/sessions_controller.rb class SessionsController < ApplicationController skip_before_action :authenticate_user!, only: [:create] def create auth_hash = request.env['omniauth.auth'] user = User.find_or_create_from_omniauth(auth_hash) session[:user_id] = user.id redirect_to root_path, notice: "Signed in successfully" end def destroy session[:user_id] = nil redirect_to root_path, notice: "Signed out successfully" end end ``` ### Model-Level Authorization ```ruby # app/models/storage_location.rb class StorageLocation < ApplicationRecord def editable_by?(user) return true if user.nil? # Phase 1: no auth user.admin? # Phase 2+: only admins end def deletable_by?(user) return true if user.nil? user.admin? && videos.count.zero? end end # app/models/work.rb class Work < ApplicationRecord def editable_by?(user) return true if user.nil? user.present? # Any authenticated user end end ``` **Controller-Level Guards:** ```ruby # app/controllers/admin/base_controller.rb module Admin class BaseController < ApplicationController before_action :require_admin! private def require_admin! return if Current.user&.admin? redirect_to root_path, alert: "Access denied" end end end # app/controllers/admin/storage_locations_controller.rb module Admin class StorageLocationsController < Admin::BaseController before_action :set_storage_location, only: [:edit, :update, :destroy] before_action :authorize_storage_location, only: [:edit, :update, :destroy] def index @storage_locations = StorageLocation.all end def update if @storage_location.update(storage_location_params) redirect_to admin_storage_locations_path, notice: "Updated successfully" else render :edit, status: :unprocessable_entity end end private def authorize_storage_location head :forbidden unless @storage_location.editable_by?(Current.user) end end end ``` ### Credentials Management **Encrypted Settings:** ```ruby # app/models/storage_location.rb class StorageLocation < ApplicationRecord encrypts :settings # Rails 7+ built-in encryption serialize :settings, coder: JSON end # Generate encryption key: # bin/rails db:encryption:init # Add to config/credentials.yml.enc ``` **Rails Credentials:** ```bash # Edit credentials bin/rails credentials:edit # Add: # aws: # access_key_id: <%= ENV["AWS_ACCESS_KEY_ID"] %> # secret_access_key: <%= ENV["AWS_SECRET_ACCESS_KEY"] %> ``` ### API Authentication (Phase 4 - Federation) **API Key Authentication:** ```ruby # app/controllers/api/federation/base_controller.rb module Api module Federation class BaseController < ActionController::API include ActionController::HttpAuthentication::Token::ControllerMethods before_action :authenticate_api_key! private def authenticate_api_key! return if Rails.env.development? return unless ENV['ALLOW_FEDERATION'] == 'true' authenticate_or_request_with_http_token do |token, options| ActiveSupport::SecurityUtils.secure_compare( token, ENV.fetch('VELOUR_API_KEY') ) end end end end end ``` **Rate Limiting:** ```ruby # config/initializers/rack_attack.rb (optional, recommended) class Rack::Attack throttle('api/ip', limit: 300, period: 5.minutes) do |req| req.ip if req.path.start_with?('/api/') end throttle('federation/api_key', limit: 1000, period: 1.hour) do |req| req.env['HTTP_AUTHORIZATION'] if req.path.start_with?('/api/federation/') end end ``` --- ## API Controller Architecture ### Controller Organization ``` app/ └── controllers/ ├── application_controller.rb ├── videos_controller.rb # HTML views ├── works_controller.rb ├── admin/ │ ├── base_controller.rb │ ├── storage_locations_controller.rb │ └── import_jobs_controller.rb └── api/ ├── base_controller.rb └── v1/ ├── videos_controller.rb ├── works_controller.rb ├── playback_sessions_controller.rb └── storage_locations_controller.rb └── federation/ # Phase 4 ├── base_controller.rb ├── videos_controller.rb └── works_controller.rb ``` ### API Base Controllers **Internal API Base:** ```ruby # app/controllers/api/base_controller.rb module Api class BaseController < ActionController::API include ActionController::Cookies rescue_from ActiveRecord::RecordNotFound, with: :not_found rescue_from ActiveRecord::RecordInvalid, with: :unprocessable_entity private def not_found render json: { error: "Not found" }, status: :not_found end def unprocessable_entity(exception) render json: { error: "Validation failed", details: exception.record.errors.full_messages }, status: :unprocessable_entity end end end ``` **Example API Controller:** ```ruby # app/controllers/api/v1/videos_controller.rb module Api module V1 class VideosController < Api::BaseController before_action :set_video, only: [:show, :stream] def index @videos = Video.includes(:work, :storage_location) .order(created_at: :desc) .page(params[:page]) .per(params[:per] || 50) render json: { videos: @videos.as_json(include: [:work, :storage_location]), meta: pagination_meta(@videos) } end def show render json: @video.as_json( include: { work: { only: [:id, :title, :year] }, storage_location: { only: [:id, :name, :location_type] } }, methods: [:stream_url, :formatted_duration] ) end def stream # Route to appropriate streaming strategy case @video.stream_type when :presigned render json: { stream_url: @video.stream_url } when :direct send_file @video.stream_url, type: @video.format, disposition: 'inline', stream: true when :proxy # Implement proxy logic redirect_to @video.stream_url, allow_other_host: true end end private def set_video @video = Video.find(params[:id]) end def pagination_meta(collection) { current_page: collection.current_page, next_page: collection.next_page, prev_page: collection.prev_page, total_pages: collection.total_pages, total_count: collection.total_count } end end end end ``` **Playback Tracking API:** ```ruby # app/controllers/api/v1/playback_sessions_controller.rb module Api module V1 class PlaybackSessionsController < Api::BaseController def update video = Video.find(params[:video_id]) user = Current.user # nil in Phase 1, actual user in Phase 2 session = PlaybackSession.update_position( video, user, params[:position].to_f, params[:duration_watched].to_f ) render json: { success: true, session: session } rescue ActiveRecord::RecordNotFound render json: { error: "Video not found" }, status: :not_found end end end end ``` --- ## Turbo Streams & Real-Time Updates ### Scan Progress Broadcasting **Job with Turbo Streams:** ```ruby # app/jobs/file_scanner_job.rb class FileScannerJob < ApplicationJob queue_as :default def perform(storage_location_id) @storage_location = StorageLocation.find(storage_location_id) broadcast_update(status: "started", progress: 0) result = FileScannerService.new(@storage_location).call if result.success? broadcast_update( status: "completed", progress: 100, videos_found: result.videos_found, new_videos: result.new_videos ) else broadcast_update(status: "failed", error: result.error) end end private def broadcast_update(**data) Turbo::StreamsChannel.broadcast_replace_to( "storage_location_#{@storage_location.id}", target: "scan_status", partial: "admin/storage_locations/scan_status", locals: { storage_location: @storage_location, **data } ) end end ``` **View with Turbo Stream:** ```erb <%# app/views/admin/storage_locations/show.html.erb %> <%= turbo_stream_from "storage_location_#{@storage_location.id}" %>
<%= render "scan_status", storage_location: @storage_location, status: "idle" %>
<%= button_to "Scan Now", scan_admin_storage_location_path(@storage_location), method: :post, data: { turbo_frame: "_top" }, class: "btn btn-primary" %> ``` **Partial:** ```erb <%# app/views/admin/storage_locations/_scan_status.html.erb %>
<% case status %> <% when "started" %>

Scanning... <%= progress %>%

<% when "completed" %>

Scan completed! Found <%= videos_found %> videos (<%= new_videos %> new)

<% when "failed" %>

Scan failed: <%= error %>

<% else %>

Ready to scan

<% end %>
``` ### Import Progress Updates Similar pattern for VideoImportJob with progress broadcasting. --- ## Testing Strategy ### Test Organization ``` test/ ├── models/ │ ├── video_test.rb │ ├── work_test.rb │ ├── storage_location_test.rb │ ├── playback_session_test.rb │ └── concerns/ │ ├── streamable_test.rb │ └── processable_test.rb ├── services/ │ ├── file_scanner_service_test.rb │ ├── video_metadata_extractor_test.rb │ ├── duplicate_detector_service_test.rb │ └── storage_adapters/ │ ├── local_adapter_test.rb │ └── s3_adapter_test.rb ├── jobs/ │ ├── video_processor_job_test.rb │ └── file_scanner_job_test.rb ├── controllers/ │ ├── videos_controller_test.rb │ ├── works_controller_test.rb │ └── api/ │ └── v1/ │ └── videos_controller_test.rb └── system/ ├── video_playback_test.rb ├── library_browsing_test.rb └── work_grouping_test.rb ``` ### Example Tests **Model Test:** ```ruby # test/models/video_test.rb require "test_helper" class VideoTest < ActiveSupport::TestCase test "belongs to storage location" do video = videos(:one) assert_instance_of StorageLocation, video.storage_location end test "validates presence of required fields" do video = Video.new assert_not video.valid? assert_includes video.errors[:title], "can't be blank" assert_includes video.errors[:file_path], "can't be blank" end test "formatted_duration returns correct format" do video = videos(:one) video.duration = 3665 # 1 hour, 1 minute, 5 seconds assert_equal "1:01:05", video.formatted_duration end test "streamable? returns true when video is processed and storage is enabled" do video = videos(:one) video.duration = 100 video.storage_location.update!(enabled: true) assert video.streamable? end end ``` **Service Test:** ```ruby # test/services/file_scanner_service_test.rb require "test_helper" class FileScannerServiceTest < ActiveSupport::TestCase setup do @storage_location = storage_locations(:local_movies) end test "scans directory and creates video records" do # Stub adapter scan method @storage_location.adapter.stub :scan, ["movie1.mp4", "movie2.mkv"] do result = FileScannerService.new(@storage_location).call assert result.success? assert_equal 2, result.videos_found end end test "updates last_scanned_at timestamp" do @storage_location.adapter.stub :scan, [] do FileScannerService.new(@storage_location).call @storage_location.reload assert_not_nil @storage_location.last_scanned_at end end end ``` **System Test:** ```ruby # test/system/video_playback_test.rb require "application_system_test_case" class VideoPlaybackTest < ApplicationSystemTestCase test "playing a video updates playback position" do video = videos(:one) visit video_path(video) assert_selector "video#video-player" # Simulate video playback (would need JS execution) # assert_changes -> { video.playback_sessions.first&.position } end test "resume functionality loads saved position" do video = videos(:one) PlaybackSession.create!(video: video, position: 30.0) visit video_path(video) # Assert player starts at saved position # (implementation depends on Video.js setup) end end ``` --- ## API Design (RESTful) ### Video Playback API (Internal) ``` GET /api/v1/videos/:id/stream # Stream video (route to appropriate source) GET /api/v1/videos/:id/presigned # Get presigned S3 URL GET /api/v1/videos/:id/metadata # Get video metadata POST /api/v1/videos/:id/playback # Update playback position GET /api/v1/videos/:id/assets # Get thumbnails, sprites ``` ### Library Management API ``` GET /api/v1/videos # List all videos (unified view) GET /api/v1/works # List works with grouped videos POST /api/v1/works/:id/merge # Merge videos into work GET /api/v1/storage_locations # List all sources POST /api/v1/storage_locations # Add new source POST /api/v1/storage_locations/:id/scan # Trigger scan GET /api/v1/storage_locations/:id/scan_status ``` ### Import API (Phase 3) ``` POST /api/v1/videos/:id/import # Import video to writable storage GET /api/v1/import_jobs # List import jobs GET /api/v1/import_jobs/:id # Get import status DELETE /api/v1/import_jobs/:id # Cancel import ``` ### Federation API (Phase 4) - Public to other Velour instances ``` GET /api/v1/federation/videos # List available videos GET /api/v1/federation/videos/:id # Get video details GET /api/v1/federation/videos/:id/stream # Stream video (with API key auth) GET /api/v1/federation/works # List works ``` --- ## Required Gems Add these to the Gemfile: ```ruby # Gemfile # Core gems (already present in Rails 8) gem "rails", "~> 8.1.1" gem "sqlite3", ">= 2.1" gem "puma", ">= 5.0" gem "importmap-rails" gem "turbo-rails" gem "stimulus-rails" gem "tailwindcss-rails" gem "solid_cache" gem "solid_queue" gem "solid_cable" gem "image_processing", "~> 1.2" # Video processing gem "streamio-ffmpeg" # FFmpeg wrapper for metadata extraction # Pagination gem "pagy" # Fast, lightweight pagination # AWS SDK for S3 support (Phase 3, but add early) gem "aws-sdk-s3" # Phase 2: Authentication # gem "omniauth-openid-connect" # gem "omniauth-rails_csrf_protection" # Phase 3: Remote sources # gem "httparty" # For JellyFin/Web APIs # gem "down" # For downloading remote files # Development & Test group :development, :test do gem "debug", platforms: %i[mri windows] gem "bundler-audit" gem "brakeman" gem "rubocop-rails-omakase" end group :development do gem "web-console" gem "bullet" # N+1 query detection gem "strong_migrations" # Catch unsafe migrations end group :test do gem "capybara" gem "selenium-webdriver" gem "mocha" # Stubbing/mocking end # Optional but recommended # gem "rack-attack" # Rate limiting (Phase 4) ``` --- ## Route Structure ```ruby # config/routes.rb Rails.application.routes.draw do # Health check get "up" => "rails/health#show", as: :rails_health_check # Root root "videos#index" # Main UI routes resources :videos, only: [:index, :show] do member do get :watch # Player page post :import # Phase 3: Import to writable storage end end resources :works, only: [:index, :show] do member do post :merge # Merge videos into this work end end # Admin routes (Phase 2+) namespace :admin do root "dashboard#index" resources :storage_locations do member do post :scan get :scan_status end end resources :import_jobs, only: [:index, :show, :destroy] do member do post :cancel end end resources :users, only: [:index, :edit, :update] # Phase 2 end # Internal API (for JS/Stimulus) namespace :api do namespace :v1 do resources :videos, only: [:index, :show] do member do get :stream get :presigned # For S3 presigned URLs end end resources :works, only: [:index, :show] resources :playback_sessions, only: [] do collection do post :update # POST /api/v1/playback_sessions/update end end resources :storage_locations, only: [:index] end # Federation API (Phase 4) namespace :federation do resources :videos, only: [:index, :show] do member do get :stream end end resources :works, only: [:index, :show] end end # Authentication routes (Phase 2) # get '/auth/:provider/callback', to: 'sessions#create' # delete '/sign_out', to: 'sessions#destroy', as: :sign_out end ``` --- ## Development Phases ### Phase 1: MVP (Local Filesystem) Phase 1 is broken into 4 sub-phases for manageable milestones: #### Phase 1A: Core Foundation (Week 1-2) **Goal:** Basic models and database setup 1. **Generate models with migrations:** - Works - Videos - StorageLocations - PlaybackSessions (basic structure) 2. **Implement models:** - Add validations, scopes, associations - Add concerns (Streamable, Processable, Searchable) - Serialize metadata fields 3. **Create storage adapter pattern:** - BaseAdapter interface - LocalAdapter implementation - Test adapter with sample directory 4. **Create basic services:** - FileScannerService (scan local directory) - Result object pattern 5. **Simple UI:** - Videos index page (list only, no thumbnails yet) - Basic TailwindCSS styling - Storage locations admin page **Deliverable:** Can scan a local directory and see videos in database #### Phase 1B: Video Playback (Week 3) **Goal:** Working video player with streaming 1. **Video streaming:** - Videos controller with show action - Stream action for serving video files - Byte-range support for seeking 2. **Video.js integration:** - Add Video.js via Importmap - Create VideoPlayerController (Stimulus) - Basic player UI on videos#show page 3. **Playback tracking:** - Implement PlaybackSession.update_position - Create API endpoint for position updates - Basic resume functionality (load last position) 4. **Playback tracking plugin:** - Custom Video.js plugin to track position - Send updates to Rails API every 10 seconds - Save position on pause/stop **Deliverable:** Can watch videos with resume functionality #### Phase 1C: Processing Pipeline (Week 4) **Goal:** Video metadata extraction and asset generation 1. **Video metadata extraction:** - VideoMetadataExtractor service - FFmpeg integration via streamio-ffmpeg - Extract duration, resolution, codecs, file hash 2. **Background processing:** - VideoProcessorJob - Queue processing for new videos - Error handling and retry logic 3. **Thumbnail generation:** - Generate thumbnail at 10% mark - Store via Active Storage - Display thumbnails on index page 4. **Video assets:** - VideoAssets model - Store thumbnails, previews (phase 1C: thumbnails only) - VTT sprites (defer to Phase 1D if time-constrained) 5. **Processing UI:** - Show processing status on video cards - Processing failed indicator - Retry processing action **Deliverable:** Videos automatically processed with thumbnails #### Phase 1D: Works & Grouping (Week 5) **Goal:** Group duplicate videos into works 1. **Works functionality:** - Works model fully implemented - Works index/show pages - Display videos grouped by work 2. **Duplicate detection:** - DuplicateDetectorService - Find videos with same file hash - Find videos with similar titles 3. **Work grouping UI:** - WorkGrouperService - Manual grouping interface - Drag-and-drop or checkbox selection - Create work from selected videos 4. **Works display:** - Works index with thumbnails - Works show with all versions - Version selector (resolution, format) 5. **Polish:** - Search functionality - Filtering by source, resolution - Sorting options - Pagination with Pagy **Deliverable:** Full MVP with work grouping and polished UI ### Phase 2: Authentication & Multi-User 1. User model and OIDC integration 2. Admin role management (ENV: ADMIN_EMAIL) 3. Per-user playback history 4. User management UI 5. Storage location management UI ### Phase 3: Remote Sources & Import 1. **S3 Storage Location:** - S3 scanner (list bucket objects) - Presigned URL streaming - Import to/from S3 2. **JellyFin Integration:** - JellyFin API client - Sync metadata - Proxy streaming 3. **Web Directories:** - HTTP directory parser - Auth support (basic, bearer) 4. **Import System:** - VideoImportJob with progress tracking - Import UI with destination selection - Background download with resume support 5. **Unified Library View:** - Filter by source - Show source badges on videos - Multi-source search ### Phase 4: Federation 1. Public API for other Velour instances 2. API key authentication 3. Velour storage location type 4. Federated video discovery 5. Cross-instance streaming --- ## File Organization ### Local Storage Structure ``` storage/ ├── assets/ # Active Storage (thumbnails, previews, sprites) │ └── [active_storage_blobs] └── tmp/ # Temporary processing files ``` ### Video Files - **Local:** Direct filesystem paths (not copied/moved) - **S3:** Stored in configured bucket - **Remote:** Referenced by URL, optionally imported to local/S3 --- ## Key Implementation Decisions ### Storage Flexibility - Videos are NOT managed by Active Storage - Local videos: store absolute/relative paths - S3 videos: store bucket + key - Remote videos: store source URL + metadata - Active Storage ONLY for generated assets (thumbnails, etc.) ### Import Strategy (Phase 3) 1. User selects video from remote source 2. User selects destination (writable storage location) 3. VideoImportJob starts download 4. Progress tracked via Turbo Streams 5. On completion: - New Video record created (imported: true) - Linked to same Work as source - Assets generated - Original remote video remains linked ### Unified View - Single `/videos` index shows all sources - Filter dropdown: "All Sources", "Local", "S3", "JellyFin", etc. - Source badge on each video card - Search across all sources - Sort by: title, date added, last watched, etc. ### Streaming Strategy by Source ```ruby # Local filesystem send_file video.full_path, type: video.mime_type, disposition: 'inline' # S3 redirect_to s3_client.presigned_url(:get_object, bucket: ..., key: ..., expires_in: 3600) # JellyFin redirect_to "#{jellyfin_url}/Videos/#{video.source_id}/stream?api_key=..." # Web directory # Proxy through Rails with auth headers # Velour federation redirect_to "#{velour_url}/api/v1/federation/videos/#{video.source_id}/stream?api_key=..." ``` ### Performance Considerations - Lazy asset generation (only when video viewed) - S3 presigned URLs (no proxy for large files) - Caching of metadata and thumbnails - Cursor-based pagination for large libraries - Background scanning with incremental updates --- ## Configuration (ENV) ```bash # Admin ADMIN_EMAIL=admin@example.com # OIDC (Phase 2) OIDC_ISSUER=https://auth.example.com OIDC_CLIENT_ID=velour OIDC_CLIENT_SECRET=secret # Default Storage DEFAULT_SCAN_PATH=/path/to/videos # S3 (optional default) AWS_REGION=us-east-1 AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... AWS_S3_BUCKET=velour-videos # Processing FFMPEG_THREADS=4 THUMBNAIL_SIZE=1920x1080 PREVIEW_DURATION=30 SPRITE_INTERVAL=5 # Federation (Phase 4) VELOUR_API_KEY=secret-key-for-federation ALLOW_FEDERATION=true ``` --- ## Video.js Implementation (Inspiration from Stash) Based on analysis of the Stash application, we'll use: - **Video.js v8.x** as the core player - **Custom plugins** for: - Resume functionality - Playback tracking - Quality/version selector - VTT thumbnails on seek bar - **Streaming format support:** - Direct MP4/MKV streaming with byte-range - DASH/HLS for adaptive streaming (future) - Multi-quality source selection ### Key Features from Stash Worth Implementing: 1. Scene markers on timeline (our "chapters" equivalent) 2. Thumbnail sprite preview on hover 3. Keyboard shortcuts 4. Mobile-optimized controls 5. Resume from last position 6. Play duration and count tracking 7. AirPlay/Chromecast support (future) --- ## Error Handling & Job Configuration ### Job Retry Strategy ```ruby # app/jobs/video_processor_job.rb class VideoProcessorJob < ApplicationJob queue_as :default # Retry with exponential backoff retry_on StandardError, wait: :polynomially_longer, attempts: 3 # Don't retry if video not found discard_on ActiveRecord::RecordNotFound do |job, error| Rails.logger.warn "Video not found for processing: #{error.message}" end def perform(video_id) video = Video.find(video_id) # Extract metadata result = VideoMetadataExtractor.new(video).call return unless result.success? # Generate thumbnail (Phase 1C) ThumbnailGeneratorService.new(video).call rescue FFMPEG::Error => e video.mark_processing_failed!(e.message) raise # Will retry end end ``` ### Background Job Monitoring ```ruby # app/controllers/admin/jobs_controller.rb (optional) module Admin class JobsController < Admin::BaseController def index @running_jobs = SolidQueue::Job.where(finished_at: nil).limit(50) @failed_jobs = SolidQueue::Job.where.not(error: nil).limit(50) end def retry job = SolidQueue::Job.find(params[:id]) job.retry! redirect_to admin_jobs_path, notice: "Job queued for retry" end end end ``` --- ## Configuration Summary ### Environment Variables ```bash # .env (Phase 1) DEFAULT_SCAN_PATH=/path/to/your/videos FFMPEG_THREADS=4 THUMBNAIL_SIZE=1920x1080 # .env (Phase 2) ADMIN_EMAIL=admin@example.com OIDC_ISSUER=https://auth.example.com OIDC_CLIENT_ID=velour OIDC_CLIENT_SECRET=your-secret # .env (Phase 3) AWS_REGION=us-east-1 AWS_ACCESS_KEY_ID=your-key AWS_SECRET_ACCESS_KEY=your-secret AWS_S3_BUCKET=velour-videos # .env (Phase 4) VELOUR_API_KEY=your-api-key ALLOW_FEDERATION=true ``` ### Database Encryption ```bash # Generate encryption keys bin/rails db:encryption:init # Add output to config/credentials.yml.enc: # active_record_encryption: # primary_key: ... # deterministic_key: ... # key_derivation_salt: ... ``` --- ## Implementation Checklist ### Phase 1A: Core Foundation - [ ] Install required gems (`streamio-ffmpeg`, `pagy`) - [ ] Generate models with proper migrations - [ ] Implement model validations and associations - [ ] Create model concerns (Streamable, Processable, Searchable) - [ ] Build storage adapter pattern (BaseAdapter, LocalAdapter) - [ ] Implement FileScannerService - [ ] Create Result object pattern - [ ] Build videos index page with TailwindCSS - [ ] Create admin storage locations CRUD - [ ] Write tests for models and adapters ### Phase 1B: Video Playback - [ ] Create videos#show action with byte-range support - [ ] Add Video.js via Importmap - [ ] Build VideoPlayerController (Stimulus) - [ ] Implement PlaybackSession.update_position - [ ] Create API endpoint for position tracking - [ ] Build custom Video.js tracking plugin - [ ] Implement resume functionality - [ ] Write system tests for playback ### Phase 1C: Processing Pipeline - [ ] Implement VideoMetadataExtractor service - [ ] Create VideoProcessorJob with retry logic - [ ] Build ThumbnailGeneratorService - [ ] Set up Active Storage for thumbnails - [ ] Display thumbnails on index page - [ ] Add processing status indicators - [ ] Implement retry processing action - [ ] Write tests for processing services ### Phase 1D: Works & Grouping - [ ] Fully implement Works model - [ ] Create Works index/show pages - [ ] Implement DuplicateDetectorService - [ ] Build WorkGrouperService - [ ] Create work grouping UI - [ ] Add version selector on work pages - [ ] Implement search functionality - [ ] Add filtering and sorting - [ ] Integrate Pagy pagination - [ ] Polish UI with TailwindCSS --- ## Next Steps ### To Begin Implementation: 1. **Review this architecture document** with team/stakeholders 2. **Set up development environment:** - Install FFmpeg (`brew install ffmpeg` on macOS) - Verify Rails 8.1.1+ installed - Create new Rails app (already done in `/Users/dkam/Development/velour`) 3. **Start Phase 1A:** ```bash # Add gems bundle add streamio-ffmpeg pagy aws-sdk-s3 # Generate models rails generate model Work title:string year:integer director:string description:text rating:decimal organized:boolean poster_path:string backdrop_path:string metadata:text rails generate model StorageLocation name:string path:string location_type:integer writable:boolean enabled:boolean scan_subdirectories:boolean priority:integer settings:text last_scanned_at:datetime rails generate model Video work:references storage_location:references title:string file_path:string file_hash:string file_size:bigint duration:float width:integer height:integer resolution_label:string video_codec:string audio_codec:string bit_rate:integer frame_rate:float format:string has_subtitles:boolean version_type:string source_type:integer source_url:string imported:boolean processing_failed:boolean error_message:text metadata:text rails generate model VideoAsset video:references asset_type:integer metadata:text rails generate model PlaybackSession video:references user:references position:float duration_watched:float last_watched_at:datetime completed:boolean play_count:integer # Run migrations rails db:migrate # Start server bin/dev ``` 4. **Follow the Phase 1A checklist** (see above) 5. **Iterate through Phase 1B, 1C, 1D** --- ## Architecture Decision Records Key architectural decisions made: 1. **SQLite for MVP** - Simple, file-based, perfect for single-user. Migration path to PostgreSQL documented. 2. **Storage Adapter Pattern** - Pluggable backends allow adding S3, JellyFin, etc. without changing core logic. 3. **Service Objects** - Complex business logic extracted from controllers/models for testability. 4. **Hotwire over React** - Server-rendered HTML with Turbo Streams for real-time updates. Less JS complexity. 5. **Video.js** - Proven, extensible, well-documented player with broad format support. 6. **Rails Enums (integers)** - SQLite-compatible, performant, database-friendly. 7. **Active Storage for assets only** - Videos managed by storage adapters, not Active Storage. 8. **Pagy over Kaminari** - Faster, simpler pagination with smaller footprint. 9. **Model-level authorization** - Simple for MVP, easy upgrade path to Pundit/Action Policy. 10. **Phase 1 broken into 4 sub-phases** - Manageable milestones with clear deliverables. --- ## Support & Resources - **Rails Guides:** https://guides.rubyonrails.org - **Hotwire Docs:** https://hotwired.dev - **Video.js Docs:** https://docs.videojs.com - **FFmpeg Docs:** https://ffmpeg.org/documentation.html - **TailwindCSS:** https://tailwindcss.com/docs --- **Document Version:** 1.0 **Last Updated:** <%= Time.current.strftime("%Y-%m-%d") %> **Status:** Ready for Phase 1A Implementation