Doom9's Forum - View Single Post - Auto Target Encoder

Leo 69 · 19th August 2025, 23:53

Here is a stage-by-stage description of what it does:

Stage 1: Initialization and Learning
When the application starts, it first loads all user settings from a config.ini file. It then initializes its Machine Learning (ML) models by training them on historical data stored in a local SQLite database. This allows the app to learn from previous encodes to make smarter predictions about encoding speed and quality settings for new videos.
Stage 2: File Queuing and Pre-Filtering
The user selects a folder of videos to process. The script scans this folder and populates a queue. Before any heavy processing begins, it performs a fast pre-filter, automatically skipping files that are too short, too small, or have a bitrate lower than user-defined thresholds.
Stage 3: Video Analysis
For each valid file, the application performs a detailed analysis. It uses ffprobe to get technical details (resolution, frame rate, etc.) and performs a complexity analysis to understand the video's content (e.g., detecting scene changes). This data is compiled into a set of "features" that the ML models can understand.
Stage 4: ML-Driven Quality Search
This is the core of the application. To find the perfect quality setting (CQ/CRF value) that meets the user's target (e.g., a VMAF score of 95):
It first creates a short, high-quality "master sample" by stitching together representative clips from the video.
It uses its trained Quality Model to predict the best CQ value needed to hit the target score.
Based on the model's confidence, it intelligently tests one or two CQ values by encoding only the small sample file, which is extremely fast.
If the prediction is wrong or the model is not confident, it falls back to an efficient search to find the optimal CQ value. All test results are cached in the database to avoid re-doing work.
Stage 5: Final Encoding
Once the optimal CQ value has been found, the script proceeds to encode the full-length original video using that setting. It monitors the encoding process in real-time to provide progress updates and detect if the process has stalled.
Stage 6: Finalization, Logging, and Learning
After the encode is complete, the script verifies the new file. If it meets the criteria (e.g., sufficient size reduction), it saves the final file and can optionally delete the original. Critically, it logs the performance data (how long it took, the final file size, etc.) back into the SQLite database. This act of logging completes the feedback loop, ensuring that the ML models become more accurate with every video it processes.

19th August 2025, 23:53	#10 \| Link
Leo 69 Registered User Join Date: Nov 2004 Posts: 268	Here is a stage-by-stage description of what it does: Stage 1: Initialization and Learning When the application starts, it first loads all user settings from a config.ini file. It then initializes its Machine Learning (ML) models by training them on historical data stored in a local SQLite database. This allows the app to learn from previous encodes to make smarter predictions about encoding speed and quality settings for new videos. Stage 2: File Queuing and Pre-Filtering The user selects a folder of videos to process. The script scans this folder and populates a queue. Before any heavy processing begins, it performs a fast pre-filter, automatically skipping files that are too short, too small, or have a bitrate lower than user-defined thresholds. Stage 3: Video Analysis For each valid file, the application performs a detailed analysis. It uses ffprobe to get technical details (resolution, frame rate, etc.) and performs a complexity analysis to understand the video's content (e.g., detecting scene changes). This data is compiled into a set of "features" that the ML models can understand. Stage 4: ML-Driven Quality Search This is the core of the application. To find the perfect quality setting (CQ/CRF value) that meets the user's target (e.g., a VMAF score of 95): It first creates a short, high-quality "master sample" by stitching together representative clips from the video. It uses its trained Quality Model to predict the best CQ value needed to hit the target score. Based on the model's confidence, it intelligently tests one or two CQ values by encoding only the small sample file, which is extremely fast. If the prediction is wrong or the model is not confident, it falls back to an efficient search to find the optimal CQ value. All test results are cached in the database to avoid re-doing work. Stage 5: Final Encoding Once the optimal CQ value has been found, the script proceeds to encode the full-length original video using that setting. It monitors the encoding process in real-time to provide progress updates and detect if the process has stalled. Stage 6: Finalization, Logging, and Learning After the encode is complete, the script verifies the new file. If it meets the criteria (e.g., sufficient size reduction), it saves the final file and can optionally delete the original. Critically, it logs the performance data (how long it took, the final file size, etc.) back into the SQLite database. This act of logging completes the feedback loop, ensuring that the ML models become more accurate with every video it processes.