Wayne's Image/Video Encoding

发表于 更新于 分类于 本文字数： 4.4k 阅读时长 ≈ 8 分钟

What is Webp?

Webp是一种图像格式，支持单张图片，动图；
什么平台支持？
如何使用？

Open-Source projects

libwebp: https://chromium.googlesource.com/webm/libwebp
mirror: https://github.com/webmproject/libwebp/

User Guide

Options

  1. q: 指定quality factor
  2. alpha_q: 指定透明通道的压缩率
  3. preset: 指定图片的type
  4. -z: 指定lossless压缩的效率
  5. m: 压缩的速度档次，0最快，6是最慢
  6. segment: 指定要使用的segments，默认是4
  7. -size: 指定目标size (in bytes)
  8. -psnr: 指定目标的PSNR
  9. -sns: spatial noise shaping, important
  10. -f: filter strength, default 60，是指loop filter的强度
  11. -sharpness: filter sharpness, 0: most sharp, 7: 最小sharp
  12. -strong: use strong filter
  13. -sharp_yuv: use sharper RGB->YUV的转换
  14. -partition_limit
  15. -pass: analysis pass number
  16. -qrange，指定最小和最大QP的范围
  17. -mt: 使用multi-threading
  18. -alpha_method: 指定透明通道压缩的方法
  19. -alpha_filter: predictive filtering for alpha plane
  20. -exact: preserve RGB value in transparent area; 也就是不修改完全透明区域的
  21. -noalpha: 丢掉alpha的特征
  22. -lossless: 无损压缩图像
  23. -near_lossless: use near-lossless image preprocessing
  24. -hint: 指定图片的特征或者提示；
  25. -af: using autoaf，指标是ssim

如何把其他图片转换成Webp?

JPG format
YUVJ420: YUV420P with JPEG color-range” (i.e. pixels use 0-255 range instead of 16-235 range, where 0 instead of 16 is black, and 255 instead of 235 is white);

使用工具cwebp，最简单的使用就是:
cwebp input.png -q 80 -o output.webp

重要的参数

重要的参数包括-preset -sns -f -m:
-preset: 用于指定content的类型，应该最先被设置，这样后续的设置就在这个基础上来修改; 编码器内部会根据preset去调整sns_strength, filter_sharpness, filter_strength和preprocessing.
-f: 对应的是编码器内部的in-loop filter的filter strength;
-m: 用来指定编码速度，[0, 6]，0最快，6最慢，default是4
-sns: 用来打开一些visual optimization，主要用来调节bit allocation. 用来把easy part的bit分配给hard part。通常升高sns，会导致文件增大，质量变好；

sns

method >= 4: distortion会包括spectural distortion
Distortion: 在method > 4的时候，会有distortion和spectral distortion
设置segment的时候，可以控制幅度。
控制uvac，越小的话，dq_uv_ac变大，和dq_uv_dc，越大,uv_dc boost的越多
It will set do_search

how to calculate alpha:
Do transform, collect coefficients stats, find last_none_zero and max_values;
alpha = last_non_zero/max_value

  1. 计算每个mb的alpha
    原理：
    TX: 4x4
    convert coefficients to bins，calculate the histogram of abs(level)
    maxvalue: 出现系数最多的系数的出现次数，越大，说明0系数的越多，说明能量越少
    last_non_zero: 最后一个非零系数，越大，说明高频的能量越多，
    alpha = ALPHA_SCALE * last_non    +zero / max_value;
    best_alpha = MAX_ALPHA - (3 * best_alpha + best_uv_alpha + 2) >> 2
    高能量:
    1
    2
    3
    4
    5
    MBAnalyze:323 analyze mbx 0, mby 1
    mode 0, last_nonzero_count 31, max_value 95
    mode 1, last_nonzero_count 31, max_value 88
    MBAnalyze:330 best_alpha 179
    MBAnalyze:341 mbx 0, mby 1, alpha 57, uv_alpha 255
    低能量：
    1
    2
    3
    4
    5
    MBAnalyze:323 analyze mbx 0, mby 3
    mode 0, last_nonzero_count 11, max_value 234
    mode 1, last_nonzero_count 4, max_value 244
    MBAnalyze:330 best_alpha 23
    MBAnalyze:341 mbx 0, mby 3, alpha 234, uv_alpha 16
    alpha越大，Q越小；
    beta

filter params

Decoder perspective:
9.4 Loop Filter Type and Levels
frame-level loop filter params:

  1. filter_type: normal or simple, simple filter only applied to luma edges;
  2. loop_filter_level: defines the threshold, if the different is below this threshold, it should be filtered, otherwise, it should not be changed. Usually the level correlated to quantizer levels.
  3. sharpness_level, constant over the frame,
    如果loop_filter_level=0，那么应该跳过loop_filter;
    Differences in excess of a threshold(associated to the loop_filter_level) are assumed to not being modified.

mode_ref_lf_delta_update=1, allow per-macroblock adjustment, through delta; There are two types of delta values, one for reference frame-based adjustment, and the other group is for mode-based adjustment.

Filter header to write into bitstreams:
simple:
level: 对应fstrength_
sharpness

sharpness设置越大，fstrength越大；
Q越大，fstrength越大；

fstrength是针对segment来的；

lf_delta

process;
basestrength = F(q, sharpness)
strength = 5 filter_strength base_strength / *(256 + beta)
level0 = fstrength_

只filter 4x4的块，不filter 16x16和skip的块；

How to decide the strength for each segments?

Why VP8StoreFilterStats? 存储不同的deltaq下，do filter之后的ssim并存储起来；
How to decide final strength;
VP8AdjustFilterStrength

Internal Process

Speed 0/1

Speed 2~6

Alpha Encoding

主要通过一下几个接口来处理

  1. VP8EncInitAlpha
  2. VP8EncStartAlpha
  3. VP8EncFinishAlpha
  4. VP8EncDeleteAlpha
graph TD;
    A-->B;
    A-->C;
    B-->D;
    C-->D;

主要调用EncodeAlpha进行处理

Feature

Specify target_size/target_PSNR

sns:

Region-based quantization

Adaptive Loop-filter

根据prediction mode或参考帧来决定loop filter strength
Adjustment of loop-filter stregnth for each segment.

RDO

Preprocessing

sharp YUV

This is applied when converting RGB to YUV

Smooth segment

pseudo-random dithering log2

Alpha cleanup

Lossless

主要通过调用WebPEncodeLosslessRGB，内部是调用EncodeLossless.
VP8LEncodeImage is used to encoding lossless.
内部调用VP8LEncodeStream
QuantizeLevels
VP8FiltersInit
ApplyFiltersAndEncode
EncodeAlphaInternal —-> EncodeLossless —-> VP8LEncodeStream

Segment

不打开Segment情况下，

rd_opt_level:

  1. no_rd
  2. rd_opt_basic
  3. rd_opt_trellis, perform trellis-quant on the final decision only
  4. rd_opt_trellis_all

if rd_opt_level < rd_opt_basic>, then use VP8EncLoop

Reference

Official-Webp-Doc
React Native using Webp
Webp Tools
VP8 Encode Parameter Guide
VP8 Tech Overview
Webp Compression Techniques

发表于 更新于 分类于 本文字数： 2.4k 阅读时长 ≈ 4 分钟

nni是微软的一个AutoML工具，可以用来做一个自动化特征选择，自动剪枝和量化，神经网络架构搜索以及超参数调优。本文重点介绍下使用nni做超参数调优部分

支持的算法

nni中包含了好多种超参数调优算法，大概有三类搜索策略，

Exhaustive seach

  1. Grid Search
  2. Random
  1. Anneal
  2. Evolution
  3. Hyperband
  4. PBT

Bayesian optimization

  1. BOHB
  2. DNGO
  3. GP
  4. Metis
  5. SMAC
  6. TPE

使用方式

使用nni需要准备三个文件，包括

  1. nni_search.py: 计算每个参数的score
1
2
3
4
5
6
7
8
9
10
11
import nni

def main():
    para = nni.get_next_parameter()
    x = para['x']
    # square(x) = 2
    score = abs(x*x - 2)
    nni.report_final_result(score)

if __name__ == '__main__':
    main()

  1. search_space.json: 定义参数搜索的空间

    1
    2
    3
    4
    {
            "x": {"_type": "uniform", "_value": [1, 2]},
    }

  2. config.yml: nni配置文件

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    experimentName: search_sqrt_2

    # 并行度
    trialConcurrency: 1

    # 最大允许时长,
    maxExecDuration: 1h
    # 最多的运行次数，如果不设置则意味着永远不会停止
    maxTrialNum: 10

    #choice: local, remote
    trainingServicePlatform: local

    # search_space file
    searchSpacePath: search_space.json

    #choice: true, false
    useAnnotation: false

    tuner:
      #choice: TPE, Random, Anneal, Evolution, BatchTuner, MetisTuner
      #SMAC (SMAC should be installed through nnictl)
      builtinTunerName: TPE
      classArgs:
        #choice: maximize, minimize
        optimize_mode: minimize

    trial:
      command: python3 nni_search.py
      codeDir: .
      gpuNum: 0

运行命令:
nnictl create --config config.yml 启动试验，nni会打印出具体配置，并启动一个webserver，可以检查试验结果，以下为上面例子启动后的输出:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[2023-10-06 21:51:13] WARNING: Config field "authorName" is no longer supported and has been ignored
[2023-10-06 21:51:13] WARNING: You are using legacy config file, please update it to latest format:
================================================================================
experimentName: search_sqrt_2
trialConcurrency: 1
maxExperimentDuration: 1h
maxTrialNumber: 100
searchSpaceFile: search_space.json
useAnnotation: false
trialCommand: python3 nni_search.py
trialCodeDirectory: .
trialGpuNumber: 0
tuner:
  name: TPE
  classArgs:
    optimize_mode: minimize
trainingService:
  platform: local
================================================================================
Reference: https://nni.readthedocs.io/en/stable/reference/experiment_config.html
[2023-10-06 21:51:13] Creating experiment, Experiment ID: vl4y6ge3
[2023-10-06 21:51:13] Starting web server...
[2023-10-06 21:51:14] Setting up...
[2023-10-06 21:51:14] Web portal URLs: http://127.0.0.1:8080 http://192.168.50.103:8080
[2023-10-06 21:51:14] To stop experiment run "nnictl stop vl4y6ge3" or "nnictl stop --all"
[2023-10-06 21:51:14] Reference: https://nni.readthedocs.io/en/stable/reference/nnictl.html

运行完成以后，通过
nnictl stop -all来停止试验

发表于 更新于 分类于 本文字数： 367 阅读时长 ≈ 1 分钟

Welcome to Hexo! This is your very first post. Check documentation for more info. If you get any problems when using Hexo, you can find the answer in troubleshooting or you can ask me on GitHub.

Quick Start

Create a new post

1
$ hexo new "My New Post"

More info: Writing

Run server

1
$ hexo server

More info: Server

Generate static files

1
$ hexo generate

More info: Generating

Deploy to remote sites

1
$ hexo deploy

More info: Deployment

0%