3d Generative modeling

From spa

Perhaps you have heard model VQ-VAE and VQ Gan that use quantization.

Intuit

Thanks to takl Stéphane Mallat - Multiscale Models for Image Classification and Physics with Deep Networks. https://www.youtube.com/watch?v=R5hSqeLSQC0&t=2820s

Sparsity

Dictionary learning.

There are cases where wavelets or Fourier components are not good enough for restoration. In most cases we are aimed to build with as little parameters as possilbe.

Often that happens due to the case that in specefic situation better basis can be introduced than general. Dictionary matching is way of introducing better basis, that comes from data

Sparse learning

\[\|x_i - \mathbf{D} r_i \|_2^2 + \lambda \| r\|\]

Matching pursuit

Is quite simple algorithm that represent as linear sum

\[D = X R^+\]

$+$ here means Moore-Penrose pseudoinverse, which comes due to fact that dictonary can contain not orthogonal. So that determinant can be equal to zero, we need something more robust.

SGD

\[\]

Lee, Honglak, et al. “Efficient sparse coding algorithms.” Advances in neural information processing systems. 2006.

Ugly

Sparse_dictionary_learning