[3.0] add AMF VQ Enhancer video filter

Only support with D3D11, especially as on 3.0 the decoders are tied to the picture display pool.

The code is slightly different from 4.0 for the output picture usage and Direct3D11 device creation.

Draft as as long as !5469 (merged) is not merged.