shaders/icc: fix LUT signature check

This was missing checks for `max_luma` in both `pl_icc_encode` and
`pl_icc_decode`, as well as a check for `intent` in `pl_icc_encode`.

Fix this and make it more robust by calculating the LUT signature ahead
of time. Also include the 3DLUT size in the resulting calculation, to
pave the way for the 3DLUT caching API in the next commit.