demux: mp4: handle failed allocation as error

As proposed in !4137 (comment 407083)

It returns ENOMEM and clean the format. I've added failure everywhere the only exported functions are

int SetupVideoES( demux_t *p_demux, const mp4_track_t *p_track,
                  const MP4_Box_t *p_sample, es_format_t *, track_config_t *);
int SetupAudioES( demux_t *p_demux, const mp4_track_t *p_track,
                  const MP4_Box_t *p_sample, es_format_t *, track_config_t * );
int SetupSpuES( demux_t *p_demux, const mp4_track_t *p_track,
                const MP4_Box_t *p_sample, es_format_t *, track_config_t * );

They are used only in TrackFillConfig() which will fail if they return 0

It is used in TrackCreateES() that will fail if TrackFillConfig fails thus not putting the track b_ok.

And TrackUpdateFormat() that will now fail if TrackFillConfig fails too but that will require François' validation.