Consider removing some of the UNet connections

[JoshVarty]

If we want out network to learn and compress information about the scene, we should consider removing some of the cross connections in our UNet.

For example, the long gray "copy and crop" lines that go from early layers to later layers.

[JoshVarty]

Blurriness in the outputs in the pretext task probably isn't something you need to fix. Indeed - you may want to actually make the architecture here simpler, and get a worse pretext task result. Specifically - using a unet may be less good that just using a plain upsampling head

That would force nearly all the computation into the downstream part of the architecture, which is what you want - that's the bit you transfer learn with