A short follow up regarding my previous post. Michael Hoffman is actually making a library called Selene capable of nailing down quite many items from my list. No .RGBE and .EXR support so far (both are vital for me), but pretty solid otherwise.
Give it a try: https://github.com/kmhofmann/selene