Fix skybox under orthographic view

[maz_1]

I am testing SkyPostprocessing in a object-centered control style. It works totally fine under perspective view, the challenge is to make it behave normal under orthographic view.
Under perspective view , when I want to zoom in/zoom out on a object, I only need to move the camera towards or away from the object.
But under orthographic view, moving the camera won't change the rendered size of the object, so I have to change the extents of the frustums to make rendered size bigger or smaller. The problem is that changing the frustum extents will change how skybox is rendered, making it distorted.
Under perspective view, I can easily understand how the fov of camera is related to the visible area of skybox. But under orthographic view, all camera rays are parallel to each other, so I'm not sure how skybox work under this situation.
Could I possibly modify the SkyPostprocess vertex shader so the projection matrix always has a fixed size of frustum?

[maz_1]

When I zoom out (increase frustum size)
https://imgur.com/a/8GmQGBv

When I zoom in (decrease frustum size)
https://imgur.com/a/BdZtypc