Shifting tower side to side

Made a tower with some of the levels that shift sideways (using “closest point”). When it creates another shift, the bottom part goes back to the centrepoint (that shift part is flat). Is it possible to lift that part too?

gap between (78.5 KB)

The problems are more evident here…