I think you first rule is gonna conflict with the second because, the 1st rectangle in the corner is a special case, also you need a way keeping track of every group of rectangles, I would personally use the union polygon for this(So that you dont need to keep track of internal edges). And you need some placement rule, you said that every rectangle needs to have at least two neighbors. Normally when you do nesting algorithms you need a placement criteria, so that you check that criteria every time you place a new geometry. In Polygon Nesting that Criteria is a measurement of the convexity, and wat you do is you calculate that criteria all around group polygon, to see which new polygon gives more a convex result. In this case because you want random rectangles you need a way measuring randomness or something like that, so that you check which of the neighbors is more random and select that one. You can also just select randomly out all the neighbors, but I have the suspicion that this method is not gonna give you the expected result. Because after the first rectangle the grow is gonna be bias towards one side and well that is not really gonna look random.

Also using only the 2 neighbors criteria is not gonna work(grow) on corners. because when there is 4 rectangles together in grid like in this image, all the rectangles satisfy the criteria so you cant really place any other around them, look at the image:

Im wondering wat the expected result should look like in a bigger grid with more loops, because the 2 example drawing you upload, they maybe have like 2 or 3 loops, and you need a bigger case so that we can understand wat the expected result is. Maybe just run your algorithm by hand on a drawing, maybe for 5 to 8 loops to see how it should look like. The reason why Im telling you this is because based on your rules you could have an algorithm that endup looking like is growing radially(this result would look just like voronoi on a grid) instead of linearly like nesting