Anyway, no need to remove anything. You just need to align your seams and fit your curves. (would not have to do any of this if you had drawn the curves correctly, starting from same location, and fitted).
I move the seam to a corner rather than in the middle of a length as you had, then I make this corner the seam of both lines. Then I fit the curves to degree 3 to fix the fillet arc (fillet arcs are degree 2 and dont really play nice with lofts when mixed in the same polyline curve as line or curve segments). Fit just gives the arc section much more control points as a nurbs curve.
It is important to remember that the way we draw things matter. Operations like loft are quite literal interpolations which have some automatic alignment stuff but they are not really perfect. Always better to construct the curve in the way you want it to loft (line things up between curves like seams, directions, control points)