Yes, absolutely essential to lift one end of the fuse out of the holder to test properly.

Imagine this circuit;  A transformer feeds a GI globe through a fuse - very standard.  The fuse blows.  Now stick your meter across the fuse whilst still in the holder, ie the fuse is still in the circuit. What you are now actually measuring is the continuity through the transformer winding and globe - likely to be several ohms, yet the fuse is blown or open circuit. You are not measuring what you think or hope you are!