Should just loosen.   

Maybe someone has used loctite on it . 

Have you tried using an allen key?  It might let you apply a stronger turning force on the screw than a screwdriver or even drill based tool.
Use a ring spanner or similar as a lever on the allen key if neccessary.

Good luck